Heavy release of Clusternet v0.5.0: Comprehensively solve the differential configuration problem of multi-cluster application distribution

author

Xu Di, Tencent Cloud Container Technology Expert.
Ru Yingzhe, Senior Product Manager of Tencent Cloud.

Summary

When doing multi-cluster application distribution, the following differentiation problems are often encountered, such as:

1. apps.my.company/deployed-by: my-platform uniform label on all distributed resources, such as 06176873820d81;
2. Mark the cluster information on the resources distributed to the sub-cluster, such as apps.my.company/running-in: cluster-01 ;
3. Adjust the number of copies, mirror names, etc. of the application in each cluster. For example, there is a Deployment my-nginx (the number of copies declared is 3) to be distributed to clusters cluster-01, cluster-02, cluster-03 In, I hope that the number of replicas in these three clusters is 3, 5, and 7 respectively;
4. Before distributing to the cluster cluster-01, adjust some configurations applied in the cluster, such as injecting a Sidecar container, etc.;
5. When encountering some special scenarios, such as big promotion, dynamic expansion, and application gray scale upgrade, it is hoped that the operation can be performed on a certain cluster, and the scope of the change is small, without affecting other clusters, and when problems occur at the same time, you can roll back in time To restore to the state before the change;
6. If multiple differentiated configurations are defined and conflicts arise between them, how to resolve them;

Introduction to the open source Clusternet project

Clusternet ( Cluster Inter net ) is Tencent Cloud's open source cloud native management and control project that combines multi-cluster management and cross-cluster application orchestration, making using multiple clusters as easy as surfing the Internet. Whether your Kubernetes cluster is running on a public cloud, private cloud, hybrid cloud or edge cloud, you have a consistent management/access experience, using K8s API to centrally deploy and coordinate multi-cluster applications and services.

Clusternet adopts Addon plug-in method, which is convenient for users to install, operate and integrate, and easily manage millions of Kubernetes clusters, making cloud computing ubiquitous, free and convenient like the Internet.

Clusternet supports the distribution and management of various application resources to different clusters, including various native Kubernetes resources (Deployment/StatefulSet/ConfigMap/Secret, etc.), various CRD resources, and HelmChart applications, etc.

How Clusternet solves these differentiated configuration problems

When Clusternet is designing the application distribution model, it fully considers the above-mentioned scenarios. It does not want to introduce too many complex designs, minimize the repeated definition of users, and achieve simplification, easy configuration, strong scalability, and easy change back. Go ahead and wait.

If we summarize the above-mentioned differentiation problems, they can be roughly classified into the following two categories:

1. Generalized configuration or global configuration, such as undifferentiated tagging, pre-configuration, etc. for certain resources;
2. The configuration specific to a cluster, such as changing the Deployment in a cluster, upgrading mirroring, adding Sidecar containers, etc.;

The figure below is the clusternet's multi-cluster application distribution model. The green modules need to be created by users, and the purple modules are resource objects that are circulated within Clusternet. Clusternet provides the kubectl plug-in, which can be used to create resources through the "kubectl clusternet apply" command. Welcome to Clusternet-a new generation of open source multi-cluster management and application governance project to understand the related concepts in the figure.

The Clusternet resource distribution model adopts a loosely coupled design. Users do not need to change or rewrite existing resource objects. They only need to define additional distribution strategies ( Subscription ) and differentiated configurations ( Localization / Globalization ) to realize multi-cluster application distribution.

Localization and Globalization

In Clusternet, each registered cluster will have a dedicated namespace (namespace), so we have defined the Localization and Globalization to declare differentiated configurations. Among them, Localization describes the differentiated configuration strategy of namespace-scoped (namespace scope), which can be used to configure a single cluster, such as the number of copies of Deployment And Globalization describes the differentiated configuration strategy of cluster-scoped (cluster scope), such as modifying the general configuration of HelmChart

Override strategy

Clusternet also provides two Overide strategies: ApplyLater (the default strategy) and ApplyNow . ApplyLater means that the Localization / Globalization differentiated configuration are not applied immediately to the resource, and then only in the newly created out of Description objects or HelmChart / Subscription / Description take effect when the updated object and other resources. And ApplyNow means that it will take effect immediately after creation, and Clusternet will apply the defined differentiated configuration to all matching objects and immediately deliver it to the corresponding sub-cluster.

Priority

In addition, both support Priority (priority). The priority is defined by the value of 0-1000. The smaller the value, the lower the priority. The default is 500. When performing differentiated rendering, Clusternet will follow the order of Globalization (low priority) -> Globalization (high priority) -> Localization (low priority) -> Localization (high priority), and proceed with the declared Override in turn apply.

It is by means of this two stages based on priority ( TWO-Stage priority based ) differentiated configuration capability, Clusternet can easily support for multi-cluster of blue-green publishing, canary release, version upgrades Scenes. During use, you can define multiple Globalization and Localization objects and set different priority strategies.

Support Patch operation

Clusternet supports two formats of Override, JSON Patch ( RFC 6902 ) and JSON Merge Patch ( RFC 7396 ). For the comparison of JSON patch and JSON merge patch, you can check JSON Patch and JSON Merge Patch , or you can refer to the following typical examples.

Typical example

Let's look at a few typical differentiated configuration scenarios below. In the following example, we use the Localization object to display uniformly. Globalization is also possible to use 061768738210e6 here. The Spec definitions of the two are the same, the only difference is the scope and priority of the two. When you actually use it, you can rewrite it as needed.

Add/update label

If we want to add or update a label to an object, we can define the following Localization object. When using it, please metadata.namespace the value of 0617687382110e with the exclusive namespace of the real registered cluster.

apiVersion: apps.clusternet.io/v1alpha1
kind: Localization
metadata:
  name: nginx-local-overrides-demo-label
  namespace: clusternet-5l82l # 请更新这个值为对应集群的 namespace
spec:
  overridePolicy: ApplyLater
  # 优先级反映着该对象的重要性，数值范围从 0 到 1000，值越小表示优先级越低
  # 默认的值为 500.
  priority: 300
  feed: # 这里表示要 override 的对象
    apiVersion: apps/v1
    kind: Deployment
    name: my-nginx
    namespace: foo
  overrides: # 这里可以定义着多个 override
    - name: add-update-labels
      type: MergePatch # 这里需要指定 override 的类型
      # value 可以是 yaml 格式，也可以是 json 格式。
      # 如下是 json 格式的例子
      value: '{"metadata":{"labels":{"deployed-in-cluster":"clusternet-5l82l"}}}'

You can Localization object. In the above example, we only define an add-update-labels , whose value is a string in json format. The purpose is to add or update a tag defined by deployed-in-cluster: clusternet-5l82l to spec.feed Object.

The override value here can also be in yaml format, as shown in the following example.

apiVersion: apps.clusternet.io/v1alpha1
kind: Localization
metadata:
  name: nginx-local-overrides-demo-label
  namespace: clusternet-5l82l # 请更新这个值为对应集群的 namespace
spec:
  overridePolicy: ApplyLater
  # 优先级反映着该对象的重要性，数值范围从 0 到 1000，值越小表示优先级越低
  # 默认的值为 500.
  priority: 300
  feed: # 这里表示要 override 的对象
    apiVersion: apps/v1
    kind: Deployment
    name: my-nginx
    namespace: foo
  overrides: # 这里定义着 override value
    - name: add-update-labels
      type: MergePatch
      # value 可以是 yaml 格式，也可以是 json 格式。
      # 如下是 yaml 格式的例子
      value: |-
        metadata:
          labels:
            deployed-in-cluster: clusternet-5l82l

Number of replacement mirrors and copies

The type of Override can also be specified as JSONPatch . In actual use, you can choose a suitable override type according to your needs.

Through the following example, you can change the number of copies of foo/my-nginx in the clusternet-5l82l 3 , the mirror of the replacement container is nginx:1.14.0-alpine , and add a new comment foo: bar .

apiVersion: apps.clusternet.io/v1alpha1
kind: Localization
metadata:
  name: nginx-local-overrides-demo-image-replicas
  namespace: clusternet-5l82l # 请更新这个值为对应集群的 namespace
spec:
  overridePolicy: ApplyLater
  # 优先级反映着该对象的重要性，数值范围从 0 到 1000，值越小表示优先级越低
  # 默认的值为 500.
  priority: 400
  feed: # 这里表示要 override 的对象
    apiVersion: apps/v1
    kind: Deployment
    name: my-nginx
    namespace: foo
  overrides: # 这里定义着 override value
    - name: scale-and-add-annotations
      type: JSONPatch
      # value 可以是 yaml 格式，也可以是 json 格式。
      value: |-
        - path: /spec/replicas
          value: 3
          op: replace
        - path: "/spec/template/spec/containers/0/image"
          value: "nginx:1.14.0-alpine"
          op: replace
        - path: /metadata/annotations
          value:
            foo: bar
          op: add

Inject the Sidecar container

We can also use Localization to inject the Sidecar container for the instance of Deployment foo/my-nginx under the clusternet-5l82l

apiVersion: apps.clusternet.io/v1alpha1
kind: Localization
metadata:
  name: nginx-local-overrides-demo-sidecar
  namespace: clusternet-5l82l # 请更新这个值为对应集群的 namespace
spec:
  overridePolicy: ApplyLater
  # 优先级反映着该对象的重要性，数值范围从 0 到 1000，值越小表示优先级越低
  # 默认的值为 500.
  priority: 600
  feed: # 这里表示要 override 的对象
    apiVersion: apps/v1
    kind: Deployment
    name: my-nginx
    namespace: foo
  overrides: # 这里定义着 override value
    - name: inject-new-container
      type: JSONPatch
      # value 可以是 yaml 格式，也可以是 json 格式。
      value: |-
        - op: add
          path: "/spec/template/spec/containers/1"
          value:
            name: "redis-container"
            image: "redis:6.2.5"

Through Localization and Globalization not only can the above differentiated configuration be done, there are more scenarios waiting for everyone to explore.

In order to make it easy for everyone to get started, Clusternet provides the example You can refer to the step README to practice multi-cluster application distribution.

join us

When the Clusternet project is open source, please follow https://github.com/clusternet/clusternet Like support, welcome to join us and contribute more features.

Related Links

[1] https://github.com/clusternet/clusternet

[2] https://github.com/clusternet/kubectl-clusternet

[3] https://krew.sigs.k8s.io/plugins/

about us

For more cases and knowledge about cloud native, please follow the public account of the same name [Tencent Cloud Native]~

Welfare:

①公众号后台回复【手册】，可获得《腾讯云原生路线图手册》&《腾讯云原生最佳实践》~

②公众号后台回复【系列】，可获得《15个系列100+篇超实用云原生原创干货合集》，包含Kubernetes 降本增效、K8s 性能优化实践、最佳实践等系列。

[Tencent Cloud Native] Yunshuo new products, Yunyan new technology, Yunyou Xinhuo, Yunxiang information, scan the QR code to follow the public account of the same name, and get more dry goods in time! !