KubeVela v1.3 multi-cluster first experience, easy to manage application distribution and differentiated configuration

Author: Duan Wei (Duan Shao)

In today's multi-cluster business scenarios, we often encounter requirements such as: distribution to multiple designated clusters, group distribution according to business plans, and differential configuration of multiple clusters, etc.

KubeVela v1.3 has iterated on the previous multi-cluster function. This article will reveal to you how to use KubeVela to deploy and manage multi-cluster applications to meet the above business requirements.

before the start

1. Prepare a Kubernetes cluster as KubeVela's control plane.
2. Make sure KubeVela v1.3[1] and KubeVela CLI v1.3.0 are installed successfully.
3. List of subclusters you want to manage in kubeconfig. We will use the 3 clusters beijing-1, beijing-2 and us-west-1 as examples.
4. Download and combine with multi-cluster-demo[2] to better understand how to use KubeVela's multi-cluster capabilities.

Distribute to multiple specified clusters

Distributing to multiple designated clusters is the most basic multi-cluster management operation. In KubeVela, you will implement this using an application strategy called topology. Clusters are listed in the clusters field of their properties as an array.
First let's make sure to switch KUBECONFIG to the prepared management cluster, and use vela cluster join to manage all the 3 clusters beijing-1, beijing-2 and us-west-1:

 ➜   vela cluster join beijing-1.kubeconfig --name beijing-1
➜   vela cluster join beijing-2.kubeconfig --name beijing-2
➜   vela cluster join us-west-1.kubeconfig --name us-west-1
➜   vela cluster list
CLUSTER          TYPE             ENDPOINT                   ACCEPTED  LABELS
beijing-1        X509Certificate  https://47.95.22.71:6443   true
beijing-2        X509Certificate  https://47.93.117.83:6443  true
us-west-1        X509Certificate  https://47.88.31.118:6443  true

Then open multi-cluster-demo and view basic.yaml:

 apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: example-app
  namespace: default
spec:
  components:
    - name: hello-world-server
      type: webservice
      properties:
        image: crccheck/hello-world
        port: 8000
      traits:
        - type: scaler
          properties:
            replicas: 3
        - type: gateway
          properties:
            domain: testsvc-mc.example.com
            # classInSpec : true   如果你所下发的集群里有安装 v1.20 以下版本的 Kubernetes ，请加上这个字段
            http:
              "/": 8000
  policies:
    - type: topology
      name: beijing-clusters
      properties:
        clusters: ["beijing-1","beijing-2"]

It can be seen that this application uses components of the webservice type, and finally distributes 3 copies of Deployment to the two clusters beijing-1 and beijing-2 respectively through the application strategy of the topology.

Please note that the prerequisite for the management and control cluster to successfully distribute resources to the sub-cluster is that the sub-cluster must have a corresponding namespace that has been created. Since each cluster has the default namespace by default, it can be delivered normally. Suppose we change the namespace of basic.yaml to multi-cluster, we will get an error:

 apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: example-app
  namespace: default
spec:
  components:
    - name: hello-world-server
      type: webservice
      properties:
        image: crccheck/hello-world
        port: 8000
      traits:
        - type: scaler
          properties:
            replicas: 3
        - type: gateway
          properties:
            domain: testsvc-mc.example.com
            # classInSpec : true   如果你所下发的集群里有安装 v1.20 以下版本的 Kubernetes ，请加上这个字段
            http:
              "/": 8000
  policies:
    - type: topology
      name: beijing-clusters
      properties:
        clusters: ["beijing-1","beijing-2"]

In the future version of KubeVela, we will support the use of the authentication system to complete this operation more conveniently and safely: create namespaces in sub-clusters with one click by managing and controlling the cluster.

After the subcluster namespace is created, switch back to the control cluster to create an application and deliver resources:

 ➜   vela up -f basic.yaml
Applying an application in vela K8s object format...
"patching object" name="example-app" resource="core.oam.dev/v1beta1, Kind=Application"
✅ App has been deployed 🚀🚀🚀
    Port forward: vela port-forward example-app
             SSH: vela exec example-app
         Logging: vela logs example-app
      App status: vela status example-app

Service status: vela status example-app --svc hello-world-server

 我们通过 vela status <应用名> 查看服务相关信息：

➜ vela status example-app
About:
Name: example-app
Namespace: default
Created at: 2022-03-25 17:42:33 +0800 CST
Status: running
Workflow:
mode: DAG
finished: true
Suspend: false
Terminated: false
Steps

• id: wftf9d4exj
  name:deploy-beijing-clusters
  type:deploy
  phase:succeeded
  message:

Services:

• Name: hello-world-server
  Cluster: beijing-1 Namespace: default
  Type: webservice
  Healthy Ready: 3/3
  Traits:
  ✅ scaler ✅ gateway: Visiting URL: testsvc-mc.example.com, IP: 60.205.222.30
• Name: hello-world-server
  Cluster: beijing-2 Namespace: default
  Type: webservice
  Healthy Ready: 3/3
  Traits:
  ✅ scaler ✅ gateway: Visiting URL: testsvc-mc.example.com, IP: 182.92.222.128

 beijing-1 和 beijing-2 都下发了对应的资源，它们可供外部访问的 IP 地址也显示出来，你因而可以用你希望的方式供用户访问了。

## 使用集群 labels 按需分组分发

除了上述的基本操作，我们常常会遇到另外的情况：跨地域部署到某些集群、指定哪个云厂商的集群等等。为了实现类似这样的需求，可以使用多集群的 labels 功能。

在这里，假设 us-west-1 集群来自 AWS，我们要额外分发应用到 AWS 的集群，则可以使用 vela cluster labels add 来对集群进行标记。当然，如果还有 us-west-2 等多个 AWS 相关集群，同样进行标记后，将会统一下发：

➜ ~ vela cluster labels add us-west-1 provider=AWS
Successfully update labels for cluster us-west-1 (type: X509Certificate).
provider=AWS
➜ ~ vela cluster list
CLUSTER TYPE ENDPOINT ACCEPTED LABELS
beijing-1 X509Certificate https://47.95.22.71:6443 true
beijing-2 X509Certificate https://47.93.117.83:6443 true
us-west-1 X509Certificate https://47.88.31.118:6443 true provider=AWS

 接下来我们对 basic.yaml 进行更新，新增一个应用策略 topology-aws：

...
policies:

 - type: topology
  name: beijing-clusters
  properties:
    clusters: ["beijing-1","beijing-2"]
- type: topology
  name: topology-aws
  properties:
    clusterLabelSelector:
      provider: AWS

 为了方便你学习，请直接部署基于 basic.yaml 更新后的 intermediate.yaml：

➜ ~ vela up -f intermediate.yaml

 再次查看应用的状态：

➜ vela status example-app
...

• Name: hello-world-server
  Cluster: us-west-1 Namespace: default
  Type: webservice
  Healthy Ready: 3/3
  Traits:
  ✅ scaler ✅ gateway: Visiting URL: testsvc-mc.example.com, IP: 192.168.40.10

 ## 通过应用策略进行配置差异化

除了在 basic.yaml 里定义的 deploy-beijing 这种应用策略，我们往往有更多的应用策略需求，比如高可用，希望单独给某些资源分发 5 个副本。这样的话，使用 override 类型的应用策略即可：

...

 clusterLabelSelector:
      provider: AWS
-  type: override
   name: override-high-availability
   properties:
      components:
        - type: webservice
          traits:
          - type: scaler
            properties:
              replicas: 5

 同时假设，我们希望的是，给 AWS 的应用分发并设置为高可用。那我们可以使用 KubeVela 提供的专门用于定义过程控制的工作流来管理。我们使用如下的一个工作流，它希望将本次应用部署，首先通过 deploy-beijing 的应用策略，分发给北京的集群们，接着给 Label 为 AWS 的集群分发 5 个副本高可用的应用策略：

...

 properties:
              replicas: 5

workflow:

 steps:
  - type: deploy
    name: deploy-beijing
    properties:
      policies: ["beijing-clusters"]
  - type: deploy
    name: deploy-aws
    properties:
      policies: ["override-high-availability","topology-aws"]

 接着我们给 intermediate.yaml 加上以上的应用策略和工作流后，更新为 advanced.yaml：

...
policies:

 - type: topology
  name: beijing-clusters
  properties:
    clusters: ["beijing-1","beijing-2"]
- type: topology
  name: topology-aws
  properties:
    clusterLabelSelector:
      provider: AWS
-  type: override
   name: override-high-availability
   properties:
      components:
        - type: webservice
          traits:
          - type: scaler
            properties:
              replicas: 5

workflow:

 steps:
  - type: deploy
    name: deploy-beijing
    properties:
      policies: ["beijing-clusters"]
  - type: deploy
    name: deploy-aws
    properties:
      policies: ["override-high-availability","topology-aws"]

 然后对其进行部署，并再次查看应用的状态：

➜ vela up -f advanced.yaml
Applying an application in vela K8s object format...
"patching object" name="example-app" resource="core.oam.dev/v1beta1, Kind=Application"
✅ App has been deployed 🚀🚀🚀

 Port forward: vela port-forward example-app
         SSH: vela exec example-app
     Logging: vela logs example-app
  App status: vela status example-app

Service status: vela status example-app --svc hello-world-serverapplication.core.oam.dev/podinfo-app configured

➜ vela status example-app
...

• Name: hello-world-server
  Cluster: us-west-1 Namespace: default
  Type: webservice
  Healthy Ready: 5/5
  Traits:
  ✅ scaler ✅ gateway: Visiting URL: testsvc-mc.example.com, IP: 192.168.40.10

 以上就是本次的全部分享，感谢你的阅读和试玩。

欢迎你继续探索 KubeVela v1.3 正式版[3]，这里有更多差异化配置的进阶用法等你发现和使用，比如 override 应用策略如何完成资源类型通配还是针对某些特定组件进行覆盖等等，以满足更加复杂的场景需求。

## 相关链接

[1] KubeVela v1.3
https://github.com/oam-dev/kubevela/releases/tag/v1.3.0

[2] multi-cluster-demo
https://github.com/oam-dev/samples/tree/master/12.Multi_Cluster_Demo

[3] 继续探索 KubeVela v1.3 正式版