GitOps delivery based on KubeVela

Introduction to KubeVela is a simple, easy-to-use, and highly scalable cloud-native application management and delivery platform. The OAM model behind KubeVela naturally solves the management problems of the combination and orchestration of complex resources in the process of application construction. Modeling the later operation and maintenance strategy means that KubeVela can manage complex large-scale applications in conjunction with GitOps, and converge on system complexity problems caused by the growth of the team and system scale.

Author | Dong Tianxin (Fog)

Review & proofreading: Xiyang, Haizhu

Editing & Typesetting: Wen Yan

KubeVela is a simple, easy-to-use, and highly scalable cloud-native application management and delivery platform that allows developers to easily and quickly define and deliver modern microservice applications on Kubernetes without knowing any Kubernetes infrastructure-related details.

The OAM model behind KubeVela naturally solves the management problems of complex resource combination and orchestration in the application construction process. It also models the later operation and maintenance strategy, which means that KubeVela can combine GitOps to manage complex and large-scale applications and converge due to the team The system complexity problem caused by the growth of the system scale.

What is GitOps

Its core idea is to store the declarative description of the infrastructure and application configuration required by the application system in the Git warehouse, and cooperate with an automated process, so that every time the warehouse is updated, the automated process can gradually update the environment to the latest configuration .

This method allows developers to automatically deploy applications by directly changing the code and configuration in the Git repository. Using GitOps can bring many values ​​to application development, such as:

• Increase productivity. Automatic continuous deployment can speed up the average deployment time and increase development efficiency.

• Lower the barriers for developers to deploy. By pushing code instead of container configuration, developers can deploy easily without knowing the internal implementation of Kubernetes.

• Make the change record traceable. Git is used to manage the cluster, so that every change can be tracked, and the audit trail is strengthened.

• The cluster can be restored through Git's rollback/branch function.

KubeVela and GitOps

As a declarative application delivery control plane, KubeVela naturally supports the use of GitOps, and makes users more clearly feel the benefits brought by GitOps and the end-to-end application delivery and management experience, including:

• Application delivery workflow (CD pipeline)
• • That is: KubeVela supports the description of procedural application delivery in the GitOps model, rather than simply declaring the final state;

• Deal with various dependencies and topological structures in the deployment process;

• Provide a unified upper-level abstraction on top of the semantics of various existing GitOps tools to simplify the application delivery and management process;

• Unified declaration, deployment and service binding of cloud services;

• Provide out-of-the-box delivery strategies (canary, blue-green release, etc.);

• Provide out-of-the-box hybrid cloud/multi-cloud deployment strategies (placement rules, cluster filtering rules, etc.);

• Provide Kustomize-style Patches in multi-environment delivery to describe deployment differences without having to learn any details of Kustomize itself;

• …… 

In this article, we mainly explain the steps of directly using KubeVela to deliver in GitOps mode.

GitOps workflow

The GitOps workflow is divided into two parts: CI and CD:

• CI (Continuous Integration): Continuous integration builds code for business applications, builds images, and pushes them to the image warehouse. There are many mature CI tools, such as GitHub Action and Travis, which are commonly used in open source projects, and Jenkins and Tekton, which are commonly used in enterprises. In this article, we use GitHub Action to complete the CI step. Of course, you can also use other CI tools instead. KubeVela revolves around GitOps to connect to the CI process under any tool.

• CD (Continuous Delivery): Continuous deployment will automatically update the configuration in the cluster, such as updating the latest image in the mirror warehouse to the cluster. There are currently two main types of CDs:

• • 1) Push-Based: The CD in Push mode is mainly completed by configuring the CI pipeline. In this way, the access key of the cluster needs to be shared with the CI, so that the CI pipeline can push the changes to the cluster through commands. You can refer to Our previous blog: Use Jenkins + KubeVela to complete continuous delivery of applications (see related links at the end of the article for details).

2) Pull-Based: The CD in the pull mode will monitor changes in the warehouse (code warehouse or configuration warehouse) in the cluster, and synchronize these changes to the cluster. Compared with Push mode, Pull-Based is actively pulled by the cluster for updates, thus avoiding the problem of secret key exposure. This is also the core content of this article.

There are two types of delivery personnel, which we will introduce separately:

1. for the platform administrator / infrastructure operation and maintenance personnel to deliver , users can directly update the configuration file in the repository, thereby updating the cluster configuration infrastructure, such as software-dependent systems, security policy, storage, network and other infrastructure Configuration.

1. to terminal developers , once the user's code is merged into the application code repository, it will automatically trigger the update of the application in the cluster, which can complete the iteration of the application more efficiently, and KubeVela's gray-scale release, traffic allocation, and multi-cluster deployment The combination of other functions can form a more powerful automated publishing capability.

Oriented platform management /operations and maintenance personnel 16196242b630b8

As shown in the figure, for platform administrators/operations and maintenance personnel, they do not need to care about the application code, so they only need to prepare a Git configuration repository and deploy the KubeVela configuration file. Follow-up changes to the application and infrastructure configuration This can be done by directly updating the Git configuration repository, so that every configuration change can be tracked.

Prepare to configure the warehouse

For specific configuration, please refer to Example Warehouse 1 (see related links at the end of the article for details).

In this example, we will deploy a MySQL database software as the project's infrastructure, while deploying a business application to use this database. The directory structure of the configuration warehouse is as follows:

• clusters/ KubeVela GitOps included in the cluster configuration, users will need to clusters/ deployed to the cluster manually file. This is a one-time control operation. After execution, KubeVela can automatically monitor file changes in the configuration warehouse and automatically update the configuration in the cluster. Which, clusters/apps.yaml will listen apps/ change all applications under, clusters/infra.yaml will listen infrastructure/ change all the infrastructure next.

​

• apps/ all configuration directory contains business applications, in the present embodiment is used for a service application database.

​

• infrastructure/ contains some infrastructure-related configuration and policy, in this case MySQL database.

├── apps
│   └── my-app.yaml
├── clusters
│   ├── apps.yaml
│   └── infra.yaml
└── infrastructure
    └── mysql.yaml

KubeVela recommends using the above directory structure to manage your GitOps repository. clusters/ stored configuration-related KubeVela GitOps and need to be manually deployed to a cluster, apps/ and infrastructure/ respectively store your applications and infrastructure configuration. By separating the application from the basic configuration, you can manage your deployment environment more reasonably and isolate the impact of application changes.

clusters/ directory

First, let's take a look at the clusters directory, which is also the initial configuration directory for KubeVela to connect to GitOps.

To clusters/infr a.yaml example:

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: infra
spec:
  components:
  - name: database-config
    type: kustomize
    properties:
      repoType: git
      # 将此处替换成你需要监听的 git 配置仓库地址
      url: https://github.com/FogDong/KubeVela-GitOps-Infra-Demo
      # 如果是私有仓库，还需要关联 git secret
      # secretRef: git-secret
      # 自动拉取配置的时间间隔，由于基础设施的变动性较小，此处设置为十分钟
      pullInterval: 10m
      git:
        # 监听变动的分支
        branch: main
      # 监听变动的路径，指向仓库中 infrastructure 目录下的文件
      path: ./infrastructure

apps.yaml and infra.yaml remained almost the same, but the monitor file directory differ. In apps.yaml in, properties.path value will be changed ./apps , showed that listening apps/ file directory changes.
cluster folder GitOps control configuration files need to be deployed at initialization time manual to a cluster, after KubeVela will automatically monitor apps/ and infrastructure/ configuration file in the directory synchronization and updated regularly.

apps/ directory

​

apps/ directory to store the application configuration file, which is equipped with a simple application and database information of Ingress. The application will connect to a MySQL database and simply start the service. Under the default service path, the current version number will be displayed. In /db of the path, lists information about the current database.

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: my-app
  namespace: default
spec:
  components:
    - name: my-server
      type: webservice
      properties:
        image: ghcr.io/fogdong/test-fog:master-cba5605f-1632714412
        port: 8088
        env:
          - name: DB_HOST
            value: mysql-cluster-mysql.default.svc.cluster.local:3306
          - name: DB_PASSWORD
            valueFrom:
              secretKeyRef:
                name: mysql-secret
                key: ROOT_PASSWORD
      traits:
        - type: ingress
          properties:
            domain: testsvc.example.com
            http:
              /: 8088

This is an application that uses KubeVela's built-in component type webservice, which is bound to Ingress operation and maintenance features. By declaring the operation and maintenance capabilities in the application, only one file is required to gather the underlying Deployment, Service, and Ingress, so as to manage the application more conveniently.

​

infrastructure/ directory

​

infrastructure/ store some configuration of infrastructure directory. Here we use the mysql controller (see related links at the end of the article for details) to deploy a MySQL cluster.

​

Note, make sure that your cluster has a secret, and by ROOT_PASSWORD declared MySQL password.

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: mysql
  namespace: default
spec:
  components:
    - name: mysql-controller
      type: helm
      properties:
        repoType: helm
        url: https://presslabs.github.io/charts
        chart: mysql-operator
        version: "0.4.0"
    - name: mysql-cluster
      type: raw
      dependsOn:
        - mysql-controller
      properties:
        apiVersion: mysql.presslabs.org/v1alpha1
        kind: MysqlCluster
        metadata:
          name: mysql-cluster
        spec:
          replicas: 1
          # 关联 secret 名称
          secretName: mysql-secret

In this MySQL application, we use the KubeVela workflow capabilities. The workflow is divided into two steps, the first step is to deploy the MySQL controller. After the controller is successfully deployed and running correctly, the second step will begin to deploy the MySQL cluster.

Deployment clusters/ files in a directory

After you configure coexistence after the above file into the Git repository configuration, we need to be manually deployed in a cluster clusters/ KubeVela GitOps configuration files in the directory.

First deployed in a cluster clusters/infra.yaml . It can be seen in the cluster automatically pull up infrastructure/ MySQL deployment of files in the directory:

kubectl apply -f clusters/infra.yaml

$ vela ls
APP     COMPONENT           TYPE        TRAITS  PHASE   HEALTHY STATUS  CREATED-TIME
infra   database-config     kustomize           running healthy         2021-09-26 20:48:09 +0800 CST
mysql   mysql-controller    helm                running healthy         2021-09-26 20:48:11 +0800 CST
└─      mysql-cluster       raw                 running healthy         2021-09-26 20:48:11 +0800 CST

Then, deployed in a cluster clusters/apps.yaml , we can see it automatically pull up apps/ application deployment files in the directory:

kubectl apply -f clusters/apps.yaml

APP     COMPONENT           TYPE        TRAITS  PHASE   HEALTHY STATUS  CREATED-TIME
apps    apps                kustomize           running healthy         2021-09-27 16:55:53 +0800 CST
infra   database-config     kustomize           running healthy         2021-09-26 20:48:09 +0800 CST
my-app  my-server           webservice  ingress running healthy         2021-09-27 16:55:55 +0800 CST
mysql   mysql-controller    helm                running healthy         2021-09-26 20:48:11 +0800 CST
└─      mysql-cluster       raw                 running healthy         2021-09-26 20:48:11 +0800 CST

So far, we have deployed the KubeVela GitOps configuration file to automatically pull up the application and database in the cluster.

Through the Ingress of the curl application, you can see that the current version is 0.1.5 and successfully connected to the database:

$ kubectl get ingress
NAME        CLASS    HOSTS                 ADDRESS         PORTS   AGE
my-server   <none>   testsvc.example.com   <ingress-ip>    80      162m
$ curl -H "Host:testsvc.example.com" http://<ingress-ip>
Version: 0.1.5
$ curl -H "Host:testsvc.example.com" http://<ingress-ip>/db
User: KubeVela
Description: It's a test user

Change setting

After completing the first deployment, we can complete the configuration update of the application in the cluster by modifying the configuration in the configuration warehouse.

Modify the domain of the application Ingress:

...
      traits:
        - type: ingress
          properties:
            domain: kubevela.example.com
            http:
              /: 8089

After a period of time, check the Ingress in the cluster again:

NAME        CLASS    HOSTS                 ADDRESS         PORTS   AGE
my-server   <none>   kubevela.example.com  <ingress-ip>    80      162m

As you can see, the Host address of Ingress has been successfully updated.

In this way, we can easily update the files in the Git configuration repository to automatically update the configuration in the cluster.

for terminal developers

For terminal developers, in addition to the KubeVela Git configuration warehouse, an application code warehouse is also required. As shown in the figure, after the user updates the code in the application code warehouse, a CI needs to be configured to automatically build the image and push it to the image warehouse. KubeVela will monitor the latest mirror in the mirror warehouse, and automatically update the mirror configuration in the configuration warehouse, and finally update the application configuration in the cluster. Allow users to achieve the effect that the configuration in the cluster is automatically updated after the code is updated.

Prepare code warehouse

Prepare a code repository, which contains some source code and the corresponding Dockerfile.

These codes will connect to a MySQL database and simply start the service. Under the default service path, the current version number will be displayed. Under the /db path, the information in the current database will be listed.

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        _, _ = fmt.Fprintf(w, "Version: %s\n", VERSION)
    })
    http.HandleFunc("/db", func(w http.ResponseWriter, r *http.Request) {
        rows, err := db.Query("select * from userinfo;")
        if err != nil {
            _, _ = fmt.Fprintf(w, "Error: %v\n", err)
        }
        for rows.Next() {
            var username string
            var desc string
            err = rows.Scan(&username, &desc)
            if err != nil {
                _, _ = fmt.Fprintf(w, "Scan Error: %v\n", err)
            }
            _, _ = fmt.Fprintf(w, "User: %s \nDescription: %s\n\n", username, desc)
        }
    })
    if err := http.ListenAndServe(":8088", nil); err != nil {
        panic(err.Error())
    }

We hope that after the user changes the code and submits it, the latest mirror will be automatically constructed and pushed to the mirror warehouse. This step of CI can be achieved by integrating GitHub Actions, Jenkins or other CI tools. In this example, we use GitHub Actions to complete continuous integration. For specific code files and configurations, please refer to Sample Warehouse 2 (see related links at the end of the article for details).

Configure key information

After the new image is pushed to the mirror warehouse, KubeVela will recognize the new image and update the Application configuration files in the warehouse and cluster. Therefore, we need a Secret containing Git information to enable KubeVela to submit to the Git repository. Deploy the following files and replace the username and password with your Git username and password (or Token):

apiVersion: v1
kind: Secret
metadata:
  name: git-secret
type: kubernetes.io/basic-auth
stringData:
  username: <your username>
  password: <your password>

Prepare to configure the warehouse

The configuration warehouse is similar to the previous configuration for operation and maintenance personnel, only the configuration related to the mirror warehouse needs to be added. For specific configuration, please refer to Example Warehouse 1 (see related links at the end of the article for details).

Modify clusters/ in apps.yaml , the GitOps configuration will monitor the warehouse apps/ application file and image changes at the warehouse in the mirror update:

...
  imageRepository:
    # 镜像地址
    image: <your image>
    # 如果这是一个私有的镜像仓库，可以通过 `kubectl create secret docker-registry` 创建对应的镜像秘钥并相关联
    # secretRef: imagesecret
    filterTags:
      # 可对镜像 tag 进行过滤
      pattern: '^master-[a-f0-9]+-(?P<ts>[0-9]+)'
      extract: '$ts'
    # 通过 policy 筛选出最新的镜像 Tag 并用于更新
    policy:
      numerical:
        order: asc
    # 追加提交信息
    commitMessage: "Image: {{range .Updated.Images}}{{println .}}{{end}}"

Modify apps/my-app.yaml Image field, followed by the # { "$ imagepolicy": " default: apps"} notes. KubeVela will use this comment to update the corresponding mirror field. default:apps is arranged above GitOps corresponding name and namespace.

spec:
  components:
    - name: my-server
      type: webservice
      properties:
        image: ghcr.io/fogdong/test-fog:master-cba5605f-1632714412 # {"$imagepolicy": "default:apps"}

The clusters/ the warehouse configuration file contains a mirror of the update to the cluster, we will be able to complete the update by modifying the application code.

Modify the code

The code file Version changed 0.1.6 , and modify data in the database:

const VERSION = "0.1.6"
...
func InsertInitData(db *sql.DB) {
    stmt, err := db.Prepare(insertInitData)
    if err != nil {
        panic(err)
    }
    defer stmt.Close()
    _, err = stmt.Exec("KubeVela2", "It's another test user")
    if err != nil {
        panic(err)
    }
}

Submit the change to the code repository, and you can see that the CI pipeline we configured starts to build the mirror and push it to the mirror repository.

The KubeVela will be mirrored by listening to the warehouse, the warehouse to update the configuration according to the latest mirrored Tag apps/ applications in my-app .

​

At this point, you can see there is a configuration repository from kubevelabot submitted, all information submitted with Update image automatically. prefix. You can also {{range .Updated.Images}}{{println .}}{{end}} in commitMessage additional information you need in the field.

It is worth noting that if you want to code and configuration in the same warehouse, to be filtered out from kubevelabot submit to prevent duplicate build the pipeline. You can filter through the following configuration in CI:

​

jobs:
publish:
  if: "!contains(github.event.head_commit.message, 'Update image automatically')"

Re-view the application in a cluster, you can see after a period of time, the application my-app the image has been updated.

KubeVela will pass your configuration interval time interval, respectively, from time to time to get the latest information from the configuration storage and warehouse Mirror:

• When the configuration file in the Git repository is updated, KubeVela will update the applications in the cluster according to the latest configuration.

• When a new tag is added to the mirror repository, KubeVela will filter out the latest mirror tag according to the policy rules you configured, and update it to the Git repository. When the files in the code warehouse are updated, KubeVela will repeat the first step to update the files in the cluster, thus achieving the effect of automatic deployment.

By curl corresponding Ingress current version and database information to view:

$ kubectl get ingress
NAME        CLASS    HOSTS                 ADDRESS         PORTS   AGE
my-server   <none>   kubevela.example.com  <ingress-ip>    80      162m
$ curl -H "Host:kubevela.example.com" http://<ingress-ip>
Version: 0.1.6
$ curl -H "Host:kubevela.example.com" http://<ingress-ip>/db
User: KubeVela
Description: It's a test user
User: KubeVela2
Description: It's another test user

The version has been successfully updated! So far, we have completed all operations from changing the code to automatically deploying to the cluster.

Summary

On the operation and maintenance side, if you need to update the configuration of the infrastructure (such as the database) or the configuration items of the application, you only need to modify the files in the configuration warehouse, and KubeVela will automatically synchronize the configuration to the cluster, simplifying the deployment process.

On the R&D side, after users modify the code in the code warehouse, KubeVela will automatically update the mirror in the configuration warehouse. In order to update the version of the application.

By combining with GitOps, KubeVela accelerates the entire process from application development to deployment.

original to check the official homepage and documentation of the KubeVela project! !

related links

1) __Use Jenkins + KubeVela to complete the continuous delivery of the application__:

https://kubevela.io/zh/blog/2021/09/02/kubevela-jenkins-cicd

2) Example warehouse 1:

https://github.com/oam-dev/samples/tree/master/9.GitOps\_Demo/for-SREs

3）mysql controller：

https://github.com/bitpoke/mysql-operator

4) Example warehouse 2:

https://github.com/oam-dev/samples/tree/master/9.GitOps\_Demo/for-developers/app-code

Copyright Statement: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users, and the copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.