How do we do full-link grayscale on the database?

Author: ten sleep

What is full link grayscale?

In the microservice architecture, the dependencies between services are intricate, and sometimes a function release depends on multiple services being upgraded and launched at the same time. We hope that the new versions of these services can be verified with small traffic at the same time. This is the unique full-link grayscale scene in the microservice architecture. By building an environment isolation from the gateway to the entire backend service, multiple different versions can be verified. service for grayscale verification.

Click the link below to view the live tutorial:

https://yqh.aliyun.com/live/detail/29004

During the release process, we only need to deploy the grayscale version of the service. When the traffic flows on the call link, the grayscale traffic is identified by the gateways, middleware and microservices it flows through, and dynamically forwarded to the corresponding service. Grayscale version. As shown below:

The above figure can well show the effect of this scheme. We use different colors to represent the gray-scale traffic of different versions. It can be seen that both the microservice gateway and the microservice itself need to identify the traffic and make dynamic decisions according to the governance rules. . When the service version changes, the forwarding of this call link will also change in real time. Compared with the grayscale environment built by machines, this solution can not only save a lot of machine cost and O&M manpower, but also help developers to perform refined full-link control of online traffic in real time and quickly.

OpenSergo[1] Traffic Routing Standard

Q: What is OpenSergo?

A: OpenSergo is a set of open, general-purpose service governance standards oriented to distributed service architecture and covering the whole-link heterogeneous ecology. It forms a general service governance standard based on industry service governance scenarios and practices. The biggest feature of OpenSergo is to define service governance rules with a unified set of configuration/DSL/protocol, oriented towards multi-language heterogeneous architecture, and achieve full-link ecological coverage . Whether the language of the microservice is Java, Go, Node.js or other languages, whether it is a standard microservice or Mesh access, from gateway to microservice, from database to cache, from service registration discovery to configuration, developers can use The same set of OpenSergo CRD standard configuration performs unified governance and control for each layer, without paying attention to the differences between frameworks and languages, reducing the complexity of heterogeneous and full-link service governance and control

Q: Why did you introduce OpenSergo to me before learning about full-link grayscale?
A: OpenSergo defines a unified YAML configuration method to implement full-link service governance specifications for distributed architectures. While introducing the specifications and standards, we can understand the implementation of the technical details, and we can also apply new The components are implemented with the OpenSergo standard.

Traffic routing, as the name suggests, is to route traffic with certain attribute characteristics to a specified destination. Traffic routing is an important part of traffic governance. Developers can implement various scenarios based on traffic routing standards, such as grayscale publishing, canary publishing, disaster recovery routing, and label routing.

Full link grayscale example:

Traffic routing rules (v1alpha1) are mainly divided into three parts:

• Workload Label Rule (WorkloadLabelRule): Label a group of workloads with corresponding labels. This block can be understood as labeling the database load (database, table) for the application or the corresponding storage layer.
• Traffic Label Rule (TrafficLabelRule): Label traffic with certain attribute characteristics with corresponding labels
• Make matching routes according to the workload label and traffic label, and route the traffic with the specified label to the matching workload

To mark traffic:

Traffic with certain attribute characteristics needs to be marked with corresponding labels.

Suppose now that users in the Shenzhen region need to be grayscaled to the new homepage, the test user location=cn-shenzhen, and cn-shenzhen is located in the location header:

 apiVersion: traffic.opensergo.io/v1alpha1
kind: TrafficLabelRule
metadata:
  name: my-traffic-label-rule
  labels:
    app: spring-cloud-zuul
spec:
  selector:
    app: spring-cloud-zuul
  trafficLabel: gray
  match:
  - condition: "=="    # 匹配表达式
    type: header       # 匹配属性类型
    key: 'location'   # 参数名
    value: cn-shenzhen       # 参数值

Through the above configuration, the location header is the HTTP traffic of cn-shenzhen, and the gray mark is marked, which means that the traffic is grayscale traffic.

Label the Workload:

In a business system that uses Nacos as a service discovery, it is generally necessary for the business to decide the marking method according to the microservice framework it uses. If the Java application uses the Spring Cloud microservice development framework, we can add the corresponding environment variables to the business container to complete the operation of adding tags. For example, if we want to add a version grayscale label to the node, then add traffic.opensergo.io/label: gray to the business container, so that the framework will add a gray label to the node when it registers with Nacos.

For some complex workload marking scenarios (such as database instances, cache instance labels), we can use WorkloadLabelRule CRD for marking. Example:

 apiVersion: traffic.opensergo.io/v1alpha1
kind: WorkloadLabelRule
metadata:
  name: gray-sts-label-rule
spec:
  workloadLabels: ['gray']
  selector:
    db: mse-demo
    table: mse_demo_table_gray

Common schemes of full-link grayscale on databases

Option 1: Shadow Library

Each maintains a separate set of libraries. Assuming that the name of the library in the baseline environment is mse-demo, the traffic of the gray environment can be mapped to the library of mse-demo-gray. We establish the corresponding environment traffic on the same instance. In the shadow library, we maintain a connection pool for each library connection in our business, and select the corresponding shadow library connection to access according to different traffic indicators, so as to achieve the effect of isolating the data and the baseline environment library, thus avoiding the gray-scale environment traffic The resulting data contaminates the baseline environment repository.

Option 2: Shadow Table

Similar to the shadow database scheme, for the shadow table scheme, a corresponding shadow table is created on the same database on the same instance. In the process of executing SQL, we parse and modify the SQL of gray traffic, so that the SQL of different environmental traffic can access the corresponding tables respectively. Assuming that the name of the table in the baseline environment is mse_demo_table, then the traffic of gray environment can be mapped to mse_demo_table_gray in the table. So as to achieve the effect of isolation of grayscale data and baseline environmental data table.

MSE[2] Database full-link grayscale capability

MSE provides a data isolation solution, and you can achieve full-link grayscale at the database level without modifying any business code. The following introduces the ability of MSE to achieve full-link grayscale through the shadow table scheme based on Mysql data storage.

Preconditions

• Application access to MSE
• Deploy the demo application

Deploy three applications A, B, and C in Alibaba Cloud Container Service, each of which deploys a base version and a gray version; and deploys a Nacos Server application for service discovery. For details, please refer to the tutorial to complete the application deployment: deploying the Demo application [3 ] .

Demo application introduction, the C application in this Demo will execute the following statement to the database:

 CREATE TABLE `mse_demo_table` (
  `id` int NOT NULL AUTO_INCREMENT,
  `location` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb3

The database table creation statement involved:

 CREATE TABLE `mse_demo_table` (
  `id` int NOT NULL AUTO_INCREMENT,
  `location` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb3

• To create a shadow table, the database table involved in our demo has mse_demo_table, because we need to create a grayscale gray environment, so we need to create the mse_demo_table_gray table in advance.

 CREATE TABLE `mse_demo_table_gray` (
  `id` int NOT NULL AUTO_INCREMENT,
  `location` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb3
;

Step 1: Configure full-link grayscale rules

You need to configure and complete the full-link release of MSE. For specific operation details, please refer to Tutorial: Configuring Full-Link Grayscale [ 4] .

Create the following swimlane rules:

Step 2: Configure the database full link grayscale

• We need to configure the following environment variables to additionally enable/configure the full-link grayscale capability of the database

Step 3: Result Verification

We initiate a grayscale request and find that all traffic requests access the grayscale environment:

 curl -H "location: cn-shenzhen" http://106.14.XX.XX/a
Agray[172.18.XX.XX] -> Bgray[172.18.XX.XX] -> Cgray[172.18.XX.XX]%

We query the shadow table with the following SQL:

 SELECT * FROM `mse_demo_table_gray`

The data found in the grayscale environment is inserted into the shadow table.

Not just full link grayscale

So far, the full-link grayscale capability of MSE service governance has supported cloud native gateways, ALB, APISIX, Apache Dubbo, Spring Cloud, RocketMQ, and databases. At the database level, we have implemented data-level traffic isolation through shadow tables. In the next step, we will further commercialize this capability, and the full-link grayscale will also support the cache-level capability.

Service governance is the only way to go through the transformation of microservices to a certain stage. During this process, we continue to have new problems.

• In addition to full-link grayscale, is there any other capability for service governance?
• Is there a standard definition of service governance capability, and what does service governance capability include?
• In multilingual scenarios, are there any best practices or standards for full links?
• How can heterogeneous microservices be managed uniformly?

In the process of exploring service governance, when we are connecting with other microservices, we find that the trouble caused by different governance systems is huge, and the cost of connecting two or even multiple governance systems is also huge. For this we propose the OpenSergo project. What OpenSergo wants to solve is the fragmentation and inability to communicate with different frameworks and languages in the concept of microservice governance.

The OpenSergo community is also uniting various communities for further cooperation, and the community will discuss and define a unified service governance standard. The current community is also working with bilibili, ByteDance and other companies to jointly build standards. Interested developers, communities and companies are also welcome to join in the joint construction of OpenSergo service governance standards. Welcome to join the OpenSergo community exchange group (Dingding group) for discussion: 34826335

Reference link:

[1] OpenSergo:

https://opensergo.io/en-us

[2] MSE microservice engine:

https://www.aliyun.com/product/aliware/mse

[3] Deploy the Demo application:

https://help.aliyun.com/document_detail/404845.html#h3-url-3

[4] Configure full link grayscale:

https://help.aliyun.com/document_detail/404845.html

10% discount for the first purchase of MSE Registration and Configuration Center Professional Edition, 10% discount for MSE Cloud Native Gateway Prepaid Full Specifications. Click here to take advantage of the discount!