New function of full-link grayscale: MSE online configuration label push

Author: Xun Mu, Liu Shi

background

In the microservice scenario, full-link grayscale, as a low-cost new function verification method, has been more and more widely used. In addition to the grayscale of microservice instances and traffic, the configuration items in the microservice application should also have corresponding grayscale capabilities to meet the grayscale application's demands for special configurations.

Why do you need to configure tag push

From the whole link grayscale

In the microservice scenario, the grayscale release of applications has ushered in new challenges. Different from the monolithic architecture where the application is packaged as a whole to release the test version, microservice applications are often composed of multiple services. These services are usually developed independently by different teams. A new feature usually only involves part of the service, and when testing new features, we only need to publish this part of the service. In order for the microservice application to run normally, a solution needs to be designed so that grayscale traffic can also pass through other services that do not need to be published.

This function usually has two solutions of physical environment isolation and logical environment isolation. The former needs to build a network isolation and resource-independent environment for each grayscale environment. In order for the application to run normally, it is also necessary to make redundant releases in the environment for services and various middleware that have not been reached by grayscale. as the picture shows:

There is a lot of waste of machine resources in the solution of physical environment isolation. Therefore, the industry generally adopts the latter, that is, the method of logical environment isolation, as a full-link grayscale solution. It is only necessary to deploy the grayscale service, and by invoking the flow control on the link, the grayscale traffic can flow between the grayscale environment and the formal environment, realize the normal operation of the grayscale microservice application, and help the business side to verify new functions. as the picture shows:

Application Scenarios for Configuring Grayscale

The application of configuring grayscale capabilities may be involved in many business scenarios. The following are several typical scenarios.

• Canary release

Canary release has been widely implemented in the industry and is a common means of new version release. The canary conference divides the traffic proportionally. Initially, a small proportion of traffic is allocated to the instances of the new version. After a period of operation, after confirming that the new version is running normally, the proportion of the allocated traffic is gradually increased until the full amount of traffic is finally cut off. flow. Publishing in this way can control the impact surface when a problem occurs in the new version, and improve the stability of the system.

Canary releases are usually achieved by flow dyeing and machine marking. The new version of the machine is marked with the canary version, and some traffic is marked with the canary version, which eventually evolves into a full-link grayscale solution. Configuration items in the canary version of the application may need to use different configuration values than in the old version, which requires the ability to configure grayscale.

• New features are online

When the changes involve the launch of larger functions, the stability of the functions is often verified by gradually increasing the volume. A typical way of increasing volume is whitelisting, that is, users/devices configured in the whitelist can use new functions, and users not in the whitelist still use the old version. After running online for a period of time, collect feedback from whitelisted users, and gradually increase the number of users/devices in the whitelist while optimizing the function. After the function reaches the final stable state, it will be released in full.

Users/devices from the whitelist are specially marked and routed to the grayscale environment. If the configuration in the new function needs to use different configuration values than in the old version, the configuration grayscale needs to be synchronized.

• database migration

Database migration is also a common problem in business development. With the rapid growth of business, the original database may no longer be able to meet future business needs in terms of capacity and performance. In this case, database migration is required. In order to ensure the stability of the migration process, the migration is usually gradual. During this process, some traffic will be written to the new database, and some traffic will be written to the old database. After the migration is completed, all traffic will be switched to the new database. During the migration process, we can operate on different databases through traffic coloring and configuring grayscale.

Problems and Solutions

Microservice applications usually introduce a configuration center for configuration management, which provides dynamic configuration push capabilities so that applications can dynamically change the running logic without restarting. However, the management dimension of the configuration center is only the configuration item itself, and cannot perceive the environment information of the service instance that comes to obtain the configuration. In this context, if a certain configuration needs to use different values in the formal environment and the grayscale environment, they must be used as different configuration items in the configuration center, we may need to write such code:

 ...
if (env == "gray") {
    cfg = getConfig('cfg1');
} else {
    cfg = getConfig('cfg2');
}
...

If there are multiple configuration items with different configuration values in the grayscale environment, such code needs to be repeated many times. In a more extreme scenario, if there are multiple sets of grayscale environments under test, the configuration values in each environment are different, and the code responsible for obtaining configuration items will be more complicated. In addition, there are often multiple services in a grayscale environment, and each service needs to independently maintain a set of similar codes. The final solution is shown in the figure. The configuration values used by the same configuration item in different environments need to be actively distinguished in the user application.

The reason is that the configuration center cannot perceive the environmental information of the service instance, so we must perform this task in the code instead of the configuration center, which leads to the intrusion of the environmental information into the business code. Although we can reduce the complexity of using business code by encapsulating the configuration code, as long as the configuration center cannot perceive the environmental information of the service instance, the intrusiveness of such environmental information to the business code cannot be avoided. .

Features

The newly launched configuration label push function of MSE sinks the perception of environmental information in the configuration management scenario to the platform side, and the agent is responsible for it. Users only need to access the MSE, and can easily use the configuration push capability in the full-link grayscale scene, eliminating the cumbersome environment information detection logic in the business code. as the picture shows:

Features provided by configuring tag push include:

• Configuration management of label dimensions

In the configuration list page, you can view various configuration items in the application. Currently, MSE supports the collection of three types of configurations:

1. 1. A configuration class marked with the annotation @Switch provided by the SDK
   2. Configuration items marked with Spring's annotation @Value
   3. Configuration classes marked with SpringBoot's annotation @ConfigurationProperties

The configuration values of all service instances will be displayed under each configuration item. Users can intuitively see the grayscale environment where different instances are located through the label name, and can also view the distribution of configuration values in different grayscale environments.

• Application configuration push capability of tag dimension

With "Push by Tag", users can easily push persistent new configuration values to all service instances in a specified grayscale environment. Persistence means that even if the application restarts, the configuration items for that environment will not be lost.

• Instance Dynamic Marking in Configuration Scenarios

In addition to setting labels for service instances through the MSE marking method when the application starts, users can dynamically add/delete labels for service instances in the MSE console to adapt to the management of configuration items in different grayscale environments.

• Traceability around the entire configuration tag push process

MSE provides full-process traceability capabilities for tag push, including instance tag change records and tag push records, helping users troubleshoot problems in the tag push process easily.

Configuring Tag Push Practices

Next, we demonstrate the use process of configuring tag push through practice, which can be completed in just three simple steps.

• Preparation

1. Integrate applications into MSE microservice governance
2. Enter the MSE console and select the region
3. Go to Governance Center > Application Governance, enter the application you just accessed

• Step 1: Add a label

The first step in configuring label push is to set labels for service instances. The tag of the service instance can be set at startup via -Dalicloud.service.tag, and can also be set dynamically in the MSE console.

Next we demonstrate the process of dynamically marking service instances. Select Application Configuration > Label List, all service instances under the current application will be displayed here. Click the "Add Label" button in the upper left corner, and in the pop-up node marking window, we can select a batch of service instances to mark. Users can filter the instances that need to be marked by node IP. In addition to filtering by node IP, MSE provides the ability to filter service instances by hostname. After selecting the machine to be marked, enter the name of the label and click "Add Label" to complete the machine marking.

• Step 2: Push by tag

After the machine is marked, we can push configuration values for the specified labels. Go back to Application Configuration > Configuration List, select the "customName" configuration item, and click "Push by Tag". In the pop-up push window, we select the label to be pushed, and set the configuration value to be pushed. Click "Next: Value Comparison" to see the difference between the old and new configuration items. Then click "Tag Push" to complete the configuration delivery.

You can see that the configuration items have been successfully pushed to two machine instances with the "gray" label.

After that, we do the same for the "gray2" tag, pushing the "testGray2" configuration value.

Configuration value pushes for tags are persistent. The user only needs to operate on the console once, and when the service instance with the specified label is restarted, the MSE will deliver the persistent configuration value to the application through the Agent.

• Step 3: View the distribution of configuration values

Go back to the application configuration > configuration list, select the "customName" configuration item, we can see that the two configuration values just pushed for the tags "gray" and "gray2" have taken effect.

Click on "Value Distribution". In the pop-up window, you can see the distribution of the configuration value of the configuration item on all machine instances, and you can also see its persistent value under different labels.

concluding remarks

This paper introduces the problems brought by the full-link grayscale scene to configuration management, introduces the solution of MSE for this scene, and demonstrates the use process of configuration label push through practice. In the future, MSE will also do more exploration on configuration governance to help users better solve the problems in microservice configuration management and improve the stability of microservice applications.

MSE registered configuration 20% off the first purchase, 30% off the first purchase of 1 year and above. 10% discount for MSE cloud native gateway prepaid full specification. Click here to enjoy the discount now!