Two sides of Ctrip: How does the distributed configuration center server perceive configuration changes?

introduction

I wrote an article "How does the distributed configuration center apollo perceive the configuration modification in real time" , that is, how does the client client know that the configuration has been modified? Many readers privately message me since you said how the client side perceives it. , how does the server know that the configuration has been modified. Today, let's take a look at how Apollo modifies the configuration file in Portal and how to notify configService. What are portal and configService? It is recommended to read this article "How does the distributed configuration center apollo perceive the configuration modification in real time" , which has a brief introduction to these modules. If you really don't want to read it, I can just cut it a picture here

How does the server perceive updates

Let's take a look at a picture provided by the official website

1. The user operates the configuration release in the Portal
2. Portal calls the interface operation release of Admin Service
3. After the Admin Service publishes the configuration, it sends a ReleaseMessage to each Config Service
4. After the Config Service receives the ReleaseMessage, it notifies the corresponding client

The above process is the main process from Portal to ConfigService. Let's take a look at the specific details. To know the details, we have to debug a source code by ourselves.
We can run the project locally by ourselves according to the documentation on the official website. The documentation is still very detailed, as long as you follow the steps, you can run it. We just create a new project and edit the key, then open F12 of the browser. When we click the submit button, we will know which interfaces she has called. With the interfaces, we will know that the rest of the entry is to break the point. debugged.

How to get AdminService

According to this method, can we locate the controller of the back-end code of the portal module? Find the corresponding controller and open it to see that there is basically no business logic

Then portal is followed by a call to adminService .

According to the above figure, we can find the corresponding adminService, how does the portal find the corresponding adminService service, because adminService can deploy multiple machines, here we need to use service registration and discover that adminService can only be registered After arriving at the service center, the portal can obtain the corresponding adminService service through the service registration center. Apollo The default is to use eureka as service registration and discovery, it also provides nacos, consul as service registration and discovery, and also provides a kind of kubernetes does not use a third party for service registration and discovery, directly Configure the address of the service in the database. If multiple addresses can be separated by commas in the database.

It provides four ways to get the service list. If the registry we use is eureka, do we need to get the service list through the eureka api? If our service discovery uses nacos, do we need to pass the nacos API? to get a list of services. . . Therefore, Apollo provides a MetaService layer to encapsulate the details of service discovery. For Portal and Client, the service information of Admin Service and Config Service is always obtained through an Http interface, without caring about the actual service registration and discovery components behind it. Just like we usually move bricks, there is no problem that cannot be solved by adding an intermediate layer. If one fails, then add another. So MetaService provides two interfaces services/admin and services/config to obtain service information of Admin Service and Config Service respectively. So how does Portal call the services/admin interface? In the com.ctrip.framework.apollo.portal.component#AdminServiceAddressLocator class in the apollo-portal project,

• When this class is loaded, it will obtain the service address of adminService through the services/admin interface provided by MetaService for caching.

   @PostConstruct
  public void init() {
    allEnvs = portalSettings.getAllEnvs();
    //init restTemplate
    restTemplate = restTemplateFactory.getObject();
    
    refreshServiceAddressService =
        Executors.newScheduledThreadPool(1, ApolloThreadFactory.create("ServiceLocator", true));
    // 创建延迟任务，1s后开始执行获取AdminService服务地址
    refreshServiceAddressService.schedule(new RefreshAdminServerAddressTask(), 1, TimeUnit.MILLISECONDS);
  }

  
  The above is to request the address of MetaService, so what is the address of MetaService? How to get this? com.ctrip.framework.apollo.portal.environment#DefaultPortalMetaServerProvider This class.

The portal module is finished, let's go back to adminService . Through portal calling the interface address of adminService, we can quickly find its entry
The implementation of AdminService is also very simple

 
  @PreAcquireNamespaceLock
@PostMapping("/apps/{appId}/clusters/{clusterName}/namespaces/{namespaceName}/items")
  public ItemDTO create(@PathVariable("appId") String appId,
                        @PathVariable("clusterName") String clusterName,
                        @PathVariable("namespaceName") String namespaceName, @RequestBody ItemDTO dto) {
    Item entity = BeanUtils.transform(Item.class, dto);

    ConfigChangeContentBuilder builder = new ConfigChangeContentBuilder();
    Item managedEntity = itemService.findOne(appId, clusterName, namespaceName, entity.getKey());
    if (managedEntity != null) {
      throw new BadRequestException("item already exists");
    }
    entity = itemService.save(entity);
    builder.createItem(entity);
    dto = BeanUtils.transform(ItemDTO.class, entity);

    Commit commit = new Commit();
    commit.setAppId(appId);
    commit.setClusterName(clusterName);
    commit.setNamespaceName(namespaceName);
    commit.setChangeSets(builder.build());
    commit.setDataChangeCreatedBy(dto.getDataChangeLastModifiedBy());
    commit.setDataChangeLastModifiedBy(dto.getDataChangeLastModifiedBy());
    commitService.save(commit);

    return dto;
  }

PreAcquireNamespaceLock annotation

First of all, there is a @PreAcquireNamespaceLock annotation on the method, which should be able to guess a distributed lock based on the name, which is probably to acquire NameSpace. Now the more common way of distributed lock is to use redis and zookeeper. But here apollo is implemented using a database. You can read the source code to understand the details. It is nothing more than locking and inserting a piece of data into the DB, releasing the lock and then deleting the data. The slight difference is that if the acquisition of the lock fails, it will return to the failure directly, and will not continue to spin or sleep to acquire the lock again. Because the failure to acquire the lock means that someone else has modified the configuration before you, and only after the new configuration added by this person is published or deleted, other people can continue to add configuration, so that a NameSpace can only be used by one person at the same time. Revise. This restriction is turned off by default and needs to be configured in the database (ServiceConfig table of ApolloConfigDb)
Generally, the configuration modification of our application should be relatively infrequent. If multiple people modify it at the same time, the situation will be less. Besides, some companies are developing and submitting configuration, and testing to release configuration. The submission and modification cannot be made by the same person. Even less, there should be no need to configure namespace.lock.switch=true A namespace can only be modified by one person.

The following code is very simple and clear. It is a simple parameter judgment and then performs the storage operation, and inserts the data into the Item table. This is our newly added configuration data and it has been saved. The effect is as follows

At this time, the newly added configuration will not work and will not be pushed to the client. It's just a state similar to a draft.

publish configuration

Next, we need to make the new configuration above take effect and push it to the client. Similarly, we click the publish button and then we can know the corresponding back-end method entry

We can directly find the method entry of adminService through this interface

  public ReleaseDTO publish(@PathVariable("appId") String appId,
                            @PathVariable("clusterName") String clusterName,
                            @PathVariable("namespaceName") String namespaceName,
                            @RequestParam("name") String releaseName,
                            @RequestParam(name = "comment", required = false) String releaseComment,
                            @RequestParam("operator") String operator,
                            @RequestParam(name = "isEmergencyPublish", defaultValue = "false") boolean isEmergencyPublish) {
    Namespace namespace = namespaceService.findOne(appId, clusterName, namespaceName);
    if (namespace == null) {
      throw new NotFoundException(String.format("Could not find namespace for %s %s %s", appId,
                                                clusterName, namespaceName));
    }
    Release release = releaseService.publish(namespace, releaseName, releaseComment, operator, isEmergencyPublish);

    //send release message
    Namespace parentNamespace = namespaceService.findParentNamespace(namespace);
    String messageCluster;
    if (parentNamespace != null) {
      messageCluster = parentNamespace.getClusterName();
    } else {
      messageCluster = clusterName;
    }
    messageSender.sendMessage(ReleaseMessageKeyGenerator.generate(appId, messageCluster, namespaceName),
                              Topics.APOLLO_RELEASE_TOPIC);
    return BeanUtils.transform(ReleaseDTO.class, release);
  }

• The above code will not be analyzed carefully. If you are interested, you can debug your own breakpoints. Let’s focus on releaseService.publish This method contains some logic related to grayscale publishing, but this is not the focus of this article. This The method is mainly to insert data into the release table.
• Next is the messageSender.sendMessage method, which is mainly to insert a record into the ReleaseMessage table. After saving ReleaseMessage this table will get the corresponding primary key ID, and then put this ID into a queue. Then when the DatabaseMessageSender is loaded, a scheduled task will be started by default to obtain the message IDs placed in the above queue, and then find the messages smaller than these IDs and delete them.
  The publishing process is over, and there is no mention of how the server perceives that the configuration has been modified.

Config Service notifies configuration changes

apolloConfigService When the service starts ReleaseMessageScanner will start a timed task to query every 1s ReleaseMessage is there any latest news, if there is, it will be notified All message listeners such as NotificationControllerV2 , ConfigFileController , etc., this message listener registration is registered in ConfigServiceAutoConfiguration.
NotificationControllerV2 After getting the configuration and publishing AppId+Cluster+Namespace , the corresponding client will be notified, so that from the portal to the configService to the client, the entire message notification changes are strung together. For the specific details of the server's notification to the client, please refer to "How the distributed configuration center apollo perceives the configuration modification in real time"

Summarize

This completes the process of how to update the server configuration.

1. The user operates the configuration release in the Portal
2. Portal calls the interface operation release of Admin Service
3. After the Admin Service publishes the configuration, it sends a ReleaseMessage to each Config Service
4. After the Config Service receives the ReleaseMessage, it notifies the corresponding client

The source code of apollo is relatively simple compared to other middleware, and it is more suitable for students who want to study the source code of the middleware, but do not know how to start.

Finish

• Due to my lack of knowledge and knowledge, it is inevitable that there will be mistakes. If you find any mistakes, please leave a message and point them out to me, and I will correct them.
• If you think the article is not bad, your forwarding, sharing, appreciation, likes, and comments are the greatest encouragement to me.
• Thank you for reading, you are very welcome and appreciate your attention.

standing on the shoulders of giants
https://www.apolloconfig.com/#/en/design/apollo-design?id=%e4%b8%80%e3%80%81%e6%80%bb%e4%bd%93%e8%ae %be%e8%ae%a1
https://www.iocoder.cn/Apollo/client-polling-config/