Talk about how to synchronize data to apollo configuration center

foreword

Friends who have implemented microservice projects should not be unfamiliar with the configuration center. The configuration center can be used to centrally manage the configuration, and can also perform hot update of the configuration. At present, the common configuration centers in the market include QConf, spring-cloud-config, diamond, disconf, apollo, nacos, etc. The most commonly used microservice projects should be spring-cloud-config, apollo, and nacos.

We may have such an application scenario that some configuration data is first dropped to the database, and then the data is persisted to the configuration center. This can be divided into two steps. The first step is to store the data in the database, and the second step is to manually write the data to the configuration center through the panel provided by the configuration center. However, we may be more inclined to synchronize the data directly to the configuration center after the data is stored in the database. Take apollo as an example today to talk about how to synchronize data to apollo configuration center

Realize ideas

Use the open API provided by apollo to operate

Implementation steps

1. Connect our application to the Apollo open platform

The Apollo administrator creates a third-party application at http://{portal_address}/open/manage.html, and it is best to check whether the AppId has been created before creating it. After the creation is successful, a token will be generated, as shown in the following figure:

2. Authorize the registered application

The Apollo administrator assigns tokens to the http://{portal_address}/open/manage.html page. After authorization, the application can manage the configuration of the authorized Namespace through the Http REST interface provided by Apollo.

3. The application calls the Apollo Open API

demo

Take synchronizing API gateway routing information to apollo as an example

1. Create third-party applications

Prompt token after creation

2. The appId that authorizes the third-party application to operate based on the token

We authorize to operate all configurations on the API gateway, and the authorization type is APP

3. Call Apollo Open API through apollo-openapi

The pom import apollo-openapi coordinates in the project

<dependency>
    <groupId>com.ctrip.framework.apollo</groupId>
    <artifactId>apollo-openapi</artifactId>
    <version>1.7.0</version>
</dependency>

After the introduction, we can directly operate the apollo open api

a, query configuration item

  public long getMaxRouteRuleIndex(){
        OpenNamespaceDTO openNamespaceDTO = apolloOpenApiClient.getNamespace(appInfoProperties.getAppId(),appInfoProperties.getEnv(),appInfoProperties.getClusterName(),appInfoProperties.getNameSpaceName());
        List<OpenItemDTO> items = openNamespaceDTO.getItems();
        if(CollectionUtils.isEmpty(items)){
            return 0;
        }
        return items.stream().filter(item -> item.getKey().matches(ID_PATTERN)).count();
    }

run unit tests

  @Test
    public void testGetMaxRouteRuleIndex(){
        long index = routeService.getMaxRouteRuleIndex();
        Assert.assertTrue(index >= 0);
    }

The apollo panel on the gateway at this time

b, create and publish configuration items

Note: Creation and publishing of apollo are two different APIs

  public boolean createRouteRule(RouteRule routeRule){
        try {
            long curRouteRuleIndex = getMaxRouteRuleIndex();
            buildOpenItemDTO(ROUTE_ID_KEY,curRouteRuleIndex,routeRule.getRouteId(),true);
            buildOpenItemDTO(ROUTE_URI_KEY,curRouteRuleIndex,routeRule.getUri(),true);
            buildOpenItemDTO(ROUTE_PREDICATES_KEY,curRouteRuleIndex,routeRule.getPredicate(),true);
            buildOpenItemDTO(ROUTE_FILTERS_KEY,curRouteRuleIndex,routeRule.getFilter(),true);
            return publish("新增网关路由","新增网关路由");
        } catch (Exception e) {
           log.error("{}",e.getMessage());
        }
        return false;
    }

run unit tests

  @Test
    public void testCreateRouteRule(){
        RouteRule routeRule = RouteRule.builder().routeId(appName)
                .uri("http://localhost:8082")
                .predicate("Path=/dashboard/**")
                .filter("StripPrefix=1").build();

        boolean isSuccess = routeService.createRouteRule(routeRule);
        Assert.assertTrue(isSuccess);
    }

View the panel of the api gateway on the apollo portal

A routing configuration is found. Because the api gateway does dynamic routing, the following output can be found from the console of the api gateway

visit your browser

Dynamic routing takes effect

b, update and release configuration item

 public boolean updateRouteRule(RouteRule routeRule){
        long ruleIndex = getRouteRuleIndex(routeRule.getRouteId());
        if(ruleIndex != -1){
            try {
                buildOpenItemDTO(ROUTE_URI_KEY,ruleIndex,routeRule.getUri(),false);
                buildOpenItemDTO(ROUTE_PREDICATES_KEY,ruleIndex,routeRule.getPredicate(),false);
                buildOpenItemDTO(ROUTE_FILTERS_KEY,ruleIndex,routeRule.getFilter(),false);
                return publish("更新网关路由","更新网关路由");
            } catch (Exception e) {
                log.error("{}",e.getMessage());
            }
        }

        return false;
    }

run unit tests

    @Test
    public void testUpdateRouteRule(){
        RouteRule routeRule = RouteRule.builder().routeId(appName)
                .uri("http://localhost:8082")
                .predicate("Path=/xxx/**")
                .filter("StripPrefix=1").build();
        boolean isSuccess = routeService.updateRouteRule(routeRule);
        Assert.assertTrue(isSuccess);
    }

View the panel of the api gateway on the apollo portal

It can be found that the Path of the predicate has been changed to xxx at this time

View API Gateway Console

Visit the browser, originally visit http://localhost:8000/dashboard/ops/index will appear

Change to visit http://localhost:8000/xxx/ops/index

Indicates that the route has been successfully changed

b, delete and publish the configuration item

 public boolean deleteRouteRule(String routeId){
        long ruleIndex = getRouteRuleIndex(routeId);

        if(ruleIndex != -1){
            try {
//                removeRouteItem(ROUTE_URI_KEY,ruleIndex);
//                removeRouteItem(ROUTE_PREDICATES_KEY,ruleIndex);
//                removeRouteItem(ROUTE_FILTERS_KEY,ruleIndex);
                buildOpenItemDTO(ROUTE_URI_KEY,ruleIndex,"http://null",false);
                buildOpenItemDTO(ROUTE_PREDICATES_KEY,ruleIndex,"Path=/-9999",false);
                return publish("删除网关路由","删除网关路由");
            } catch (Exception e) {
                log.error("{}",e.getMessage());
            }
        }

        return false;
    }

private void removeRouteItem(String key,long index){
        if(key.equalsIgnoreCase(ROUTE_PREDICATES_KEY) || key.equalsIgnoreCase(ROUTE_FILTERS_KEY)){
            key = String.format(key,index,0);
        }else{
            key = String.format(key,index);
        }
        apolloOpenApiClient.removeItem(appInfoProperties.getAppId(),appInfoProperties.getEnv(),appInfoProperties.getClusterName(),appInfoProperties.getNameSpaceName(),key,appInfoProperties.getAuthUser());
    }

Note: Because the gateway deletion is relatively complicated and involves recalculation of the route set, here is a trick to update the inaccessible route. If it is a physical deletion, just call apollo's removeItem

Summarize

The API provided by the apollo open platform is actually an http restful operation, which provides a series of addition, deletion, modification, and query operations. A small detail here is that the additions, deletions and releases of apollo are separate operations. If you only call additions, deletions, and changes, you need to publish on the portal, or use the publish interface to operate. For more details, see apollo's open platform link

https://www.apolloconfig.com/#/zh/usage/apollo-open-api-platform?id=_3210-%e6%96%b0%e5%a2%9e%e9%85%8d%e7%bd%ae%e6%8e%a5%e5%8f%a3

The examples provided in this article are for reference only and cannot be used directly in the production environment. If a friend's configuration center uses nacos, similar operations can be achieved. Because nacos also provides open api interface, interested friends can check the following link

https://nacos.io/zh-cn/docs/open-api.html

demo link

https://github.com/lyb-geek/springboot-learning/tree/master/springboot-sync-apollo