Explain in detail the source code implementation of dynamic update of client configuration cache in Nacos configuration center

As a configuration center, when an application accesses Nacos to dynamically obtain the configuration source, it will be cached in local memory and disk.
Since Nacos acts as a dynamic configuration center, it means that all related clients need to be aware of subsequent configuration changes and update the local memory!
So where is this function implemented? And what way does it use to update the configuration? Let's explore the implementation of the source code together!

Client configuration cache update

When the client gets the configuration, it needs to be refreshed dynamically to ensure that the data is consistent with the server. How is this process implemented? In this section we will do a detailed analysis.

Nacos uses a long rotation training mechanism to synchronize data changes. The principle is as follows!

The overall workflow is as follows:

• The client initiates a long training request
• After the server receives the request, it first compares whether the data in the cache of the server is the same. If it does not work, it returns directly.
• If they are the same, then perform the comparison after a delay of 29.5s through the schedule
• In order to ensure that the server can notify the client in time when data changes occur within 29.5s, the server uses event subscription to monitor the local data change event of the server. Once an event is received, it will trigger the notification of DataChangeTask and traverse the ClientLongPolling in the allStubs queue, writes the result back to the client, and completes a data push
• What if the scheduled task in ClientLongPolling starts to execute again after the DataChangeTask task completes the "push" of data?
  It's very simple, as long as you cancel the original scheduling task waiting to be executed before performing the "push" operation, this prevents the scheduling task from writing the response data after the push operation finishes writing the response data. wrong. So, in the ClientLongPolling method, the first step is to delete the subscription event

Long rotation training task start entrance

In the constructor of NacosConfigService, when the class is instantiated, there are some things to do

• Initialize an HttpAgent, and the decoration mode is used here. The actual working class is ServerHttpAgent, and the method of ServerHttpAgent is also called internally by MetricsHttpAgent, which adds monitoring statistics information.
• ClientWorker, a work class of the client, the agent is passed as a parameter to the clientworker, and it can be basically guessed that the agent will be used to do some remote related things.

public NacosConfigService(Properties properties) throws NacosException {
    ValidatorUtils.checkInitParam(properties);
    String encodeTmp = properties.getProperty(PropertyKeyConst.ENCODE);
    if (StringUtils.isBlank(encodeTmp)) {
        this.encode = Constants.ENCODE;
    } else {
        this.encode = encodeTmp.trim();
    }
    initNamespace(properties); //
    this.configFilterChainManager = new ConfigFilterChainManager(properties);
    //初始化网络通信组件
    this.agent = new MetricsHttpAgent(new ServerHttpAgent(properties));
    this.agent.start(); 
    //初始化ClientWorker
    this.worker = new ClientWorker(this.agent, this.configFilterChainManager, properties);
}

ClientWorker

In the above initialization code, we need to focus on the ClientWorker class, whose construction method is as follows

public ClientWorker(final HttpAgent agent, final ConfigFilterChainManager configFilterChainManager,
        final Properties properties) {
    this.agent = agent;
    this.configFilterChainManager = configFilterChainManager; //初始化配置过滤管理器
    
    // Initialize the timeout parameter
    
    init(properties); //初始化配置
    
    //初始化一个定时调度的线程池，重写了threadfactory方法
    this.executor = Executors.newScheduledThreadPool(1, new ThreadFactory() {
        @Override
        public Thread newThread(Runnable r) {
            Thread t = new Thread(r);
            t.setName("com.alibaba.nacos.client.Worker." + agent.getName());
            t.setDaemon(true);
            return t;
        }
    });
    
     //初始化一个定时调度的线程池，从里面的name名字来看，似乎和长轮训有关系。而这个长轮训应该是和nacos服务端的长轮训
    this.executorService = Executors
            .newScheduledThreadPool(Runtime.getRuntime().availableProcessors(), new ThreadFactory() {
                @Override
                public Thread newThread(Runnable r) {
                    Thread t = new Thread(r);
                    t.setName("com.alibaba.nacos.client.Worker.longPolling." + agent.getName());
                    t.setDaemon(true);
                    return t;
                }
            });
    //设置定时任务的执行频率，并且调用checkConfigInfo这个方法，猜测是定时去检测配置是否发生了变化
        //首次执行延迟时间为1毫秒、延迟时间为10毫秒
    this.executor.scheduleWithFixedDelay(new Runnable() {
        @Override
        public void run() {
            try {
                checkConfigInfo();
            } catch (Throwable e) {
                LOGGER.error("[" + agent.getName() + "] [sub-check] rotate check error", e);
            }
        }
    }, 1L, 10L, TimeUnit.MILLISECONDS);
}

It can be seen that in addition to maintaining HttpAgent within itself, ClientWorker also creates two thread pools:

1. The first thread pool is an executor that only has one thread to perform timed tasks. The executor executes the checkConfigInfo() method every 10ms. From the method name, it can be known that the configuration information is checked every 10ms.
2. The second thread pool is an ordinary thread pool. From the name of ThreadFactory, we can see that this thread pool is used for long polling.

checkConfigInfo

During the initialization of the ClientWorker structure, a scheduled task is started to execute the checkConfigInfo() method. This method is mainly to regularly check the changes of the local configuration and the configuration on the server. This method is defined as follows.

public void checkConfigInfo() {
    // Dispatch tasks.
    int listenerSize = cacheMap.size(); //
    // Round up the longingTaskCount.
     // 向上取整为批数，监听的配置数量除以3000，得到一个整数，代表长轮训任务的数量
    int longingTaskCount = (int) Math.ceil(listenerSize / ParamUtil.getPerTaskConfigSize());
     //currentLongingTaskCount表示当前的长轮训任务数量，如果小于计算的结果，则可以继续创建
    if (longingTaskCount > currentLongingTaskCount) {
        for (int i = (int) currentLongingTaskCount; i < longingTaskCount; i++) {
            // The task list is no order.So it maybe has issues when changing.
            executorService.execute(new LongPollingRunnable(i));
        }
        currentLongingTaskCount = longingTaskCount;
    }
}

The main purpose of this method is to check whether the configuration information of the server has changed. If there is a change, trigger the listener notification

• cacheMap : AtomicReference<Map<String, CacheData>> cacheMap is used to store the cache collection for monitoring changes. The key is the value concatenated according to dataID/group/tenant (tenant). Value is the content corresponding to the configuration file stored on the nacos server.
• By default, each LongPullingRunnable task processes 3000 listening profiles by default. If it exceeds 3000, you need to start multiple LongPollingRunnables to execute.
• currentLongingTaskCount holds the number of LongPullingRunnable tasks that have been started
• executorService is the thread pool initialized in the ClientWorker constructor

LongPollingRunnable.run

The implementation logic of the LongPollingRunnable long rotation training task, the code is relatively long, we will analyze it in sections.

The first part mainly has two logics

1. Sort tasks into batches
2. Check whether the cache of the current batch is consistent with the data of the local file, and if there is a change, trigger the monitoring.

class LongPollingRunnable implements Runnable {
    
    private final int taskId; //表示当前任务批次id
    
    public LongPollingRunnable(int taskId) {
        this.taskId = taskId;
    }
    
    @Override
    public void run() {
        
        List<CacheData> cacheDatas = new ArrayList<CacheData>();
        List<String> inInitializingCacheList = new ArrayList<String>();
        try {
            // 遍历CacheMap，把CacheMap中和当前任务id相同的缓存，保存到cacheDatas
            // 通过checkLocalConfig方法
            for (CacheData cacheData : cacheMap.values()) {
                if (cacheData.getTaskId() == taskId) {
                    cacheDatas.add(cacheData);
                    try {
                        checkLocalConfig(cacheData);
                        if (cacheData.isUseLocalConfigInfo()) { //这里表示数据有变化，需要通知监听器
                            cacheData.checkListenerMd5(); //通知所有针对当前配置设置了监听的监听器
                        }
                    } catch (Exception e) {
                        LOGGER.error("get local config info error", e);
                    }
                }
            }
           //省略部分
            
        } catch (Throwable e) {
            
            // If the rotation training task is abnormal, the next execution time of the task will be punished
            LOGGER.error("longPolling error : ", e);
            executorService.schedule(this, taskPenaltyTime, TimeUnit.MILLISECONDS); //出现异常，到下一次taskPenaltyTime后重新执行任务
        }
    }
}

checkLocalConfig

Check the local configuration, there are three situations

• If isUseLocalConfigInfo is false, it means that the local configuration is not used, but the file of the local cache path exists, so set isUseLocalConfigInfo to true, and update the content of cacheData and the update time of the file
• If isUseLocalConfigInfo is true, it means that the local configuration file is used, but the local cache file does not exist, then it is set to false and the listener is not notified.
• If isUseLocalConfigInfo is true and the local cache file also exists, but the cached time is inconsistent with the file update time, the content in cacheData is updated and isUseLocalConfigInfo is set to true.

private void checkLocalConfig(CacheData cacheData) {
    final String dataId = cacheData.dataId;
    final String group = cacheData.group;
    final String tenant = cacheData.tenant;
    File path = LocalConfigInfoProcessor.getFailoverFile(agent.getName(), dataId, group, tenant);
    // 没有 -> 有
    if (!cacheData.isUseLocalConfigInfo() && path.exists()) {
        String content = LocalConfigInfoProcessor.getFailover(agent.getName(), dataId, group, tenant);
        final String md5 = MD5Utils.md5Hex(content, Constants.ENCODE);
        cacheData.setUseLocalConfigInfo(true);
        cacheData.setLocalConfigInfoVersion(path.lastModified());
        cacheData.setContent(content);
        String encryptedDataKey = LocalEncryptedDataKeyProcessor
                .getEncryptDataKeyFailover(agent.getName(), dataId, group, tenant);
        cacheData.setEncryptedDataKey(encryptedDataKey);
        
        LOGGER.warn(
                "[{}] [failover-change] failover file created. dataId={}, group={}, tenant={}, md5={}, content={}",
                agent.getName(), dataId, group, tenant, md5, ContentUtils.truncateContent(content));
        return;
    }
     // 有 -> 没有。不通知业务监听器，从server拿到配置后通知。
    // If use local config info, then it doesn't notify business listener and notify after getting from server.
    if (cacheData.isUseLocalConfigInfo() && !path.exists()) {
        cacheData.setUseLocalConfigInfo(false);
        LOGGER.warn("[{}] [failover-change] failover file deleted. dataId={}, group={}, tenant={}", agent.getName(),
                dataId, group, tenant);
        return;
    }
    
     // 有变更
    if (cacheData.isUseLocalConfigInfo() && path.exists() && cacheData.getLocalConfigInfoVersion() != path
            .lastModified()) {
        String content = LocalConfigInfoProcessor.getFailover(agent.getName(), dataId, group, tenant);
        final String md5 = MD5Utils.md5Hex(content, Constants.ENCODE);
        cacheData.setUseLocalConfigInfo(true);
        cacheData.setLocalConfigInfoVersion(path.lastModified());
        cacheData.setContent(content);
        String encryptedDataKey = LocalEncryptedDataKeyProcessor
                .getEncryptDataKeyFailover(agent.getName(), dataId, group, tenant);
        cacheData.setEncryptedDataKey(encryptedDataKey);
        LOGGER.warn(
                "[{}] [failover-change] failover file changed. dataId={}, group={}, tenant={}, md5={}, content={}",
                agent.getName(), dataId, group, tenant, md5, ContentUtils.truncateContent(content));
    }
}

checkListenerMd5

Traverse the listeners added by the user, and send a notification if the md5 value of the data is found to be different

void checkListenerMd5() {
    for (ManagerListenerWrap wrap : listeners) {
        if (!md5.equals(wrap.lastCallMd5)) {
            safeNotifyListener(dataId, group, content, type, md5, wrap);
        }
    }
}

Check server configuration

In LongPollingRunnable.run, first determine whether the data has changed by reading and checking the local configuration to realize the notification of the change

Then, the current thread also needs to go to the remote server to get the latest data and check which data has changed

• Obtain the dataid of the data change on the remote server through checkUpdateDataIds
• Traverse the collection of these changes, and then call getServerConfig to get the corresponding content from the remote server
• Update the local cache and set it to the content returned by the server
• Finally, traverse the cacheDatas and find the changed data for notification

// check server config
//从服务端获取发生变化的数据的DataID列表，保存在List<String>集合中
List<String> changedGroupKeys = checkUpdateDataIds(cacheDatas, inInitializingCacheList);
if (!CollectionUtils.isEmpty(changedGroupKeys)) {
    LOGGER.info("get changedGroupKeys:" + changedGroupKeys);
}
 
//遍历发生了变更的配置项
for (String groupKey : changedGroupKeys) {
    String[] key = GroupKey.parseKey(groupKey);
    String dataId = key[0];
    String group = key[1];
    String tenant = null;
    if (key.length == 3) {
        tenant = key[2];
    }
    try {
        //逐项根据这些配置项获取配置信息
        ConfigResponse response = getServerConfig(dataId, group, tenant, 3000L);
        //把配置信息保存到CacheData中
        CacheData cache = cacheMap.get(GroupKey.getKeyTenant(dataId, group, tenant));
        cache.setContent(response.getContent());
        cache.setEncryptedDataKey(response.getEncryptedDataKey());
        if (null != response.getConfigType()) {
            cache.setType(response.getConfigType());
        }
        LOGGER.info("[{}] [data-received] dataId={}, group={}, tenant={}, md5={}, content={}, type={}",
                    agent.getName(), dataId, group, tenant, cache.getMd5(),
                    ContentUtils.truncateContent(response.getContent()), response.getConfigType());
    } catch (NacosException ioe) {
        String message = String
            .format("[%s] [get-update] get changed config exception. dataId=%s, group=%s, tenant=%s",
                    agent.getName(), dataId, group, tenant);
        LOGGER.error(message, ioe);
    }
}
//再遍历CacheData这个集合，找到发生变化的数据进行通知
for (CacheData cacheData : cacheDatas) {
    if (!cacheData.isInitializing() || inInitializingCacheList
        .contains(GroupKey.getKeyTenant(cacheData.dataId, cacheData.group, cacheData.tenant))) {
        cacheData.checkListenerMd5();
        cacheData.setInitializing(false);
    }
}
inInitializingCacheList.clear();
 //继续传递当前线程进行轮询
executorService.execute(this);

checkUpdateDataIds

This method mainly initiates a check request to the server to determine whether its local configuration is consistent with the server configuration.

• First find the cache with isUseLocalConfigInfo false from the cacheDatas collection
• Splice the configuration items that need to be checked into a string, and call checkUpdateConfigStr to verify

/**
 * 从Server获取值变化了的DataID列表。返回的对象里只有dataId和group是有效的。 保证不返回NULL。
 */
List<String> checkUpdateDataIds(List<CacheData> cacheDatas, List<String> inInitializingCacheList) throws IOException {
    StringBuilder sb = new StringBuilder();
    for (CacheData cacheData : cacheDatas) { //把需要检查的配置项，拼接成一个字符串
        if (!cacheData.isUseLocalConfigInfo()) { //找到isUseLocalConfigInfo=false的缓存
            sb.append(cacheData.dataId).append(WORD_SEPARATOR);
            sb.append(cacheData.group).append(WORD_SEPARATOR);
            if (StringUtils.isBlank(cacheData.tenant)) {
                sb.append(cacheData.getMd5()).append(LINE_SEPARATOR);
            } else {
                sb.append(cacheData.getMd5()).append(WORD_SEPARATOR);
                sb.append(cacheData.getTenant()).append(LINE_SEPARATOR);
            }
            if (cacheData.isInitializing()) {//
                // cacheData 首次出现在cacheMap中&首次check更新
                inInitializingCacheList
                    .add(GroupKey.getKeyTenant(cacheData.dataId, cacheData.group, cacheData.tenant));
            }
        }
    }
    boolean isInitializingCacheList = !inInitializingCacheList.isEmpty();
    return checkUpdateConfigStr(sb.toString(), isInitializingCacheList);
}

checkUpdateConfigStr

Get a list of DataIDs whose values have changed from the Server. Only dataId and group are valid in the returned object. Guaranteed not to return NULL.

List<String> checkUpdateConfigStr(String probeUpdateString, boolean isInitializingCacheList) throws Exception {
    
    //拼接参数和header
    Map<String, String> params = new HashMap<String, String>(2);
    params.put(Constants.PROBE_MODIFY_REQUEST, probeUpdateString);
    Map<String, String> headers = new HashMap<String, String>(2);
    headers.put("Long-Pulling-Timeout", "" + timeout);
    
    // told server do not hang me up if new initializing cacheData added in
    if (isInitializingCacheList) {
        headers.put("Long-Pulling-Timeout-No-Hangup", "true");
    }
    
    if (StringUtils.isBlank(probeUpdateString)) {//判断可能发生变更的字符串是否为空，如果是，则直接返回。
        return Collections.emptyList();
    }
    
    try {
        // In order to prevent the server from handling the delay of the client's long task,
        // increase the client's read timeout to avoid this problem.
        // 设置readTimeoutMs，也就是本次请求等待响应的超时时间，默认是30s
        long readTimeoutMs = timeout + (long) Math.round(timeout >> 1);
        //发起远程调用
        HttpRestResult<String> result = agent
                .httpPost(Constants.CONFIG_CONTROLLER_PATH + "/listener", headers, params, agent.getEncode(),
                        readTimeoutMs);
        
        if (result.ok()) { //如果响应成功
            setHealthServer(true);
            return parseUpdateDataIdResponse(result.getData()); //解析并更新数据，返回的是确实发生了数据变更的字符串：tenant/group/dataid。
        } else {//如果响应失败
            setHealthServer(false);
            LOGGER.error("[{}] [check-update] get changed dataId error, code: {}", agent.getName(),
                    result.getCode());
        }
    } catch (Exception e) {
        setHealthServer(false);
        LOGGER.error("[" + agent.getName() + "] [check-update] get changed dataId exception", e);
        throw e;
    }
    return Collections.emptyList();
}

Summary of long rotation training mechanism for client cache configuration

The core point of the overall implementation is just a few parts

1. Split the task for the configuration of the local cache, each batch is 3000
2. Create a thread for every 3000 to execute

3. First compare the cache of each batch with the data in the local disk file,

   1. If it is inconsistent with the local configuration, it means that the cache has been updated, and the client is directly notified to listen
   2. If the local cache and disk data are consistent, you need to initiate a remote request to check for configuration changes
4. First splicing tenent/groupId/dataId into a string, sending it to the server for inspection, and returning the changed configuration
5. The client receives the change configuration list, and then traverses it item by item and sends it to the server to obtain the configuration content.

Server configuration update push

After analyzing the client, driven by curiosity, how does the server handle the client's request? Then again, we need to think about a few questions

• How does the server implement the long rotation training mechanism
• Why should the client's timeout period be set to 30s?

The request address initiated by the client is: /v1/cs/configs/listener , so I found this interface for viewing, the code is as follows.

//# ConfigController.java
@PostMapping("/listener")
@Secured(action = ActionTypes.READ, parser = ConfigResourceParser.class)
public void listener(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {
    request.setAttribute("org.apache.catalina.ASYNC_SUPPORTED", true);
    String probeModify = request.getParameter("Listening-Configs");
    if (StringUtils.isBlank(probeModify)) {
        throw new IllegalArgumentException("invalid probeModify");
    }
    
    probeModify = URLDecoder.decode(probeModify, Constants.ENCODE);
    
    Map<String, String> clientMd5Map;
    try {
        //解析客户端传递过来的可能发生变化的配置项目，转化为Map集合(key=dataId,value=md5)
        clientMd5Map = MD5Util.getClientMd5Map(probeModify);
    } catch (Throwable e) {
        throw new IllegalArgumentException("invalid probeModify");
    }
    
    // 开始执行长轮训。
    inner.doPollingConfig(request, response, clientMd5Map, probeModify.length());
}

doPollingConfig

This method is mainly used to judge long polling and short polling.

1. If it is a long rotation training, go directly to the addLongPollingClient method
2. If it is short polling, compare the data on the server directly. If there is an inconsistency in md5, return the data directly.

public String doPollingConfig(HttpServletRequest request, HttpServletResponse response,
        Map<String, String> clientMd5Map, int probeRequestSize) throws IOException {
    
    // 判断当前请求是否支持长轮训。（）
    if (LongPollingService.isSupportLongPolling(request)) {
        longPollingService.addLongPollingClient(request, response, clientMd5Map, probeRequestSize);
        return HttpServletResponse.SC_OK + "";
    }
    
    //如果是短轮询，走下面的请求，下面的请求就是把客户端传过来的数据和服务端的数据逐项进行比较，保存到changeGroups中。
    // Compatible with short polling logic.
    List<String> changedGroups = MD5Util.compareMd5(request, response, clientMd5Map);
    
    // Compatible with short polling result.
    String oldResult = MD5Util.compareMd5OldResult(changedGroups);
    String newResult = MD5Util.compareMd5ResultString(changedGroups);
    
    String version = request.getHeader(Constants.CLIENT_VERSION_HEADER);
    if (version == null) {
        version = "2.0.0";
    }
    int versionNum = Protocol.getVersionNumber(version);
    
    // Before 2.0.4 version, return value is put into header.
    if (versionNum < START_LONG_POLLING_VERSION_NUM) {
        response.addHeader(Constants.PROBE_MODIFY_RESPONSE, oldResult);
        response.addHeader(Constants.PROBE_MODIFY_RESPONSE_NEW, newResult);
    } else {
        request.setAttribute("content", newResult);
    }
    
    Loggers.AUTH.info("new content:" + newResult);
    
    // Disable cache.
    response.setHeader("Pragma", "no-cache");
    response.setDateHeader("Expires", 0);
    response.setHeader("Cache-Control", "no-cache,no-store");
    response.setStatus(HttpServletResponse.SC_OK);
    return HttpServletResponse.SC_OK + "";
}

addLongPollingClient

Save the client's request to the execution engine of the long-round training.

public void addLongPollingClient(HttpServletRequest req, HttpServletResponse rsp, Map<String, String> clientMd5Map,
        int probeRequestSize) {
    //获取客户端长轮训的超时时间
    String str = req.getHeader(LongPollingService.LONG_POLLING_HEADER); 
    //不允许断开的标记
    String noHangUpFlag = req.getHeader(LongPollingService.LONG_POLLING_NO_HANG_UP_HEADER);
    //应用名称
    String appName = req.getHeader(RequestUtil.CLIENT_APPNAME_HEADER);
    //
    String tag = req.getHeader("Vipserver-Tag");
    //延期时间，默认为500ms
    int delayTime = SwitchService.getSwitchInteger(SwitchService.FIXED_DELAY_TIME, 500);

    // Add delay time for LoadBalance, and one response is returned 500 ms in advance to avoid client timeout.
    // 提前500ms返回一个响应，避免客户端出现超时
    long timeout = Math.max(10000, Long.parseLong(str) - delayTime);
    if (isFixedPolling()) {
        timeout = Math.max(10000, getFixedPollingInterval());
        // Do nothing but set fix polling timeout.
    } else {
        long start = System.currentTimeMillis();
        //通过md5判断客户端请求过来的key是否有和服务器端有不一致的，如果有，则保存到changedGroups中。
        List<String> changedGroups = MD5Util.compareMd5(req, rsp, clientMd5Map);
        if (changedGroups.size() > 0) { //如果发现有变更，则直接把请求返回给客户端
            generateResponse(req, rsp, changedGroups);
            LogUtil.CLIENT_LOG.info("{}|{}|{}|{}|{}|{}|{}", System.currentTimeMillis() - start, "instant",
                    RequestUtil.getRemoteIp(req), "polling", clientMd5Map.size(), probeRequestSize,
                    changedGroups.size());
            return;
        } else if (noHangUpFlag != null && noHangUpFlag.equalsIgnoreCase(TRUE_STR)) { //如果noHangUpFlag为true，说明不需要挂起客户端，所以直接返回。
            LogUtil.CLIENT_LOG.info("{}|{}|{}|{}|{}|{}|{}", System.currentTimeMillis() - start, "nohangup",
                    RequestUtil.getRemoteIp(req), "polling", clientMd5Map.size(), probeRequestSize,
                    changedGroups.size());
            return;
        }
    }
    //获取请求端的ip
    String ip = RequestUtil.getRemoteIp(req);

    // Must be called by http thread, or send response.
    //把当前请求转化为一个异步请求（意味着此时tomcat线程被释放，也就是客户端的请求，需要通过asyncContext来手动触发返回，否则一直挂起）
    final AsyncContext asyncContext = req.startAsync();
    // AsyncContext.setTimeout() is incorrect, Control by oneself
    asyncContext.setTimeout(0L); //设置异步请求超时时间，
    //执行长轮训请求
    ConfigExecutor.executeLongPolling(
            new ClientLongPolling(asyncContext, clientMd5Map, ip, probeRequestSize, timeout, appName, tag));
}

ClientLongPolling

Next, let's analyze what clientLongPolling does. Or we can first guess what should be done

• This task can only be executed after blocking for 29.5s, because there is no point in executing it immediately, after all, it has been executed once before
• If the data changes within 29.5s+, advance notice is required. There needs to be a monitoring mechanism

Based on these conjectures, we can look at its implementation process

From the coarse-grained point of view of the code, its implementation seems to be consistent with our conjecture. In the run method, a scheduled task is implemented through scheduler.schedule, and its delay time is exactly 29.5s as calculated earlier. In this task, it will be calculated by MD5Util.compareMd5

The other one, when the data changes, it must not be notified until 29.5s later, so what should I do? We found something like allSubs which seems to be related to publish subscribe. Is it possible that the current clientLongPolling subscribes to the event of data change?

class ClientLongPolling implements Runnable {

    @Override
    public void run() {
        //构建一个异步任务，延后29.5s执行
        asyncTimeoutFuture = ConfigExecutor.scheduleLongPolling(new Runnable() {
            @Override
            public void run() { //如果达到29.5s，说明这个期间没有做任何配置修改，则自动触发执行
                try {
                    getRetainIps().put(ClientLongPolling.this.ip, System.currentTimeMillis());

                    // Delete subsciber's relations.
                    allSubs.remove(ClientLongPolling.this); //移除订阅关系

                    if (isFixedPolling()) { //如果是固定间隔的长轮训
                        LogUtil.CLIENT_LOG
                                .info("{}|{}|{}|{}|{}|{}", (System.currentTimeMillis() - createTime), "fix",
                                        RequestUtil.getRemoteIp((HttpServletRequest) asyncContext.getRequest()),
                                        "polling", clientMd5Map.size(), probeRequestSize);
                        //比较变更的key
                        List<String> changedGroups = MD5Util
                                .compareMd5((HttpServletRequest) asyncContext.getRequest(),
                                        (HttpServletResponse) asyncContext.getResponse(), clientMd5Map);
                        if (changedGroups.size() > 0) {//如果大于0，表示有变更，直接响应
                            sendResponse(changedGroups);
                        } else {
                            sendResponse(null); //否则返回null
                        }
                    } else {
                        LogUtil.CLIENT_LOG
                                .info("{}|{}|{}|{}|{}|{}", (System.currentTimeMillis() - createTime), "timeout",
                                        RequestUtil.getRemoteIp((HttpServletRequest) asyncContext.getRequest()),
                                        "polling", clientMd5Map.size(), probeRequestSize);
                        sendResponse(null);
                    }
                } catch (Throwable t) {
                    LogUtil.DEFAULT_LOG.error("long polling error:" + t.getMessage(), t.getCause());
                }

            }

        }, timeoutTime, TimeUnit.MILLISECONDS);

        allSubs.add(this);  //把当前线程添加到订阅事件队列中
    }
}

allSubs

allSubs is a queue in which the ClientLongPolling object is placed. This queue appears to be somehow related to configuration changes.

So what must be realized here is that when the user modifies the configuration in the nacos console, he must remove the concerned client long connection from the subscription relationship, and then return the result of the change. So let's go to the constructor of LongPollingService to find the subscription relationship

/**
 * 长轮询订阅关系
 */
final Queue<ClientLongPolling> allSubs;

allSubs.add(this);

LongPollingService

In the construction method of LongPollingService, a NotifyCenter is used to subscribe to an event. It is not difficult to find that if the instance of this event is LocalDataChangeEvent, that is, when the server data changes, a thread of DataChangeTask will be executed.

public LongPollingService() {
    allSubs = new ConcurrentLinkedQueue<ClientLongPolling>();

    ConfigExecutor.scheduleLongPolling(new StatTask(), 0L, 10L, TimeUnit.SECONDS);

    // Register LocalDataChangeEvent to NotifyCenter.
    NotifyCenter.registerToPublisher(LocalDataChangeEvent.class, NotifyCenter.ringBufferSize);

    //注册LocalDataChangeEvent订阅事件
    NotifyCenter.registerSubscriber(new Subscriber() {

        @Override
        public void onEvent(Event event) {
            if (isFixedPolling()) {
                // Ignore.
            } else {
                if (event instanceof LocalDataChangeEvent) { //如果触发了LocalDataChangeEvent，则执行下面的代码
                    LocalDataChangeEvent evt = (LocalDataChangeEvent) event;
                    ConfigExecutor.executeLongPolling(new DataChangeTask(evt.groupKey, evt.isBeta, evt.betaIps));
                }
            }
        }

        @Override
        public Class<? extends Event> subscribeType() {
            return LocalDataChangeEvent.class;
        }
    });

}

DataChangeTask

Data change event thread, the code is as follows

class DataChangeTask implements Runnable {

    @Override
    public void run() {
        try {
            ConfigCacheService.getContentBetaMd5(groupKey); //
            //遍历所有订阅事件表
            for (Iterator<ClientLongPolling> iter = allSubs.iterator(); iter.hasNext(); ) {
                ClientLongPolling clientSub = iter.next(); //得到ClientLongPolling
                //判断当前的ClientLongPolling中，请求的key是否包含当前修改的groupKey
                if (clientSub.clientMd5Map.containsKey(groupKey)) {
                    // If published tag is not in the beta list, then it skipped.
                    if (isBeta && !CollectionUtils.contains(betaIps, clientSub.ip)) { //如果是beta方式且betaIps不包含当前客户端ip，直接返回
                        continue;
                    }

                    // If published tag is not in the tag list, then it skipped.
                    if (StringUtils.isNotBlank(tag) && !tag.equals(clientSub.tag)) {//如果配置了tag标签且不包含当前客户端的tag，直接返回
                        continue;
                    }
                    //
                    getRetainIps().put(clientSub.ip, System.currentTimeMillis());
                    iter.remove(); // Delete subscribers' relationships. 移除当前客户端的订阅关系
                    LogUtil.CLIENT_LOG
                            .info("{}|{}|{}|{}|{}|{}|{}", (System.currentTimeMillis() - changeTime), "in-advance",
                                    RequestUtil
                                            .getRemoteIp((HttpServletRequest) clientSub.asyncContext.getRequest()),
                                    "polling", clientSub.clientMd5Map.size(), clientSub.probeRequestSize, groupKey);
                    clientSub.sendResponse(Arrays.asList(groupKey)); //响应客户端请求。
                }
            }
        } catch (Throwable t) {
            LogUtil.DEFAULT_LOG.error("data change error: {}", ExceptionUtil.getStackTrace(t));
        }
    }
}

Principle summary

Copyright notice: All articles in this blog are licensed under CC BY-NC-SA 4.0 unless otherwise stated. Reprint please indicate from Mic takes you to learn architecture!
If this article is helpful to you, please help to follow and like, your persistence is the driving force for my continuous creation. Welcome to follow the WeChat public account of the same name to get more technical dry goods!