Design principles and source code analysis of the three-level cache of Spring Cloud Eureka source code analysis

In order to provide response efficiency, Eureka Server provides a two-tier cache structure. The registration information required by Eureka Client is directly stored in the cache structure. The implementation principle is shown in the figure below.

First-level cache : readOnlyCacheMap, essentially ConcurrentHashMap, relies on timing to synchronize data from readWriteCacheMap, the default time is 30 seconds.

readOnlyCacheMap: It is a CurrentHashMap read-only cache. This is mainly for the client to use when obtaining registration information. The cache update depends on the update of the timer. It is compared with the value of readWriteCacheMap. If the data is inconsistent, the data of readWriteCacheMap is used. Prevail.

Second-level cache : readWriteCacheMap, essentially Guava cache.

readWriteCacheMap: The data of readWriteCacheMap is mainly synchronized with the storage layer. When the cache is obtained, it is judged whether there is no data in the cache. If the data does not exist, it is loaded through the load method of CacheLoader. After the loading is successful, the data is put into the cache and the data is returned at the same time.

The readWriteCacheMap cache expiration time, the default is 180 seconds. When the service goes offline, expires, registers, and changes status, the data in this cache will be cleared.

When Eureka Client gets full or incremental data, it will first get it from the first-level cache; if it does not exist in the first-level cache, then it will get it from the second-level cache; if the second-level cache does not exist, the storage layer The data is synchronized to the cache, and then retrieved from the cache.

Through Eureka Server's two-tier caching mechanism, the response time of Eureka Server can be improved very effectively. Through the data segmentation of the data storage layer and the cache layer, different data support can be provided according to the usage scenario.

The meaning of multi-level caching

Why design a multi-level cache here? The reason is very simple. When there is a large-scale service registration and update, if only one ConcurrentHashMap data is modified, the existence of the lock will inevitably lead to competition and affect performance.

And Eureka is an AP model, which only needs to meet the final availability. So it uses multi-level cache to achieve read-write separation. When the registration method is written, directly write to the memory registry, and actively invalidate the read-write cache after the table is written.

The interface for obtaining registration information is first fetched from the read-only cache, the read-only cache does not go to the read-write cache again, and the read-write cache does not go to the memory registry again (not just fetching, here is more complicated). And, the read-write cache will update the write-back read-only cache

• responseCacheUpdateIntervalMs: readOnlyCacheMap cache update timer interval, the default is 30 seconds
• responseCacheAutoExpirationInSeconds: readWriteCacheMap cache expiration time, the default is 180 seconds.

Cache initialization

ReadWriteCacheMap uses the LoadingCache object, which is an api provided in guava to implement memory caching. The creation method is as follows

LoadingCache<Long, String> cache = CacheBuilder.newBuilder()
    //缓存池大小，在缓存项接近该大小时， Guava开始回收旧的缓存项
    .maximumSize(10000)
    //设置时间对象没有被读/写访问则对象从内存中删除(在另外的线程里面不定期维护)
    .expireAfterAccess(10, TimeUnit.MINUTES)
    //移除监听器,缓存项被移除时会触发
    .removalListener(new RemovalListener <Long, String>() {
        @Override
        public void onRemoval(RemovalNotification<Long, String> rn) {
            //执行逻辑操作
        }
    })
    .recordStats()//开启Guava Cache的统计功能
    .build(new CacheLoader<String, Object>() {
        @Override
        public Object load(String key) {
            //从 SQL或者NoSql 获取对象
        }
    });//CacheLoader类 实现自动加载

Among them, CacheLoader is used to realize the function of automatic cache loading. When the readWriteCacheMap.get(key) method is triggered, the 061c135307541e method will be CacheLoader.load back, and the instance data will be found in the service registration information according to the key for caching.

ResponseCacheImpl(EurekaServerConfig serverConfig, ServerCodecs serverCodecs, AbstractInstanceRegistry registry) {
    this.serverConfig = serverConfig;
    this.serverCodecs = serverCodecs;
    this.shouldUseReadOnlyResponseCache = serverConfig.shouldUseReadOnlyResponseCache();
    this.registry = registry;

    long responseCacheUpdateIntervalMs = serverConfig.getResponseCacheUpdateIntervalMs();
    this.readWriteCacheMap =
        CacheBuilder.newBuilder().initialCapacity(serverConfig.getInitialCapacityOfResponseCache())
        .expireAfterWrite(serverConfig.getResponseCacheAutoExpirationInSeconds(), TimeUnit.SECONDS)
        .removalListener(new RemovalListener<Key, Value>() {
            @Override
            public void onRemoval(RemovalNotification<Key, Value> notification) {
                Key removedKey = notification.getKey();
                if (removedKey.hasRegions()) {
                    Key cloneWithNoRegions = removedKey.cloneWithoutRegions();
                    regionSpecificKeys.remove(cloneWithNoRegions, removedKey);
                }
            }
        })
        .build(new CacheLoader<Key, Value>() {
            @Override
            public Value load(Key key) throws Exception {
                if (key.hasRegions()) {
                    Key cloneWithNoRegions = key.cloneWithoutRegions();
                    regionSpecificKeys.put(cloneWithNoRegions, key);
                }
                Value value = generatePayload(key);  //注意这里
                return value;
            }
        });

The loading of the cache is based on the generatePayload method, the code is as follows.

private Value generatePayload(Key key) {
    Stopwatch tracer = null;
    try {
        String payload;
        switch (key.getEntityType()) {
            case Application:
                boolean isRemoteRegionRequested = key.hasRegions();

                if (ALL_APPS.equals(key.getName())) {
                    if (isRemoteRegionRequested) {
                        tracer = serializeAllAppsWithRemoteRegionTimer.start();
                        payload = getPayLoad(key, registry.getApplicationsFromMultipleRegions(key.getRegions()));
                    } else {
                        tracer = serializeAllAppsTimer.start();
                        payload = getPayLoad(key, registry.getApplications());
                    }
                } else if (ALL_APPS_DELTA.equals(key.getName())) {
                    if (isRemoteRegionRequested) {
                        tracer = serializeDeltaAppsWithRemoteRegionTimer.start();
                        versionDeltaWithRegions.incrementAndGet();
                        versionDeltaWithRegionsLegacy.incrementAndGet();
                        payload = getPayLoad(key,
                                             registry.getApplicationDeltasFromMultipleRegions(key.getRegions()));
                    } else {
                        tracer = serializeDeltaAppsTimer.start();
                        versionDelta.incrementAndGet();
                        versionDeltaLegacy.incrementAndGet();
                        payload = getPayLoad(key, registry.getApplicationDeltas());
                    }
                } else {
                    tracer = serializeOneApptimer.start();
                    payload = getPayLoad(key, registry.getApplication(key.getName()));
                }
                break;
            case VIP:
            case SVIP:
                tracer = serializeViptimer.start();
                payload = getPayLoad(key, getApplicationsForVip(key, registry));
                break;
            default:
                logger.error("Unidentified entity type: {} found in the cache key.", key.getEntityType());
                payload = "";
                break;
        }
        return new Value(payload);
    } finally {
        if (tracer != null) {
            tracer.stop();
        }
    }
}

This method accepts a Key type 061c1353075502 and returns a type of Value Among Key , the important fields in 061c1353075506 are:

• KeyType represents the payload text format, which has two values: JSON and XML
• EntityType represents the type of cache. There are three values: Application , VIP , SVIP
• entityName represents the name of the cache, which may be a single application name, or it may be ALL_APPS or ALL_APPS_DELTA .

Value has a String type 061c135307562c and an byte , representing the bytes after gzip compression.

Cache synchronization

In ResponseCacheImpl , a timing task is initialized, and each timing task

ResponseCacheImpl(EurekaServerConfig serverConfig, ServerCodecs serverCodecs, AbstractInstanceRegistry registry) {
    //省略...
    if (shouldUseReadOnlyResponseCache) {
        timer.schedule(getCacheUpdateTask(),
                       new Date(((System.currentTimeMillis() / responseCacheUpdateIntervalMs) * responseCacheUpdateIntervalMs)
                                + responseCacheUpdateIntervalMs),
                       responseCacheUpdateIntervalMs);
    }
}

By default, every 30s updated from readWriteCacheMap to the readOnlyCacheMap.

private TimerTask getCacheUpdateTask() {
    return new TimerTask() {
        @Override
        public void run() {
            logger.debug("Updating the client cache from response cache");
            for (Key key : readOnlyCacheMap.keySet()) { //遍历只读集合
                if (logger.isDebugEnabled()) {
                    logger.debug("Updating the client cache from response cache for key : {} {} {} {}",
                                 key.getEntityType(), key.getName(), key.getVersion(), key.getType());
                }
                try {
                    CurrentRequestVersion.set(key.getVersion());
                    Value cacheValue = readWriteCacheMap.get(key);
                    Value currentCacheValue = readOnlyCacheMap.get(key);
                    if (cacheValue != currentCacheValue) { //判断差异信息，如果有差异，则更新
                        readOnlyCacheMap.put(key, cacheValue);
                    }
                } catch (Throwable th) {
                    logger.error("Error while updating the client cache from response cache for key {}", key.toStringCompact(), th);
                } finally {
                    CurrentRequestVersion.remove();
                }
            }
        }
    };
}

Cache invalidation

In the AbstractInstanceRegistry.register method, when the service information is saved, invalidateCache will be called to invalidate the cache

public void register(InstanceInfo registrant, int leaseDuration, boolean isReplication) {
    //....
     invalidateCache(registrant.getAppName(), registrant.getVIPAddress(), registrant.getSecureVipAddress());
    //....
}

Finally, the ResponseCacheImpl.invalidate method is called to complete the cache invalidation mechanism

public void invalidate(Key... keys) {
    for (Key key : keys) {
        logger.debug("Invalidating the response cache key : {} {} {} {}, {}",
                     key.getEntityType(), key.getName(), key.getVersion(), key.getType(), key.getEurekaAccept());

        readWriteCacheMap.invalidate(key);
        Collection<Key> keysWithRegions = regionSpecificKeys.get(key);
        if (null != keysWithRegions && !keysWithRegions.isEmpty()) {
            for (Key keysWithRegion : keysWithRegions) {
                logger.debug("Invalidating the response cache key : {} {} {} {} {}",
                             key.getEntityType(), key.getName(), key.getVersion(), key.getType(), key.getEurekaAccept());
                readWriteCacheMap.invalidate(keysWithRegion);
            }
        }
    }
}

Copyright statement: All articles in this blog, except for special statements, adopt the CC BY-NC-SA 4.0 license agreement. Please indicate the reprint from Mic takes you to learn architecture!
If this article is helpful to you, please help me to follow and like. Your persistence is the motivation for my continuous creation. Welcome to follow the WeChat public account of the same name for more technical dry goods!