Cache hotspots, cache penetration, look at the ultimate solution

background

Not long ago, because of the company's business needs, it was necessary to solve the hotspot caching problem of the back-end business in the big promotion scenario, so the solution of the hotspot cache was studied.

Many companies’ caching is based on redis. The performance of redis is actually enough to cope with most scenarios. However, for a certain explosive product during a big promotion or during an event, a large number of inflows may appear in a few seconds. Because the data of a certain explosive product will be stored in a redis shard according to the hash rule in the redis cluster scenario, then the traffic of these few seconds will be pressed into the redis shard, which will cause the redis shard in an instant The paralysis of the film will also affect the blocking of subsequent redis requests.

Another scenario is that not all server-side logic of the company has a cache. When the traffic is up, these hot keys will still be pressed to the database level. Cause stress.

solution

The common solution is to increase the secondary cache, and write a copy of the hot data to the jvm. Set the expiration time. But when to set it, how to detect the hotspot, how to set the rules, and how much to set the expiration time. Even how to land quickly, this is a question that needs to be studied.

We hope to have a unified solution to solve these problems.

We discovered the open source framework Hotkey.

Hotkey originated from JD.com. Hotkey can automatically detect any sudden and unpredictable hot data in milliseconds according to the configured rules. The detected hot data will be pushed to all server JVMs, which greatly reduces the need for post-processing. The impact of the end data layer. These hot data will be consistent throughout the microservice cluster, and when the hotspot disappears, it will be automatically removed from the jvm.

Hotkey's characteristics are very good to achieve our goals. And JD.com also used Hotkey to fight 618 big promotions internally, and the stability is guaranteed.

Hotkey's architecture diagram (the following figure is quoted from Hotkey's homepage on Gitee)

The entire structure of Hotkey is divided into the following parts:

Worker: Responsible for collecting and reporting information, and calculating hotspot information according to rules. The rules come from etcd. Push hotspot information to the client

Client: Each client connects to etcd, obtains the ip and port of each worker, maintains a long link with the worker, and accepts hot information pushes from the worker

etcd: Distributed coordinator, accepts the heartbeat report of each worker, and pushes the connection information of the worker to the client. Monitor changes to the rules and push them to the worker

Dashboard: UI interface, view the status of instances and workers, view and modify rule data. The rules are stored in mysql and pushed to the worker by etcd at the same time

The project address of hotkey is given below

https://gitee.com/jd-platform-opensource/hotkey

Regarding the introduction of Hotkey and how to build it, you can read this article to understand it, so I won't go into details here.

https://mp.weixin.qq.com/s/xOzEj5HtCeh_ezHDPHw6Jw

Problems encountered

We encountered two problems in the construction of the hotkey environment and implementation:

• Although Hotkey is open source, the related client jar package is not uploaded to the central warehouse, and the dashboard and worker startup packages are not available for download. You need to download the source code for compilation, and there are some package dependency issues during the compilation process.
• Hotkey's client jar only provides an api-level method for the program to use. If it is to be implemented in a business project, a large-scale code modification is required to implement it.

We hope to provide a less intrusive way to perform proxy packaging at the RPC and interface level. No matter what RPC framework users use, they just mark the relevant interfaces without moving any business code. You can detect hotspots at this interface level. If a parameter of the interface is a hot spot, it will automatically proxy and take the hot data of the jvm. After the hot spot is eliminated, the original call will still be used.

If you find the above description too difficult to understand, then to put it bluntly:

For example, during a certain event, a product S001 was rushed for purchase, and a large amount of traffic entered the product detail page. The product details RPC method calls the following interface methods of product services to obtain product information:

public interface ProductService{
  SkuInfo getSkuInfo(String skuCode);
}

Then we hope to mark only this interface. You can adapt the Hotkey framework to detect hotspots. When the product S001 is requested in large numbers, the product S001 can become a hotspot. At this time, the getSkuInfo interface will be automatically proxied, so that data is only obtained from the Jvm, and will not go. RPC call. After the hot spot is eliminated, this interface still calls RPC to obtain data.

This method is undoubtedly less invasive and easier to land the Hotkey framework.

Hotlink client

For this reason, we developed the Hotlink client framework based on the Hotkey client. This client framework allows Hotkey to be implemented more perfectly and enhances the capabilities of the Hotkey client.

project address:

How to use Hotlink

First step

According to Hotkey's deployment requirements, build workers and dashboards. Please refer to:

https://gitee.com/jd-platform-opensource/hotkey

At the same time, in order to facilitate everyone to build, I also uploaded the compiled worker and dashboard packages

Worker download address:

Public IP version (suitable for debugging, workers can be connected locally):

https://gitee.com/openbeast/hotlink/attach_files/813746/download/worker-0.0.4-SNAPSHOT-public.jar

Intranet IP version:

https://gitee.com/openbeast/hotlink/attach_files/813747/download/worker-0.0.4-SNAPSHOT.jar

Dashboard:

https://gitee.com/openbeast/hotlink/attach_files/813749/download/dashboard-0.0.2-SNAPSHOT.jar

Second step

Local business projects rely on jar packages (this jar package is not uploaded to the central warehouse, you need to deploy to your company's private library)

<dependency>
  <groupId>com.thebeastshop</groupId>
  <artifactId>hotlink-spring-boot-starter</artifactId>
  <version>1.0.12</version>
</dependency>

The fastjson and groovy versions required by hotlink are somewhat required. If the versions of these two packages in your project are too low and at the same time cover the transitive dependency packages of hotlink, you need to specify additional versions:

<fastjson.version>1.2.70</fastjson.version>
<guava.version>29.0-jre</guava.version>

Third step

Add parameters to the local springboot configuration file

#此app-name不配置的话，会优先读取spring.application.name属性
hotlink.app-name=test
#etcd地址和端口
hotlink.etcd-url=http://xxx.xxx.xxx.xxx:2379

Fourth step

Add the label @Hotlink

Add to the interface: all methods in the interface will automatically detect hotspots

Add to the method: only this method will automatically detect hotspots

for example:

public interface ProductService{
  @Hotlink
  SkuInfo getSkuInfo(String skuCode);
}

Then when a certain SKU001 becomes a hot spot, the incoming parameter SKU001 will automatically fetch data from the JVM, and SKU002 will continue to make RPC calls.

This completes all configurations. It can be activated.

Matters needing attention when using Hotlink

Since the implementation of Hotlink is implemented by a dynamic proxy, as long as these two conditions are met, the scanner will scan at startup:

• @Hotlink on the interface level
• Related implementations will be injected into the Spring context

When labeling interfaces, try to label interfaces that are idempotent within a certain time range. For example, member query, sku information query, and related activity information query. This information does not change frequently within a certain time range, so it is suitable for hot spot detection.

For non-idempotent interfaces, even if the parameters are the same, each return will be different. Then it is not recommended to do hot spot detection. For example, place an order, check inventory, and check balance. Once such an interface is upgraded to a hot spot. That will affect the correctness of the business interface and subsequent logic judgment errors.

about me

I am an open source author and a content creator. "Yuanren Tribe" is a technology sharing account that insists on being original. It will always share original technical articles and grow with you.