Technology Sharing | Implementation of redis client connection instance under master-slave and cluster architecture

Author: Wang Yue

Copyright (C) 2021 wingYue

Source of this article: original submission

* Produced by the Aikesheng open source community, original content is not allowed to be used without authorization, please contact the editor and indicate the source for reprinting.

---

As a high-performance in-memory database, redis is widely used in various systems, such as rankings, scoring services, and so on. When we choose redis, in addition to considering whether the data type and performance it provides meets business needs, it is very important to consider high availability. redis provides redis sentinel to complete the high availability mechanism, sentinel will monitor the redis master-slave instance, and provide automatic failover function.

But a question that follows is how does the client know the current master instance address after failover?

Master-slave architecture method 1: service discovery using redis sentinel

redis sentinel provides a service discovery mechanism. Connecting sentinel to execute "SENTINEL get-master-addr-by-name" will return the current master instance address. After failover, the returned result will also be updated to the new master instance address:

There are many "smart" redis client libraries that implement automatic connection based on the service discovery mechanism. As long as the sentinel address list is passed in, clinet will obtain the current latest master address through sentinel, and then use the obtained address to connect. For example, jedis:

    sentinels.add(new HostAndPort("192.168.0.31",26379).toString());
    sentinels.add(new HostAndPort("192.168.0.32",26379).toString());
    sentinels.add(new HostAndPort("192.168.0.33",26379).toString());
    pool = new JedisSentinelPool(masterName, sentinels, config, TIMEOUT,password);
    ...
   // 获取连接：
   Jedis jedis = pool.getResource();
   try {
      jedis.set("hello", "jedis");
   } finally {
      jedis.close();
   }

As an excellent redis client, jedis uses the sentinel subscription method to maintain the latest master address in memory at all times. Some other clients first ask the sentinel for the latest master address every time they get a connection, and then execute the redis connection, so that each operation needs to send two requests, which is not very efficient. Another way is to use VIP:

Master-slave architecture method 2: binding VIP

We maintain a VIP so that it is always bound to the master node, so that the client can connect to the VIP address without thinking when it connects. VIP maintenance can be achieved through sentinel's client-reconfig-scrip t script. Every time the master-slave instance monitored by sentinel fails over, sentinel will call this script and pass in the latest master address. We can implement VIP in the script Bind and unbind operations.

Binding VIP does not depend on the "smartness" of the client. It is implemented through custom scripts, which is more flexible and controllable. However, VIP cannot support some external network access scenarios. Of course, because the script is customized, for example, if there is a DNS system, you can change the binding of VIP to binding DNS to provide access to the external network.

Master-slave architecture method three: use keepalived VRRP

Method 3 is similar to Method 2 in that both provide service access through VIP. Method 3 uses keepalived's VRRP to achieve VIP binding and does not rely on sentinel's reconfig script.

Master-slave architecture method 4: middleware agent

Some public cloud redis services provide a proxy address for client access, and the address actually corresponds to a middleware proxy. In addition to the development cost to implement a middleware, it also needs to maintain the high availability of the middleware itself at runtime. Of course, the cost will bring benefits. In addition to realizing basic connection forwarding, middleware can also provide more high-level functions. For example, Alibaba Cloud's redis Proxy provides:
1. The proxy sends the write command to the master node, and sends the read command to the master or slave node according to the weight
2. The proxy will offline the abnormal read-only node, and re-enable it after the node is restored

---

Above we are for the redis master-slave architecture, and discussed how the client can connect to the correct master node after failover.
Below we discuss how the client correctly connects to the nodes that the slot involved in this operation needs to access under the redis cluster architecture?

Cluster architecture method 1: request redirection

Redirection means that the client will randomly select a redis instance to perform the request operation. After the redis instance receives the request, it will calculate the slot corresponding to the key. If it is locally processed, it will directly process it, otherwise it will return a MOVED to the client, indicating the correct instance address. Let the client redirect. For example, the redis-cli tool is the redirection method used:

redis-cli will automatically redirect to the specified address after receiving the MOVED response. One problem with this is that it may require two requests for one operation, which will affect performance.

Cluster architecture method 2: local cache slots list

A "smart" client (such as JedisCluster) will access any node in the cluster at startup to obtain slot information, and cache a list of nodes corresponding to all slots locally. In this way, when the slots are not migrated, it can be ensured that the operation request is sent directly to the correct node. When the slots migration occurs in the cluster, some requests will return errors. At this time, the client will re-update the list of slots in the cache, and then request again. Since slot migration is not a high-frequency operation, this approach has little effect on overall performance.

Cluster architecture method three: middleware agent

Universal middleware, similar to the master-slave architecture, will not be discussed in detail here. If the client is not "smart" enough, you can throw the pot to the middleware. Of course, the cost of realizing middleware is directly proportional to the benefit, and it needs to be selected according to the actual situation.