Implementing a set of current limiters that support queuing based on Redis

1. Background

Because the project calls an official government system, I was notified by the government some time ago that our call frequency is too high, and the call frequency of the currently open interface is once per second. Then we sent a report on the ratio of our requests passed and overclocked, showing that the failure rate is as high as 80%, which means that 80% of our requests have been intercepted. There should be partners who have questions about 80% of abnormal requests to your system. How to do business. In fact, there is a cache in our system, and then there are two ways to refresh the cache in the background, so the vast majority of these 80% failed requests are requests to refresh the cache in the background, not the request of the client user. So what about us? business is not materially affected. Based on this, we also need to do request restrictions, otherwise the government will consider dropping our interface permissions because the request failure rate is too high.

2. Research

There is not much to say about the classic algorithm funnel and token pass of current limiting. There is already a lot of content on the Internet to introduce such algorithms. Here I will sort out the open source current limiting tools currently on the market and the reasons why we do not choose to use open source tools but build our own wheels.

Google Guava

The first is Google's Guava tool class library, which provides many useful tool classes, including RateLimiter, a current limiter based on the token pass algorithm.

 // 1、声明一个qps最大为1的限流器RateLimiter limiter = RateLimiter.create(1); // 2、尝试阻塞获取令牌limiter.acquire();

Alibaba Sentinel

Then there is Alibaba's Sentinel, which is more powerful and should be the most comprehensive open source project currently on the market in terms of current limiting. It not only supports flow control, but also supports distributed current limiting, fuse degradation, system monitoring, etc., as well as more flexible current limiting policy configuration support. I haven't used it here, it may take some time to master it.

Redisson RRateLimiter

The last thing to introduce is the RRateLimiter based on the token pass algorithm. It is a current limiting tool for the Redisson class library. It supports distributed current limiting and is quite convenient to use.

 // 1、 声明一个限流器RRateLimiter rateLimiter = redissonClient.getRateLimiter(key); // 2、 设置速率，5秒中产生3个令牌rateLimiter.trySetRate(RateType.OVERALL, 3, 5, RateIntervalUnit.SECONDS); // 3、试图获取一个令牌，获取到返回true rateLimiter.tryAcquire(1)

Selection

1. First of all, my requirement is that the current limiter must support distributed, then Guava can be ruled out first.
2. Then Sentinel was a bit bulky for our needs, too heavy and so was ruled out.
3. Finally, although RRateLimiter supports distribution and is relatively simple to use, it seems that it does not support fair queuing (not quite sure).

3. Building wheels

Based on the above, I decided to build a distributed current limiter that supports fair queuing. The implementation scheme is based on Redis Lua script and then with the support of business layer code, serving directly

Body code of current limiter

 public class RedisRateLimiter { public static final GenericToStringSerializer argsSerializer = new GenericToStringSerializer<>(Object.class); public static final GenericToStringSerializer resultSerializer = new GenericToStringSerializer<>(Long.class); @Resource private RedisTemplate<String, Object> redisTemplate; @Resource private RedisUtil redisUtil; public static final int DEFAULT_MAX_PERMIT_COUNT = 1; public static final float DEFAULT_INTERVAL_SECONDS = 1.3f; public static final int DEFAULT_TIMEOUT_SECONDS = 5; // TODO 目前不支持自定义该值/** * 一个周期内的最大许可数量*/ private int maxPermitCount; public RedisRateLimiter() { this.maxPermitCount = DEFAULT_MAX_PERMIT_COUNT; } public static DefaultRedisScript<Long> redisScript; static { redisScript = new DefaultRedisScript<>(); redisScript.setScriptSource(new ResourceScriptSource(new ClassPathResource("rateLimiter.lua"))); redisScript.setResultType(Long.class); } /** * * @param redisKey * @param intervalSeconds 间隔几秒创建一个许可* @param timeoutSeconds 获取许可超时时间* @return * @throws InterruptedException */ public boolean tryAcquire(String redisKey, float intervalSeconds, long timeoutSeconds) throws InterruptedException { try { if (redisKey == null) { throw new BusinessException(BusinessExceptionCode.REQUEST_PARAM_ERROR, "redisKey不能为空！"); } Preconditions.checkArgument(intervalSeconds > 0.0 && !Double.isNaN(intervalSeconds), "rate must be positive"); long intervalMillis = (long) (intervalSeconds * 1000); long timeoutMillis = Math.max(TimeUnit.SECONDS.toMillis(timeoutSeconds), 0); long pttl = redisTemplate.execute(redisScript, argsSerializer, resultSerializer, Arrays.asList(redisKey), maxPermitCount, intervalMillis, timeoutMillis); if (pttl == 0) { log.info("----------------无需排队，直接通过, 当前许可数量={}", redisUtil.get(redisKey)); return true; }else if(pttl < timeoutMillis) { Thread.sleep(pttl); log.info("----------------排队{}毫秒后，通过, 当前许可数量={}", pttl, redisUtil.get(redisKey)); return true; }else { // 直接超时log.info("----------------需排队{}毫秒，直接超时, 当前许可数量={}", pttl, redisUtil.get(redisKey)); return false; } }catch (Exception e) { log.error("限流异常", e); } return true; } }

Core Lua script file: rateLimiter.lua

 local limit = tonumber(ARGV[1]) local interval = tonumber(ARGV[2]) local timeout = tonumber(ARGV[3]) local count = tonumber(redis.call('get', KEYS[1]) or "0") local pttl = tonumber(redis.call('pttl', KEYS[1]) or "0") if pttl < 0 then pttl = 0 end -- 这个代表已被预定的令牌数local currentCount = count - math.max(math.floor((count*interval - pttl)/interval), 0) -- 新增一个令牌local newCount = currentCount + 1 -- 所有令牌总的失效毫秒local newPTTL = pttl + interval if newCount <= limit then --无需排队直接通过redis.call("PSETEX", KEYS[1], newPTTL, newCount) return 0 elseif pttl < timeout then --排队pttl毫秒后可通过redis.call("PSETEX", KEYS[1], newPTTL, newCount) else -- 超时redis.call("PSETEX", KEYS[1], pttl, currentCount) end -- 返回需等待毫秒数return pttl