Delay task - a complete implementation based on redis zset

Let me give you an example of the so-called delayed task: you buy a train ticket and you must pay within 30 minutes, otherwise the order will be automatically cancelled. If the order is not paid for within 30 minutes, it will be automatically cancelled. This task is a delayed task. I've written 2 articles about delayed tasks before:

• "Complete Implementation - Implementing Delayed Tasks through DelayQueue"
• "Delayed Task (2) - Actual Combat Based on Netty Time Wheel Algorithm"

Both of these methods have a disadvantage: they both run delayed tasks based on the memory of a single application. Once a single point of failure occurs, the data of the delayed tasks may be lost. Therefore, this article will introduce the third way to implement delayed tasks. Combined with redis zset to implement delayed tasks, it can solve the problem of single point of failure. The implementation principle, complete implementation code, and advantages and disadvantages of this implementation method are given.

First, the realization principle

First, let's introduce the implementation principle. We need to use redis zset to realize the requirements of delayed tasks, so we need to know the application characteristics of zset. Zset exists as an ordered set data structure of redis, and the sorting is based on score.

So we can use this feature of zset score sorting to implement delayed tasks

• When the user places an order, a delayed task is generated and placed in redis, and the key can be customized, for example: delaytask:order
• The value of value is divided into two parts, one is score for sorting, the other is member, and the value of member is set as the order object (such as: order number), because when the time limit of subsequent delayed tasks is reached, we need to have some Necessary order information (such as order number) can be completed to automatically cancel and close the order.
• The focus of the delay task implementation is coming. We set the score as: order generation time + delay time . In this way, redis will sort the zset according to the score delay time.
• Start the redis scan task, obtain the delayed task with "current time > score" and execute it. That is, when the current time > order generation time + delay time, the delay task is executed.

2. Preparations

To use the redis zset solution to complete the requirements of delayed tasks, first of all, redis must be required, there is no doubt about this. There are many articles on the construction of redis on the Internet, so I won't go into details here.

Secondly, the author's long-term java application system development is done using SpringBoot, so I am also used to using SpringBoot's redis integration solution. First introduce spring-boot-starter-data-redis through maven coordinates

 <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-pool2</artifactId>
</dependency>

Next, you need to configure the link information of the redis database in Spring Boot's application.yml configuration file. What I configure here is a singleton of redis. If your production environment is sentinel mode or redis in cluster mode, the configuration method here needs to be fine-tuned. In fact, this part of the content has been systematically introduced in my personal blog. Interested friends can follow my personal blog.

 spring:
  redis:
    database: 0 # Redis 数据库索引（默认为 0）
    host: 192.168.161.3 # Redis 服务器地址
    port: 6379 # Redis 服务器连接端口
    password: 123456 # Redis 服务器连接密码（默认为空）
    timeout:  5000  # 连接超时，单位ms
    lettuce:
      pool:
        max-active: 8 # 连接池最大连接数（使用负值表示没有限制） 默认 8
        max-wait: -1 # 连接池最大阻塞等待时间（使用负值表示没有限制） 默认 -1
        max-idle: 8 # 连接池中的最大空闲连接 默认 8
        min-idle: 0 # 连接池中的最小空闲连接 默认 0

3. Code Implementation

The following class is the core implementation of the delay task. It contains three core methods. Let's explain them one by one:

• The produce method is used to generate an order - order is the order information, which can be the order serial number, which is used to close the order after the delay task reaches the time limit
• The afterPropertiesSet method is a method of the InitializingBean interface. The reason for implementing this interface is that we need to start the redis scanning task when the application starts. That is: when the OrderDelayService bean is initialized, start the redis scan task cycle to obtain delayed task data.

• The consuming function is used to obtain delayed task data from redis, consume delayed tasks, and perform operations such as closing overtime orders. In order to avoid blocking the for loop and affect the execution of subsequent delayed tasks, this consuming function must be made asynchronous. Refer to Spring Boot asynchronous tasks and the usage of Async annotations. I have written an open source project of SpringBoot's observable and easy-to-configure asynchronous task thread pool, source code address: https://gitee.com/hanxt/zimug-monitor-threadpool . My zimug-monitor-threadpool open source project can monitor the usage of the thread pool. I usually use it with good results. I recommend it to everyone!

   @Component
  public class OrderDelayService  implements InitializingBean {
  //redis zset key
  public static final String ORDER_DELAY_TASK_KEY = "delaytask:order";
  
  @Resource
  private StringRedisTemplate stringRedisTemplate;
  
  //生成订单-order为订单信息，可以是订单流水号，用于延时任务达到时效后关闭订单
  public void produce(String orderSerialNo){
    stringRedisTemplate.opsForZSet().add(
            ORDER_DELAY_TASK_KEY,     // redis key
            orderSerialNo,    // zset  member
            //30分钟延时
            System.currentTimeMillis() + (30 * 60 * 1000)    //zset score
    );
  }
  
  //延时任务，也是异步任务，延时任务达到时效之后关闭订单，并将延时任务从redis zset删除
  @Async("test")
  public void consuming(){
       
      Set<ZSetOperations.TypedTuple<String>> orderSerialNos = stringRedisTemplate.opsForZSet().rangeByScoreWithScores(
              ORDER_DELAY_TASK_KEY,
              0,  //延时任务score最小值
              System.currentTimeMillis() //延时任务score最大值（当前时间）
      );
      if (!CollectionUtils.isEmpty(orderSerialNos)) {
        for (ZSetOperations.TypedTuple<String> orderSerialNo : orderSerialNos) {
          //这里根据orderSerialNo去检查用户是否完成了订单支付
          //如果用户没有支付订单，去执行订单关闭的操作
          System.out.println("订单" + orderSerialNo.getValue() + "超时被自动关闭");
          //订单关闭之后，将订单延时任务从队列中删除
          stringRedisTemplate.opsForZSet().remove(ORDER_DELAY_TASK_KEY, orderSerialNo.getValue());
        }
      }
  }
  
  //该类对象Bean实例化之后，就开启while扫描任务
  @Override
  public void afterPropertiesSet() throws Exception {
    new Thread(() -> {  //开启新的线程，否则SpringBoot应用初始化无法启动
      while(true){
        try {
          Thread.sleep(5 * 1000);   //每5秒扫描一次redis库获取延时数据，不用太频繁没必要
        } catch (InterruptedException e) {
          e.printStackTrace();  //本文只是示例，生产环境请做好相关的异常处理
        }
        consuming();
      }
    }).start();
  }
  }

  For more details, refer to the comments in the code. The points to pay attention to are:

• The rangeByScoreWithScores method above is used to obtain delayed tasks from redis. All delayed tasks with a score greater than 0 and less than the current time will be taken out of redis. It is executed every 5 seconds, so the error of the delayed task will not exceed 5 seconds.
• For the order information above, I only keep the unique serial number of the order to close the order. If your business needs to pass more order information, please use RedisTemplate to operate the order class object instead of StringRedisTemplate to operate the order serial number string.

When placing an order, use the following method to put the order serial number into the redis zset to realize the delayed task

 orderDelayService.produce("这里填写订单编号");

Fourth, the advantages and disadvantages

The advantages of using redis zset to implement delayed tasks are: compared with the two methods introduced at the beginning of this article, our delayed tasks are stored in redis. Redis has a data persistence mechanism, which can effectively avoid delayed task data. of loss. In addition, redis can effectively avoid service interruption caused by single point of failure through sentinel mode and cluster mode.
As for the downsides, I don't think there are any downsides. If we have to reluctantly say one disadvantage, it is that we need to maintain the redis service additionally, which increases the demand for hardware resources and the cost of operation and maintenance. But now with the rise of microservices, redis has almost become the standard configuration of application systems, and redis can be reused, so I feel that this is not a disadvantage!

Code text is not easy, if you find it helpful, please help click to watch or share, I may not be able to persevere without your support!
Welcome to pay attention to my announcement number: Antetokounmpo, reply 003 and present the PDF version of the author's column "The Way of Docker Cultivation", more than 30 high-quality docker articles. Antetokounmpo Blog: zimug.com