DIY a cron expression parser

background

I made a message platform for a certain mall project of the company, which is which is . This message is not a message such as SMS email notification, but refers to the message in the message queue. The platform can dynamically create consumers and producers, and handle asynchronous processing. Messages provide a variety of visualization methods to manage the whole life cycle of the message processing process. Interested friends can learn about it. Advertising time ends:), the following is the text

The platform has a small function, you can configure timed tasks, and execute some programs on a regular basis. From the beginning, simply use ScheduledThreadPoolExecutor realize it. Periodic execution of tasks can be realized. Later, it is necessary to implement non-fixed a certain day of a month. Periodic tasks cannot be achieved, which requires the introduction of cron expression . It is not difficult to find a framework that supports cron expression. Spring boot itself supports it, and quartz also supports it, but considering

• Timing is not a core function, do not want to introduce too many dependencies for a non-core function
• The cron expression has only 5 variables, so it is relatively simple to parse
• Build your own wheels, which is more controllable

As for why the cron expression function that comes with spring boot is not used (and no new dependencies are introduced), there are two reasons

• The system and spring boot are decoupled in architecture, that is, the core of the system does not depend on spring boot. Spring boot only implements the function of web api, but the timing belongs to the function of the system itself, not the function of web api.
• The cron of spring boot does not support dynamic creation and needs to be determined at startup

This article does not use any knowledge of compilation principles (in fact, I will not), it is completely hard analysis, you can rest assured that everyone can understand:)

cron expression

Cron expression is an expression language that can describe periodic tasks. A cron expression consists of 5 parts, each part is separated by a space. For example, the following expression means 20:12 execution every day

12 20 * * *

The meaning of each part of cron expression is as follows

• 1. minute
• 2. Hour
• 3. sky
• 4. moon
• 5. week

Each part allows the following types of operators

• * All numbers in the value range of
• / every number of digits
• - from X to Z
• , hashed numbers

Instance

实例1：每1分钟执行一次
* * * * *
实例2：每小时的第3和第15分钟执行
3,15 * * * * 
实例3：在上午8点到11点的第3和第15分钟执行
3,15 8-11 * * *
实例4：每隔两天的上午8点到11点的第3和第15分钟执行
3,15 8-11 */2  *  *
实例5：每周一上午8点到11点的第3和第15分钟执行
3,15 8-11 * * 1
实例6：每晚的21:30重启smb
30 21 * * *
实例7：每月1、10、22日的4 : 45重启smb
45 4 1,10,22 * *
实例8：每周六、周日的1 : 10重启smb
10 1 * * 6,0
实例9：每天18 : 00至23 : 00之间每隔30分钟重启smb
0,30 18-23 * * *
实例10：每星期六的晚上11 : 00 pm重启smb
0 23 * * 6
实例11：每一小时重启smb
0 */1 * * *

Realization idea

To complete a quartz-like program, two core components are needed to complete it. Basically all timing frameworks are based on this idea

• A fixed-period thread (actually Thread.sleep), the period depends on the accuracy supported by the timing, the thread timing (such as 5s) to check whether there is a task to be executed, then a program is needed to tell it whether to execute
• According to the + the last execution time to determine whether the task is to be executed in this execution cycle, this is what the parser wants to do, and it is also our task today

The first component is relatively simple and is beyond the scope of this discussion. This time we will mainly discuss how to implement the second component. We divide the parser into two parts.

• data structure
• algorithm

The data structure refers to how to store cron data (not a simple string). Choosing a suitable data structure can do more with less. The algorithm refers to how to determine whether the cycle is hit or not after it is parsed and stored in the specified data structure. Let's talk separately.

data structure

Through observation, we can find that each part can be divided into two categories

• Periodic execution class (for example, once every five minutes)
• Fixed time execution (such as 20:12 minutes execution)

Regardless of the type, we can abstract it as the range of 161d289e9483ba. For example, the default range of minutes is 1-59, so whether it is a periodic or a fixed time, the range cannot be escaped.

• /5 *: range 1-59, because you don’t know the number of minutes of the last execution, so the full range is possible.
• 12 20 *: range [12], only 12
• 12,13 : range [12-13]
• 12-15 : range [12-15]

Because the range can cover all the grammars we support, there is also a small problem. The minute, hour, and month can be determined, but the day cannot be determined. The day is determined according to the month and is also affected by the year (leap year). And cron also supports week, which has no specific concept. How to deal with the problem of week? To further abstract, we combine the year, month, and day to define the scope, then there are up to 366 options in the scope, and the processing of the week is also very simple. We will unify the above, we have defined the following scopes

• Minute
• Time
• Year, month and day (In fact, the year does not need to be defined, because there is no upper limit for the year, so you can keep adding up)

What data structure should be used to store the month and day, because the minutes and hours are all int type, and the hour is incremented, it is best to keep the same, considering that month<=12, day<=31, because it can be operated Combine two numbers into one number

 /**
     * 将月和日合并成一个int型整数
     * @param month
     * @param day
     * @return
     */
    public int encodeMonthday(int month, int day) {
        return (day & 0x000000FF) | (month << 8 & 0x0000FF00);
    }

    /**
     * 解码月
     * @param monthDay
     * @return
     */
    public int decodeMonth(int monthDay) {
        return (monthDay & 0x0000FF00) >> 8;
    }

    /**
     * 解码日
     * @param monthDay
     * @return
     */
    public int decodeDay(int monthDay) {
        return (monthDay & 0x000000FF);
    }

algorithm

This part is the most troublesome. I try to make it clear as much as possible. We may understand the problem better if we abstract it. We will transform the problem into the following description

There are three combinations of ABC, A value is [A1-AN], B value is [B1-BN], C value is [C1-CN], given a DEF, find the next minimum value of DEF in ABC

Is it a bit like the feeling of doing ACM questions in college, but it is like this in abstraction. My idea is like this (not necessarily the best, I dreamed of joining the ACM team when I was in college, but I didn't even make the primary election." ), so if you have a better solution, please leave a message in the comment area

• Judging from big to small
• First judge whether F is in C, if it is, then continue to judge E
• Judge whether E is in B, if you continue to judge D
• Judge whether D is in A, if it is, then just calculate Min([A1-AN]>D)
• If D is not in A, then return to E and calculate Min([B1-BN]>E)
• And so on

Of course, there are still some small issues that need to be dealt with, such as the New Year's Eve issues, etc. The detailed algorithm can be seen in the code, and the language expression ability is limited to this.

accomplish

The entire parser is implemented, and the code part does not exceed 200 lines, so it is not very difficult to read. The complete code is posted as follows

package com.definesys.mc.core.cron;

import java.util.Calendar;
import java.util.Date;
import java.util.Set;

import static java.util.Calendar.DATE;
import static java.util.Calendar.DAY_OF_YEAR;

/**
 * @Description:
 * @author: jianfeng.zheng
 * @since: 2021/12/30 3:50 下午
 * @history: 1.2021/12/30 created by jianfeng.zheng
 */
public class CronParser {

    private String cronExp;

    public CronParser(String exp) {
        this.cronExp = exp;
    }

    public Date nextDate(Date start) {
        Calendar lastCal = Calendar.getInstance();
        lastCal.setTime(start);

        //上一次执行时间字段
        int lastYear = lastCal.get(Calendar.YEAR);
        int lastMonth = lastCal.get(Calendar.MONTH) + 1;
        int lastDay = lastCal.get(Calendar.DAY_OF_MONTH);
        int lastMonthDay = this.encodeMonthday(lastMonth, lastDay);
        int lastHour = lastCal.get(Calendar.HOUR_OF_DAY);
        int lastMinute = lastCal.get(Calendar.MINUTE);
        int lastSecond = lastCal.get(Calendar.SECOND);

        //下一次执行时间字段
        Integer newMonthDay = null;
        Integer newHour = null;
        Integer newMinute = null;
        Integer newYear = lastYear;

        //解析cron表达式
        String[] exps = cronExp.split("\\s+");
        CronRange minute = parseRange(exps[0], 0, 59);
        CronRange hour = parseRange(exps[1], 0, 23);
        CronRange day = parseRange(exps[2], 1, 31);
        CronRange month = parseRange(exps[3], 1, 12);
        CronRange week = parseRange(exps[4], 1, 7);
        CronRange monthDay = this.calMonthDay(month, day, week);
        if (monthDay.isEmpty()) {
            return null;
        }

        boolean isNotFound = false;
        if (monthDay.inRange(lastMonthDay)) {
            if (hour.inRange(lastHour)) {
                if (minute.inRange(lastMinute)) {
                    newMinute = minute.getNextValue(lastMinute);
                }
                if (newMinute == null) {
                    //如果分钟找不到，需要对小时进行递增
                    newHour = hour.getNextValue(lastHour);
                    isNotFound = newHour == null;
                    newMinute = minute.getMin();
                } else {
                    newHour = lastHour;
                }
            }
            if (newHour == null) {
                if (isNotFound) {
                    //如果小时找不到，需要对天数进行递增
                    if (monthDay.isAll()) {
                        Calendar c = Calendar.getInstance();
                        c.setTime(start);
                        c.add(DATE, 1);
                        newMonthDay = this.encodeMonthday(c.get(Calendar.MONTH) + 1, c.get(Calendar.DAY_OF_MONTH));
                    } else {
                        //如果跨年了就找不到
                        newMonthDay = monthDay.getNextValue(lastMonthDay);
                    }
                } else {
                    newMonthDay = lastMonthDay;
                }
                newHour = hour.getMin();
                newMinute = minute.getMin();
            } else {
                newMonthDay = lastMonthDay;
            }
        } else {
            //天如果不在范围内，需要对天进行递增
            newMonthDay = monthDay.getNextValue(lastMonthDay);
            newHour = hour.getMin();
            newMinute = minute.getMin();
        }
        if (newMonthDay == null) {
            //跨年
            newYear = newYear + 1;
            if (monthDay.isAll()) {
                //1月1日
                newMonthDay = 0x0101;
            } else {
                newMonthDay = monthDay.getMin();
            }
            newHour = hour.getMin();
            newMinute = minute.getMin();
        }
        Calendar newCal = Calendar.getInstance();
        newCal.set(Calendar.MONTH, this.decodeMonth(newMonthDay) - 1);
        newCal.set(Calendar.DAY_OF_MONTH, decodeDay(newMonthDay));
        newCal.set(Calendar.HOUR_OF_DAY, newHour);
        newCal.set(Calendar.MINUTE, newMinute);
        newCal.set(Calendar.SECOND, lastSecond);
        newCal.set(Calendar.YEAR, newYear);
        return newCal.getTime();
    }

    /**
     * 将月和日合并成一个int型整数
     * @param month
     * @param day
     * @return
     */
    public int encodeMonthday(int month, int day) {
        return (day & 0x000000FF) | (month << 8 & 0x0000FF00);
    }

    /**
     * 解码月
     * @param monthDay
     * @return
     */
    public int decodeMonth(int monthDay) {
        return (monthDay & 0x0000FF00) >> 8;
    }

    /**
     * 解码日
     * @param monthDay
     * @return
     */
    public int decodeDay(int monthDay) {
        return (monthDay & 0x000000FF);
    }

    private CronRange calMonthDay(CronRange month, CronRange day, CronRange week) {
        CronRange monthDay = new CronRange();
        if (month.isAll() && day.isAll() && week.isAll()) {
            //如果都是全范围的就不进行计算
            monthDay.setReturnAll(true);
            return monthDay;
        }
        int[] monthDays = {31, 0, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31};
        //如果是闰年就是29天
        monthDays[1] = Calendar.getInstance().getActualMaximum(DAY_OF_YEAR) > 365 ? 29 : 28;
        Set<Integer> rangeMonth = month.getRange();
        for (Integer m : rangeMonth) {
            for (int d = 1; d <= monthDays[m - 1]; ++d) {
                if (day.inRange(d)) {
                    //判断周的逻辑
                    if (!week.isAll()) {
                        Calendar cal = Calendar.getInstance();
                        cal.set(Calendar.MONTH, m - 1);
                        cal.set(Calendar.DAY_OF_MONTH, d);
                        int w = cal.get(Calendar.DAY_OF_WEEK) - 1;
                        //周日-周六==>1-7
                        w = w == 0 ? 7 : w;
                        if (!week.inRange(w)) {
                            continue;
                        }
                    }
                    monthDay.addRange(this.encodeMonthday(m, d));
                }
            }
        }
        return monthDay;
    }

    /**
     * 解析表达式的取值范围和循环周期
     *
     * @param exp
     * @param start
     * @param end
     * @return
     */
    public CronRange parseRange(String exp, int start, int end) {
        String[] exps = exp.trim().split("/");
        CronRange range = new CronRange();
        if (exps.length > 1) {
            range.setCycle(Integer.parseInt(exps[1]));
        }

        if (exps[0].trim().length() == 0) {
            range.range(start, end);
        } else if ("*".equals(exps[0])) {
            range.range(start, end);
            range.setReturnAll(exps.length == 1);
        } else if (exps[0].contains("-")) {
            String[] ss = exps[0].split("-");
            range.range(Integer.parseInt(ss[0]), Integer.parseInt(ss[1]));
        } else if (exps[0].contains(",")) {
            String[] ss = exps[0].split(",");
            for (String s : ss) {
                range.addRange(Integer.parseInt(s));
            }
        } else {
            range.addRange(Integer.parseInt(exps[0]));
        }
        return range;
    }
}

class CronRange {
    private Set<Integer> range = new TreeSet<>();
    private Integer cycle;
    private Integer max = null;
    private Integer min = null;
    private Boolean returnAll = false;

    public CronRange range(int start, int end) {
        for (int i = start; i <= end; ++i) {
            this.addRange(i);
        }
        return this;
    }

    public CronRange addRange(int value) {
        max = (max == null || value > max) ? value : max;
        min = (min == null || value < min) ? value : min;
        this.range.add(value);
        return this;
    }

    public Set<Integer> getRange() {
        return range;
    }


    public void setCycle(Integer cycle) {
        this.cycle = cycle;
    }


    public boolean inRange(int value) {
        return returnAll ? true : range.contains(value);
    }

    public boolean isEmpty() {
        return !returnAll && range.isEmpty();
    }


    public Integer getNextValue(int lastValue) {
        Integer value = null;
        if (this.cycle != null) {
            value = this.cycle + lastValue;
            while (!inRange(value)) {
                value = value + this.cycle;
                if (value > max) {
                    value = null;
                    break;
                }
            }
        } else {
            value = this.getNextMin(lastValue);
        }
        return value;
    }

    private Integer getNextMin(int value) {
        Integer[] integers = range.toArray(new Integer[range.size()]);
        Integer minValue = null;
        for (int i = 0; i < integers.length; ++i) {
            if (integers[i] > value) {
                minValue = integers[i];
                break;
            }
        }
        return minValue;
    }


    public Boolean isAll() {
        return returnAll;
    }

    public void setReturnAll(Boolean returnAll) {
        this.returnAll = returnAll;
    }

    public Integer getMin() {
        return min;
    }
}

test

I wrote a few expressions and tested them, and they all met the expected results.

public static void main(String[] cmd) throws ParseException {
        String cronExp = "* * * * *";
        CronParser parser = new CronParser(cronExp);
        String lastExecuteDateStr = "2022-1-3 22:23:22";
        SimpleDateFormat fmt = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
        Date lastExecuteDate = fmt.parse(lastExecuteDateStr);
        for (int i = 0; i < 10; ++i) {
            lastExecuteDate = parser.nextDate(lastExecuteDate);
            if (lastExecuteDate == null) {
                return;
            }
            System.out.println(fmt.format(lastExecuteDate));
        }
    }

Output

2022-01-03 22:24:22
2022-01-03 22:25:22
2022-01-03 22:26:22
2022-01-03 22:27:22
2022-01-03 22:28:22

Other examples

# 每五分钟
*/5 * * * *
2022-01-03 22:28:22
2022-01-03 22:33:22
2022-01-03 22:38:22
2022-01-03 22:43:22
2022-01-03 22:48:22

#12点的时候每五分钟
*/5 12 * * *
2022-01-03 12:00:22
2022-01-03 12:05:22
2022-01-03 12:10:22
2022-01-03 12:15:22
2022-01-03 12:20:22

#2月3日12点的时候每五分钟
*/5 12 3 2 *
2022-02-03 12:00:22
2022-02-03 12:05:22
2022-02-03 12:10:22
2022-02-03 12:15:22
2022-02-03 12:20:22

Concluding remarks

In actual projects, we may also encounter a somewhat complicated business development like this kind. When facing this kind of development, we must not code immediately, and code rashly without clarifying the data structure and algorithm. It must be There is a problem. The parser is simple and not simple, or complicated or complicated, but the data structure and algorithm also took me a day to study on the notebook. It is recommended that programmers have a notebook and write their ideas clearly on the paper. , Writing code is just to implement the things on the paper with code (actually coding + debugging less than an hour), and attach notes that are so ugly that only me and God can understand