SpringBoot Timing Task - Classic Timing Task Structure Design: Timing Wheel Case and Implementation Principle

Timer and ScheduledExecutorService are JDK's built-in timed task solutions, and there is also a classic timed task design in the industry called Timing Wheel. Netty internally implements a HashedWheelTimer based on the time wheel to optimize millions of I/O timeouts. Detection, it is a high-performance, low-consumption data structure, which is suitable for non-quasi-real-time, delayed short-term and fast tasks, such as heartbeat detection. This article mainly introduces the Timing Wheel and its use. @pdai

• SpringBoot timing task - Netty HashedWheelTimer method

  • Knowledge preparation

    • What is the Timing Wheel
    • What is the problem with Netty's HashedWheelTimer?
    • How to use HashedWheelTimer

  • Implementation case

    • Pom dependencies
    • 2 simple examples

  • further understanding

    • How is HashedWheelTimer implemented?
    • What is a Multi-Level Timing Wheel?
  • Sample source code

Knowledge preparation

Requires a preliminary understanding of the Timing Wheel and what the Netty's HashedWheelTimer is trying to solve.

What is the Timing Wheel

The Timing Wheel is implemented by George Varghese and Tony Lauck in their 1996 paper ' Hashed and Hierarchical Timing Wheels: data structures to efficiently implement a timer facility '. It is widely used in the Linux kernel and is the Linux kernel timer. One of the realization methods and foundations.

Timing Wheel is a circular data structure, just like a clock can be divided into many grids (Tick), each grid represents the time interval, which points to a linked list of stored specific tasks (timerTask).

Taking the above picture example in the paper, here a wheel contains 8 grids (Tick), each tick is one second;

Addition of tasks : If a task is to be executed after 17 seconds, it needs to be rotated 2 rounds and finally added to the linked list at the position of Tick=1.

Task execution : When the clock turns 2Round to the position of Tick=1, start executing the task in the linked list pointed to by this position. (# here means that there are several rounds left to perform this task)

What is the problem with Netty's HashedWheelTimer?

HashedWheelTimer is a tool class developed by Netty based on the Timing Wheel. What problem does it solve? There are two main points here: delayed tasks + low timeliness . @pdai

A typical application scenario in Netty is to determine whether a connection is idle. If the connection is idle (such as the client's heartbeat to the server cannot be delivered due to network reasons), the server will actively disconnect and release resources. Determining whether the connection is idle is done through timed tasks, but Netty may maintain millions of long connections. It is not feasible to define a timed task for each connection, so how to improve the efficiency of I/O timeout scheduling?

Netty developed the HashedWheelTimer tool class based on the Timing Wheel to optimize the I/O timeout scheduling (essentially delaying tasks); there is another very important reason why the structure of the Timing Wheel is used. I /O timeout This type of task does not need to be very precise about timeliness.

How to use HashedWheelTimer

After understanding the problem to be solved by Timing Wheel and Netty's HashedWheelTimer, let's take a look at how HashedWheelTimer is used

Look at the main parameters through the constructor

 public HashedWheelTimer(
        ThreadFactory threadFactory,
        long tickDuration, TimeUnit unit, int ticksPerWheel, boolean leakDetection,
        long maxPendingTimeouts, Executor taskExecutor) {

}

The specific parameters are described as follows:

• threadFactory : Thread factory, used to create worker threads, the default is Executors.defaultThreadFactory()
• tickDuration : tick cycle, that is, how often to tick
• unit : unit of tick period
• ticksPerWheel : the length of the time wheel, how many grids there are in one circle
• leakDetection : Whether to enable memory leak detection, the default is true
• maxPendingTimeouts : The maximum number of tasks to be executed, the default is -1, that is, there is no limit. This parameter is only set in the case of high concurrency.

Implementation case

The basic use case of HashedWheelTimer is shown here. @pdai

Pom dependencies

Import pom dependencies

 <dependency>
    <groupId>io.netty</groupId>
    <artifactId>netty-all</artifactId>
    <version>4.1.77.Final</version>
</dependency>

2 simple examples

Example 1: Execute TimerTask after 5 seconds

 @SneakyThrows
public static void simpleHashedWheelTimer() {
    log.info("init task 1...");
    
    HashedWheelTimer timer = new HashedWheelTimer(1, TimeUnit.SECONDS, 8);

    // add a new timeout
    timer.newTimeout(timeout -> {
        log.info("running task 1...");
    }, 5, TimeUnit.SECONDS);
}

The execution result is as follows:

 23:32:21.364 [main] INFO tech.pdai.springboot.schedule.timer.netty.HashedWheelTimerTester - init task 1...
...
23:32:27.454 [pool-1-thread-1] INFO tech.pdai.springboot.schedule.timer.netty.HashedWheelTimerTester - running task 1...

Example 2: Cancel the task after failure and let it execute again after 3 seconds.

 @SneakyThrows
public static void reScheduleHashedWheelTimer() {
    log.info("init task 2...");

    HashedWheelTimer timer = new HashedWheelTimer(1, TimeUnit.SECONDS, 8);

    Thread.sleep(5000);

    // add a new timeout
    Timeout tm = timer.newTimeout(timeout -> {
        log.info("running task 2...");
    }, 5, TimeUnit.SECONDS);

    // cancel
    if (!tm.isExpired()) {
        log.info("cancel task 2...");
        tm.cancel();
    }

    // reschedule
    timer.newTimeout(tm.task(), 3, TimeUnit.SECONDS);
}

 23:28:36.408 [main] INFO tech.pdai.springboot.schedule.timer.netty.HashedWheelTimerTester - init task 2...
23:28:41.412 [main] INFO tech.pdai.springboot.schedule.timer.netty.HashedWheelTimerTester - cancel task 2...
23:28:45.414 [pool-2-thread-1] INFO tech.pdai.springboot.schedule.timer.netty.HashedWheelTimerTester - running task 2...

further understanding

We further understand HashedWheelTimer through the following questions. @pdai

How is HashedWheelTimer implemented?

A brief look at how HashedWheelTimer is implemented

• Worker : The worker thread is mainly responsible for task scheduling and triggering, and runs in a single thread.
• HashedWheelBucket : The grid above the time wheel holds the head and tail nodes of the linked list structure composed of HashedWheelTimeout. The time wheel composed of multiple grids forms a circle of tasks.
• HashedWheelTimeout : The tasks submitted to the time wheel will be encapsulated as HashedWheelTimeout

Constructor

 public HashedWheelTimer(
        ThreadFactory threadFactory,
        long tickDuration, TimeUnit unit, int ticksPerWheel, boolean leakDetection,
        long maxPendingTimeouts, Executor taskExecutor) {

    checkNotNull(threadFactory, "threadFactory");
    checkNotNull(unit, "unit");
    checkPositive(tickDuration, "tickDuration");
    checkPositive(ticksPerWheel, "ticksPerWheel");
    this.taskExecutor = checkNotNull(taskExecutor, "taskExecutor");

    // Normalize ticksPerWheel to power of two and initialize the wheel.
    wheel = createWheel(ticksPerWheel);
    mask = wheel.length - 1;

    // Convert tickDuration to nanos.
    long duration = unit.toNanos(tickDuration);

    // Prevent overflow.
    if (duration >= Long.MAX_VALUE / wheel.length) {
        throw new IllegalArgumentException(String.format(
                "tickDuration: %d (expected: 0 < tickDuration in nanos < %d",
                tickDuration, Long.MAX_VALUE / wheel.length));
    }

    if (duration < MILLISECOND_NANOS) {
        logger.warn("Configured tickDuration {} smaller than {}, using 1ms.",
                    tickDuration, MILLISECOND_NANOS);
        this.tickDuration = MILLISECOND_NANOS;
    } else {
        this.tickDuration = duration;
    }

    workerThread = threadFactory.newThread(worker);

    leak = leakDetection || !workerThread.isDaemon() ? leakDetector.track(this) : null;

    this.maxPendingTimeouts = maxPendingTimeouts;

    if (INSTANCE_COUNTER.incrementAndGet() > INSTANCE_COUNT_LIMIT &&
        WARNED_TOO_MANY_INSTANCES.compareAndSet(false, true)) {
        reportTooManyInstances();
    }
}

create wheel

 private static HashedWheelBucket[] createWheel(int ticksPerWheel) {
    //ticksPerWheel may not be greater than 2^30
    checkInRange(ticksPerWheel, 1, 1073741824, "ticksPerWheel");

    ticksPerWheel = normalizeTicksPerWheel(ticksPerWheel);
    HashedWheelBucket[] wheel = new HashedWheelBucket[ticksPerWheel];
    for (int i = 0; i < wheel.length; i ++) {
        wheel[i] = new HashedWheelBucket();
    }
    return wheel;
}

private static int normalizeTicksPerWheel(int ticksPerWheel) {
    int normalizedTicksPerWheel = 1;
    while (normalizedTicksPerWheel < ticksPerWheel) {
        normalizedTicksPerWheel <<= 1;
    }
    return normalizedTicksPerWheel;
}

addition of tasks

 @Override
public Timeout newTimeout(TimerTask task, long delay, TimeUnit unit) {
    checkNotNull(task, "task");
    checkNotNull(unit, "unit");

    long pendingTimeoutsCount = pendingTimeouts.incrementAndGet();

    if (maxPendingTimeouts > 0 && pendingTimeoutsCount > maxPendingTimeouts) {
        pendingTimeouts.decrementAndGet();
        throw new RejectedExecutionException("Number of pending timeouts ("
            + pendingTimeoutsCount + ") is greater than or equal to maximum allowed pending "
            + "timeouts (" + maxPendingTimeouts + ")");
    }

    start();

    // Add the timeout to the timeout queue which will be processed on the next tick.
    // During processing all the queued HashedWheelTimeouts will be added to the correct HashedWheelBucket.
    long deadline = System.nanoTime() + unit.toNanos(delay) - startTime;

    // Guard against overflow.
    if (delay > 0 && deadline < 0) {
        deadline = Long.MAX_VALUE;
    }
    HashedWheelTimeout timeout = new HashedWheelTimeout(this, task, deadline);
    timeouts.add(timeout);
    return timeout;
}

execution method

 /**
    * Starts the background thread explicitly.  The background thread will
    * start automatically on demand even if you did not call this method.
    *
    * @throws IllegalStateException if this timer has been
    *                               {@linkplain #stop() stopped} already
    */
public void start() {
    switch (WORKER_STATE_UPDATER.get(this)) {
        case WORKER_STATE_INIT:
            if (WORKER_STATE_UPDATER.compareAndSet(this, WORKER_STATE_INIT, WORKER_STATE_STARTED)) {
                workerThread.start();
            }
            break;
        case WORKER_STATE_STARTED:
            break;
        case WORKER_STATE_SHUTDOWN:
            throw new IllegalStateException("cannot be started once stopped");
        default:
            throw new Error("Invalid WorkerState");
    }

    // Wait until the startTime is initialized by the worker.
    while (startTime == 0) {
        try {
            startTimeInitialized.await();
        } catch (InterruptedException ignore) {
            // Ignore - it will be ready very soon.
        }
    }
}

stop method

 @Override
public Set<Timeout> stop() {
    if (Thread.currentThread() == workerThread) {
        throw new IllegalStateException(
                HashedWheelTimer.class.getSimpleName() +
                        ".stop() cannot be called from " +
                        TimerTask.class.getSimpleName());
    }

    if (!WORKER_STATE_UPDATER.compareAndSet(this, WORKER_STATE_STARTED, WORKER_STATE_SHUTDOWN)) {
        // workerState can be 0 or 2 at this moment - let it always be 2.
        if (WORKER_STATE_UPDATER.getAndSet(this, WORKER_STATE_SHUTDOWN) != WORKER_STATE_SHUTDOWN) {
            INSTANCE_COUNTER.decrementAndGet();
            if (leak != null) {
                boolean closed = leak.close(this);
                assert closed;
            }
        }

        return Collections.emptySet();
    }

    try {
        boolean interrupted = false;
        while (workerThread.isAlive()) {
            workerThread.interrupt();
            try {
                workerThread.join(100);
            } catch (InterruptedException ignored) {
                interrupted = true;
            }
        }

        if (interrupted) {
            Thread.currentThread().interrupt();
        }
    } finally {
        INSTANCE_COUNTER.decrementAndGet();
        if (leak != null) {
            boolean closed = leak.close(this);
            assert closed;
        }
    }
    return worker.unprocessedTimeouts();
}

What is a Multi-Level Timing Wheel?

The multi-level time wheel is easy to understand. The clock has hours, minutes, and seconds. A second turns a circle (Round) and a minute turns a tick. Tick).

PS: Apparently HashedWheelTimer is a layer of time wheel.

Sample source code

https://github.com/realpdai/tech-pdai-spring-demos

more content

Say goodbye to fragmented learning, one-stop systematic learning without routines Back-end development: Java full stack knowledge system https://pdai.tech