Disruptor notes seven: waiting strategy

Welcome to my GitHub

https://github.com/zq2599/blog_demos

Content: Classification and summary of all original articles and supporting source code, involving Java, Docker, Kubernetes, DevOPS, etc.;

"Disruptor Notes" series link

1. Quick Start
2. Disruptor class analysis
3. Basic operation of circular queue (without Disruptor class)
4. Summary of event consumption knowledge points
5. Event consumption actual combat
6. Common scenarios
7. Waiting strategy
8. Knowledge Point Supplement (Final)

Overview of this article

This article is the seventh chapter of "Notes to Disruptor". Let's read the source code together and learn an important knowledge point: waiting strategy. Because the source code of Disruptor is short, lean and easy to understand, this article is a relaxing and pleasant source learning journey;

Summary in advance

If you don't have enough time, you can get a general understanding of the waiting strategy through the following summary in advance:

1. BlockingWaitStrategy: Uses ReentrantLock's wait && wake mechanism to implement waiting logic, which is the default strategy and saves CPU
2. BusySpinWaitStrategy: continuous spinning, use it with caution under JDK9 (preferably not)
3. DummyWaitStrategy: The returned Sequence value is 0, which is not used in a normal environment
4. LiteBlockingWaitStrategy: Based on BlockingWaitStrategy, the wake-up operation will be omitted when there is no lock contention, but the author said that the test is not sufficient and it is not recommended to use
5. TimeoutBlockingWaitStrategy: Waiting with timeout, after the timeout, the processing logic specified by the business will be executed
6. LiteTimeoutBlockingWaitStrategy: Based on TimeoutBlockingWaitStrategy, the wake-up operation will be omitted when there is no lock competition
7. SleepingWaitStrategy: Three-stage, the first stage is spinning, the second stage executes Thread.yield to hand over the CPU, the third stage sleep execution time, repeated sleep
8. YieldingWaitStrategy: Two-stage, spin in the first stage, and execute Thread.yield in the second stage to hand over the CPU
9. PhasedBackoffWaitStrategy: Four-stage, the first stage spins the specified number of times, the second stage spins the specified time, the third stage executes Thread.yield to hand over the CPU, the fourth stage calls the waitFor method of the member variable, this member variable can be set to One of the three BlockingWaitStrategy, LiteBlockingWaitStrategy, SleepingWaitStrategy

About waiting strategy

• Recall the code to instantiate the Disruptor in the previous article:

disruptor = new Disruptor<>(new OrderEventFactory(),
                BUFFER_SIZE,
                new CustomizableThreadFactory("event-handler-"));

• Expand the above construction method, you will see the code to create the RingBuffer, which uses BlockingWaitStrategy as the waiting strategy by default:

    public static <E> RingBuffer<E> createMultiProducer(EventFactory<E> factory, int bufferSize)
    {
        return createMultiProducer(factory, bufferSize, new BlockingWaitStrategy());
    }

• Continue to expand the above <font color="blue">createMultiProducer</font> method, it can be seen that each Sequencer (note that not Sequence) has its own watStrategy member variable:

• The final use of this waitStrategy is to pass to SequenceBarrier as member variables when creating SequenceBarrier:

• Let’s take a look at how SequenceBarrier uses waitStrategy. It is used in two places. The first place is the red box as shown in the figure below. It turned out to be used internally in the <font color="blue">waitFor</font> method. This <font color= "blue">waitFor</font> we have already learned that for consumers, when there is data available in the specified position of the circular queue, it is done by calling the waitFor of SequenceBarrier:

• The second part of SequenceBarrier uses waitStrategy when it wakes up:

    @Override
    public void alert()
    {
        alerted = true;
        waitStrategy.signalAllWhenBlocking();
    }

• Now that we know the usage scenarios of WaitStrategy, let's take a look at the specific implementations of this interface, so that we know how to choose in programming to best suit us

BlockingWaitStrategy

• As the default waiting strategy, BlockingWaitStrategy is also characterized by its small amount of code (less than a hundred lines), which is easy to understand. In fact, it uses ReentrantLock+Condition to achieve waiting and wake-up operations, as shown in the red box below:

• If you are more inclined to save CPU resources and have relatively low requirements for high throughput and low latency, then BlockingWaitStrategy is for you;

BusySpinWaitStrategy (use with caution)

• The previous BlockingWaitStrategy has a feature that once data arrives at the designated position of the circular queue, because the thread is in a waiting state (the bottom layer calls the native UNSAFE.park method), business logic can only be executed after waking up. In some scenarios, data is expected. Consume as soon as it arrives. At this time, the BusySpinWaitStrategy is very suitable. The code is too simple, so post it all:

public final class BusySpinWaitStrategy implements WaitStrategy
{
    @Override
    public long waitFor(
        final long sequence, Sequence cursor, final Sequence dependentSequence, final SequenceBarrier barrier)
        throws AlertException, InterruptedException
    {
        long availableSequence;

        while ((availableSequence = dependentSequence.get()) < sequence)
        {
            barrier.checkAlert();
            ThreadHints.onSpinWait();
        }

        return availableSequence;
    }

    @Override
    public void signalAllWhenBlocking()
    {
    }
}

• The above code shows that the key to the entire while loop is what <font color="blue">ThreadHints.onSpinWait</font> does. The source code is as follows. Pay special attention here, if <font color="blue">ON_SPIN_WAIT_METHOD_HANDLE</font > Is empty, which means that the outer while loop is a <font color="red"> very consuming the spin of the CPU </font>:

    public static void onSpinWait()
    {
        if (null != ON_SPIN_WAIT_METHOD_HANDLE)
        {
            try
            {
                ON_SPIN_WAIT_METHOD_HANDLE.invokeExact();
            }
            catch (final Throwable ignore)
            {
            }
        }
    }

• ON_SPIN_WAIT_METHOD_HANDLE being empty is a terrible thing, let’s see where it is sacred? The code is still in ThreadHints.java, as shown below, the truth is clear at a glance, <font color="blue">it is the onSpinWait method of the Thread class</font>, if the Thread class does not have an onSpinWait method, then use BusySpinWaitStrategy as a waiting strategy The price is very high. When there is no data in the circular queue, the consuming thread will perform spin, which consumes CPU:

static
    {
        final MethodHandles.Lookup lookup = MethodHandles.lookup();

        MethodHandle methodHandle = null;
        try
        {
            methodHandle = lookup.findStatic(Thread.class, "onSpinWait", methodType(void.class));
        }
        catch (final Exception ignore)
        {
        }

        ON_SPIN_WAIT_METHOD_HANDLE = methodHandle;
    }

• Well, there are two questions left: Is it uncertain whether the Thread class has an onSpinWait method? What is the sacred method of this onSpinWait?
• Look at the official JDK documentation, as shown in the figure below. It turns out that this method is only available from JDK9, so for JDK8 users, choosing BusySpinWaitStrategy means facing a while loop that doesn't do anything:

• The second question, what does the onSpinWait method do? The previous official documents are obviously incomprehensible based on 's English level. See 1614a801d1baba stackoverflow , as shown in the figure below. Simply put, it tells the CPU that the current thread is in a loop query state, and the CPU will schedule more when it learns. Give more CPU resources to other threads:

• So far, it really looks like a big white: After the conditions of the circular queue are ready, the BusySpinWaitStrategy strategy uses a whlie infinite loop to achieve rapid response. If the JDK is version 9 or higher, the CPU loss caused by this infinite loop is helped to alleviate by Thread.onSpinWait. If the JDK version is lower than 9, here is a simple infinite while loop. As for how much CPU is consumed by this infinite loop, you can write a simple code to get a feel...
• No wonder the Disruptor source code will remind <font color="blue">it is best to bind the thread using this instance to the specified CPU core</font>:

DummyWaitStrategy

Fixed return to 0, I personally feel that this strategy is not useful in normal development, because if the available position of the circular queue is always 0, it is difficult to achieve whether it is production or consumption:

LiteBlockingWaitStrategy

• Look at the name, LiteBlockingWaitStrategy is a lightweight implementation of the BlockingWaitStrategy strategy. When there is no lock competition (such as independent consumption scenarios), the wake-up operation will be omitted, but as shown in the red box in the following figure, the author said that he has not fully verified the correctness , So it is recommended only for experience, great, this strategy <font color="red"> I will not learn! ! ! </font>

TimeoutBlockingWaitStrategy

• As the name suggests, TimeoutBlockingWaitStrategy means only waiting for a certain period of time, even if it exceeds the time, its code is similar to BlockingWaitStrategy, but there is a time limit when waiting, as shown in the figure below, which is clear at a glance:

• In fact, I am very interested in the handling after throwing an exception. Let’s take a look. Outside is the familiar BatchEventProcessor class and the familiar processEvents method, as shown in the figure below. Each timeout exception is given to <font color="blue">notifyTimeout< /font> processing, while the external main process is not affected, it still keeps waiting and getting data from the circular queue:

• Enter the notifyTImeout method, it can be seen that it is actually handed over to the member variable <font color="blue">timeoutHandler</font> for processing, and any exceptions that occur during processing will be caught, and will not be thrown to affect external calls:

• Let’s take a look at where the member variables come from. As shown in the figure below, the truth is clear. If the EventHandler implementation class we developed also implements Timeouthandler, it will be treated as a member variable timeoutHandler:

• So far, TimeoutBlockingWaitStrategy has also figured out: it is used in time-limited scenarios, and the business-customized timeout processing logic will be called after each waiting timeout. This logic is written in the EventHandler implementation class, and this implementation class should implement the Timeouthandler interface

LiteTimeoutBlockingWaitStrategy

• The relationship between LiteTimeoutBlockingWaitStrategy and TimeoutBlockingWaitStrategy is like the relationship between BlockingWaitStrategy and LiteBlockingWaitStrategy: as a variant of TimeoutBlockingWaitStrategy, it has the timeout processing feature of TimeoutBlockingWaitStrategy, and when there is no lock competition, the wake-up operation is omitted;
• The author said that LiteBlockingWaitStrategy can be used for experience, but the correctness has not been fully verified. However, in the comments of LiteTimeoutBlockingWaitStrategy, <font color="red"> did not see this statement </font>, which seems to be reliable The waiting strategy can be used in scenarios where there is a need for overtime processing and there is no lock contention (for example, independent consumption)

SleepingWaitStrategy

• The difference from the previous few is that SleepingWaitStrategy does not use locks, which means that there is no need to call the signalAllWhenBlocking method for wake-up processing, which is equivalent to eliminating the notification operation of the production thread. The official source code comment has such a sentence that arouses my interest, as follows The red box in the figure shows that the strategy has achieved a balance between performance and CPU resource consumption. Next, let’s take a look at the key code to understand this feature:

• As shown in the figure below, the process of waiting until the available data is an endless loop:

• Next is the key code, as shown in the figure below, it can be seen that the whole waiting process is divided into three sections: when the counter is higher than 100, there is only one operation of subtracting one (the fastest response), and every time the counter is between 100 and 0, it is handed over CPU execution time (the most resource-saving), other times it sleeps for a fixed time:

YieldingWaitStrategy

• After reading the SleepingWaitStrategy, it is easy to understand the YieldingWaitStrategy again. Compared with the SleepingWaitStrategy, the YieldingWaitStrategy first does a specified number of spins, and then continuously surrenders the CPU time:

• Because the <font color="blue">Thread.yield()</font> method is continuously executed, although this strategy consumes CPU, it is easy to get it from this thread once other threads have CPU demand;

PhasedBackoffWaitStrategy

• Finally, there is the PhasedBackoffWaitStrategy. The feature of this strategy is to divide the entire waiting process into four segments as shown in the figure below. The four squares represent the four phases on a timeline:

• Here are the four stages of the above picture:

1. The first is the specified number of spins, the default is 10000 times;
2. After the spin, spin with timing starts, and the execution time is the value of spinTimeoutNanos;
3. After the execution time reaches the value of spinTimeoutNanos, start executing <font color="blue">Thread.yield()</font> to hand over CPU resources. The execution time of this logic is <font color="red">yieldTimeoutNanos-spinTimeoutNanos< /font>;
4. After the execution time reaches the value of <font color="red">yieldTimeoutNanos-spinTimeoutNanos</font>, start calling <font color="blue">fallbackStrategy.waitFor</font>. There is no time or number of restrictions on this call;
5. Now the question is what is <font color="blue">fallbackStrategy</font>? The PhasedBackoffWaitStrategy class prepares three static methods, which we can choose as needed, let fallbackStrategy be one of the three: BlockingWaitStrategy, LiteBlockingWaitStrategy, and SleepingWaitStrategy:

public static PhasedBackoffWaitStrategy withLock(
        long spinTimeout,
        long yieldTimeout,
        TimeUnit units)
    {
        return new PhasedBackoffWaitStrategy(
            spinTimeout, yieldTimeout,
            units, new BlockingWaitStrategy());
    }

public static PhasedBackoffWaitStrategy withLiteLock(
        long spinTimeout,
        long yieldTimeout,
        TimeUnit units)
    {
        return new PhasedBackoffWaitStrategy(
            spinTimeout, yieldTimeout,
            units, new LiteBlockingWaitStrategy());
    }

    public static PhasedBackoffWaitStrategy withSleep(
        long spinTimeout,
        long yieldTimeout,
        TimeUnit units)
    {
        return new PhasedBackoffWaitStrategy(
            spinTimeout, yieldTimeout,
            units, new SleepingWaitStrategy(0));
    }

• At this point, the nine waiting strategies of Disruptor have all been analyzed. In addition to being more handy when choosing the waiting strategy, another gain is that you have accumulated experience in reading excellent source code, and you are more confident on the way to reading source code;

You are not alone, Xinchen and original are with you all the way

1. Java series
2. Spring series
3. Docker series
4. kubernetes series
5. database + middleware series
6. DevOps series

Welcome to pay attention to the public account: programmer Xin Chen

Search "Programmer Xin Chen" on WeChat, I am Xin Chen, and I look forward to traveling the Java world with you...
https://github.com/zq2599/blog_demos