JAVA language asynchronous non-blocking design pattern (principle)

封面2.png

There are 2 articles in this series, which are used to popularize the asynchronous and non-blocking mode of the Java language. "Principles" explains the principles of the asynchronous non-blocking model and the basic characteristics of the core design pattern "Promise". "Applications" will show more abundant application scenarios, introduce Promise variants, such as exception handling, scheduling strategies, etc., and compare Promises with existing tools.

Limited to personal level and space, this series focuses on popular science, and the content focuses more on principles, API design, and application practices, but will not explain the specific details of concurrent optimization in depth.

1 Overview

Asynchronous non-blocking [A] is a high-performance threading model that is widely used in IO-intensive systems. Under this model, the system does not need to wait for a response after initiating a time-consuming request, during which other operations can be performed; when a response is received, the system receives a notification and performs subsequent processing. Since unnecessary waiting is eliminated, this model can make full use of resources such as cpu, threads, and improve resource utilization.

However, while the asynchronous non-blocking mode improves performance, it also brings complexity in coding implementation. The request and response may be separated into different threads, and additional code needs to be written to complete the delivery of the response result. The Promise design pattern can reduce this complexity and encapsulate the implementation details of data transmission, timing control, thread safety, etc., so as to provide a concise API form.

This article first introduces the asynchronous non-blocking mode, and analyzes the difference between blocking and non-blocking modes from the perspective of the thread model. Then introduce the application scenarios and workflow of the Promise design pattern. Finally, provide a simple Java implementation that can achieve basic functional requirements and achieve thread safety.

Before officially exploring the technical issues, let's take a look at what is the asynchronous non-blocking model . As shown in Figure 1-1, it shows a scene where two villains communicate:

The two villains represent two threads communicating with each other, such as the client and server of the database; they can be deployed on different machines.
The villains deliver apples to each other, which means that they want pass the message . According to specific business scenarios, these messages may be called request, response, packet, document, record, etc.
The villains need establish a channel before the message can be transmitted. According to the scenario, the channel is called channel, connection, etc.

Suppose the villain on the left initiates a request, and the villain on the right processes the request and sends a response: the villain on the left first casts an apple request, which is received by the villain on the right; the villain on the right processes it, then casts an apple response, received by the villain on the left. We examine the behavior of the villain on the left while waiting for a response. According to whether he can handle other tasks while waiting for the response, we summarize it into two modes: "synchronous blocking" and "asynchronous non-blocking".

Figure 1-1 Two villains communicate

First, let's look at the flow of synchronous blocking communication, as shown in Figure 1-2a.

deliver . The villain on the left delivers the request and waits to receive the response.
wait for . While waiting to receive the response, the villain on the left rests. Regardless of whether there are other requests that need to be delivered, or whether there are other tasks that need to be processed, he will ignore it and will never interrupt the rest.
responds to . After receiving the response, the villain wakes up from the rest and processes the response.

Figure 1-2a Synchronous blocking communication

Next, we look at the asynchronous non-blocking communication process, as shown in Figure 1-2b.

caches . The villain on the left delivers the request and waits to receive the response. Unlike the synchronous blocking mode, the villain does not need to personally pick up the apple response, but places a plate called "buffer" on the ground; if the villain is temporarily absent, the received apples can be stored in the plate first. Deal with it later.
temporarily . Due to the existence of the plate buffer, the villain can leave temporarily after the request is delivered to handle other tasks, and of course, he can also deliver the next request; if the request needs to be delivered to a different channel, the villain can put a few more plates , And channel one-to-one correspondence.
responds to . After the villain leaves, once a certain plate receives a response, a "big horn" will sound and send out a "channelRead" notification, calling the villain to come back to deal with the response. If you want to process multiple responses or multiple channels, the channelRead notification also needs to carry parameters to indicate which response was received from which channel.

The big speakers here can be implemented with NIO or AIO . To put it simply, NIO refers to continuously polling each plate and sending a notification as soon as an apple is seen; AIO refers to directly triggering a notification when an apple is received, without a polling process. Of course, readers of this series of articles do not need to know more implementation details, just know that the asynchronous non-blocking mode relies on the "big speaker" to achieve, it replaces the villain waiting to receive the response, thereby freeing the villain to handle other tasks.

Figure 1-2b Asynchronous non-blocking communication

According to the above analysis, the synchronization pattern having the following serious disadvantages :

synchronous blocking mode is very inefficient . The villain spends most of the time resting. Only when delivering requests and processing responses, he occasionally wakes up to work for a while; while in asynchronous non-blocking mode, the villain never rests, delivering requests, processing responses, or processing non-stop Other jobs.
synchronous blocking mode will bring delay .

We consider the following two cases, as shown in Figure 1-3.

channel multiplexes , that is, the villain on the left sends multiple messages continuously on a channel. In synchronous blocking mode, only one request (Apple 1) can be delivered in one round (request + response), and subsequent requests (Apple 2-4) can only be queued, and the villain on the right needs to wait many rounds to receive All the news expected. In addition, while the villain on the left was waiting to receive a certain response, he had no chance to process other messages received, which caused a delay in data processing. I have to sigh that the villain on the left is too lazy!
thread , that is, one thread (the villain) sends messages to multiple channels (Apple 1-3, respectively sent to different channels). The villain on the left can only do one thing at the same time, either at work or resting; he lay down to rest after delivering Apple 1, waiting for a response, ignoring the villains 2 and 3 on the right are still waiting for them to want Apples 2, 3.

Figure 1-3a Channel multiplexing

Figure 1-3b Thread reuse

In this chapter, we use the form of comics to initially experience the synchronous blocking mode and asynchronous non-blocking mode, and analyze the difference between the two modes. Next, we start with the Java thread and conduct a more formal and practical analysis of the two modes.

2. Asynchronous non-blocking model

2.1 Java thread status

In a Java program, a thread is the unit of scheduling execution. Threads can obtain CPU usage rights to execute code, thereby accomplishing meaningful work. During the work, it sometimes pauses due to waiting for lock acquisition, waiting for network IO, etc. It is generally called "synchronization" or "blocking"; if multiple tasks can be performed at the same time, there is no constraint between them, and there is no need to wait for each other. It is called "asynchronous" or "non-blocking".
Limited by memory, the number of system threads, and context switching overhead, Java programs cannot create unlimited threads; therefore, we can only create a limited number of threads and try to improve the utilization of threads, that is, increase their working time and reduce blocking time. Asynchronous non-blocking model is an effective means to reduce blocking and improve thread utilization. Of course, this model does not eliminate all blocking. Let's first take a look at the state of the Java thread, which blocking is necessary, and which blocking can be avoided.

Java thread status includes:

RUNNABLE : The thread is performing meaningful work
As shown in Figure 2-1a, if the thread is performing pure memory operations, it is in the RUNNABLE state
According to whether the right to use the cpu is obtained, it is divided into two sub-states: READY, RUNNING
BLOCKED/WAITING/TIMED_WAITING : the thread is blocking
As shown in Figure 2-1b, 2-1c, 2-1d, according to the blocking reason, the thread is in one of the following states
BLOCKED: synchronized waiting to acquire the lock
WAITING/TIMED_WAITING: Lock is waiting to acquire the lock. The difference between the two states is whether to set the timeout period

图2-1.png
Figure 2-1 Java thread status

In addition, if the Java thread is performing network IO, the thread state is RUNNABLE, but it is actually blocked. Take socket programming as an example, as shown in Figure 2-2, InputStream.read() will block before receiving data, and the thread status is RUNNABLE at this time.

图2-2.png
Figure 2-2 Network IO

In summary, Java thread status includes: RUNNABLE, BLOCKED, WAITING, TIMED_WAITING. Among them, the RUNNABLE state is divided into two situations: memory computing (non-blocking) and network IO (blocking), while the rest of the states are all blocked.
According to the blocking reasons, this article summarizes the Java thread status into the following three categories: RUNNABLE, IO, BLOCKED

RUNNABLE : The Java thread state is RUNNABLE, and it is performing useful memory calculations without blocking
IO : Java thread status is RUNNABLE, but network IO is in progress, which is blocked
BLOCKED : The Java thread status is BLOCKED/WAITING/TIMED_WAITING. Under the control of the concurrent tool, the thread waits to acquire a certain kind of lock and is blocked

To improve thread utilization, it is necessary to increase the time the thread is in the RUNNABLE state and reduce the time it is in the IO and BLOCKED state. The BLOCKED state is generally inevitable, because the threads need to communicate, and the critical section needs to be controlled concurrently; however, if an appropriate thread model is adopted, the duration of the IO state can be reduced, and this is the asynchronous non-blocking model.

2.2 Thread model: blocking vs non-blocking

The asynchronous non-blocking model can reduce the IO blocking time and improve thread utilization. Let's take database access as an example to analyze the threading model of synchronous and asynchronous APIs. As shown in Figure 3, three functions are involved in the process:

writeSync() or writeAsync(): database access, send request
process(result): Process server response (represented by result)
doOtherThings(): any other operation, logically not dependent on server response

synchronization API is shown in Figure 3-a: the caller first sends the request, and then waits for the response data from the server on the network connection. The API will block until the response is received; during this time, the caller thread cannot perform other operations, even if the operation does not depend on the server response. The actual execution order is:

writeSync()
process(result)
doOtherThings() // Until the result is received, the current thread can perform other operations

asynchronous API is shown in Figure 2-3b: the caller sends a request and registers a callback, then the API returns immediately, and the caller can perform any operation. Later, the underlying network connection receives the response data and triggers the callback registered by the caller. The actual execution order is:

writeAsync()
doOtherThings() // Other operations can already be performed without waiting for a response
process(result)

图2-3.png
Figure 2-3 Synchronous API & Asynchronous API

In the above process, the function doOtherThings() does not depend on the server response, and can be executed simultaneously with database access in principle. However, for the synchronous API, the caller is forced to wait for the server to respond before executing doOtherThings(); that is, the thread is blocked in the IO state during database access and cannot perform other useful operations, and the utilization rate is very low. The asynchronous API does not have this limitation, and is more compact and efficient.

In IO-intensive systems, proper use of asynchronous non-blocking models can improve database access throughput. Consider a scenario where multiple database access requests need to be executed, and the requests are independent of each other and have no dependencies. Using synchronous API and asynchronous API, the process of thread state changes over time is shown in Figure 2-4.
The thread is alternately in RUNNABLE and IO states. In the RUNNABLE state, threads perform memory calculations, such as submitting requests and processing responses. In the IO state, the thread waits for response data on the network connection. In the actual system, the memory calculation speed is very fast, and the duration of the RUNNABLE state is basically negligible; while the network transmission time will be relatively longer (tens to hundreds of milliseconds), and the duration of the IO state is even more considerable.

a. Synchronous API : The caller thread can only submit one request at a time; the next request cannot be submitted until the request returns. The thread utilization is very low, and most of the time is spent on the IO state.

b. asynchronous API : The caller thread can submit multiple requests in a row, and the previously submitted requests have not received a response. The caller thread will register some callbacks, which are stored in memory; after receiving the response data on the network connection, a receiving thread is notified to process the response data, retrieve the registered callback from the memory, and trigger the callback. Under this model, requests can be submitted continuously and responded continuously, thereby saving time-consuming IO status.

图2-4.png
Figure 2-4 Thread timeline: database access

Asynchronous non-blocking mode is widely used in IO-intensive systems. Commonly used middleware, such as http request[D], redis[E], mongo DB[F], elasticsearch[G], influx DB[H], all support asynchronous API. Readers can refer to the sample code of these asynchronous APIs in the reference. Regarding the asynchronous API of middleware, there are a few notes below:

The common clients of redis are jedis and lettuce [E]. Among them, lettuce provides asynchronous API, while jedis can only provide synchronous API; see the article [I] for the comparison between the two.
Kafka producer[J]’s send() method also supports asynchronous APIs, but the API is not actually pure asynchronous [K]: When the underlying cache is full or the server (broker) information cannot be obtained, the send() method will block . Personally think this is a very serious design flaw. Kafka is often used in low-latency log collection scenarios. The system will write logs to the Kafka server through the network to reduce the congestion in the thread and improve the thread throughput; later other processes will consume the written logs from Kafka for persistent storage . Imagine a real-time communication system. A single thread needs to process tens of thousands to hundreds of thousands of messages per second, and the response time is generally from a few milliseconds to tens of milliseconds. The system needs to call send() frequently to report the log during processing. If there is a delay of even 1 second (in fact, it may be up to tens of seconds) for each call, the accumulation of delay will seriously degrade throughput and delay.

Finally, there are multiple implementations of asynchronous API, including thread pool, select (such as netty 4.x[L]), epoll, etc. The common point is that the caller does not need to block on a certain network connection to wait to receive data; on the contrary, the bottom layer of the API resides in a limited number of threads. When data is received, a certain thread is notified and triggers a callback. This model is also called the "responsive" model, which is very appropriate. Due to space limitations, this article mainly focuses on the asynchronous API design , and does not explain the implementation principle of the asynchronous API in depth.

3. Promise design pattern

3.1 API form: synchronous, asynchronous listener, asynchronous Promise

The previous chapter introduced the asynchronous non-blocking mode and the functional form of the asynchronous API. Asynchronous API has the following characteristics:

Register the callback when submitting the request;
After submitting the request, the function returns immediately without waiting for a response;
After receiving the response, the registered callback is triggered; according to the underlying implementation, a limited number of threads can be used to receive the response data, and callbacks can be executed in these threads.

On the basis of retaining the asynchronous characteristics, the form of the asynchronous API can be further optimized. Figure 2-3b in the previous chapter shows the listener version of the asynchronous API. The feature is that exactly one callback must be registered when submitting the request; therefore, in the following scenarios, the listener API will not meet the functional requirements and requires the caller to do further processing:

Multiple objects are concerned with response data, that is, multiple callbacks need to be registered; however, the listener only supports the registration of one callback.
Need to convert asynchronous calls to synchronous calls. For example, some frameworks (such as spring) need to return synchronously, or we want the main thread to block until the operation is completed, and then the main thread ends and the process exits; but the listener only supports pure asynchrony, and the caller needs to repeatedly write code from asynchronous to synchronous.

In order to cope with the above scenarios, we can use the Promise design pattern to refactor the asynchronous API to support multiple callbacks and synchronous calls. The following compares the functional forms of synchronous API, asynchronous listener API, and asynchronous Promise API, as shown in Figure 3-1:

a. Synchronous : call the writeSync() method and block; after receiving the response, the function stops blocking and returns the response data
b. Asynchronous listener : call the writeAsync() method and register the listener, the function returns immediately; after receiving the response, trigger the registered listener in other threads;
c. Asynchronous Promise : call writeAsync(), but there is no need to register the listener in the function, and the function returns the Promise object immediately. The caller can call the asynchronous Promise.await(listener), register any number of listeners, and trigger them in order after receiving the response; in addition, they can also call the synchronous Promise.await() to block until the response is received.

图3-1.png
Figure 3-1 API form: synchronous, asynchronous listener, asynchronous Promise

In summary, the Promise API provides greater flexibility while maintaining asynchronous features. The caller can freely choose whether the function is blocked, and register any number of callbacks.

3.2 Features and Implementation of Promise

The previous section introduced a sample of the Promise API. Its core is a Promise object, which supports registering listeners and obtaining the response result synchronously; and this section will define the function of Promise in more detail. Note that this section does not limit a specific implementation of Promise (for example: jdk CompletableFuture, netty DefaultPromise), but only shows the common and must-have features; without these features, Promise will not be able to complete the work of asynchronously delivering response data.

3.2.1 Features

The basic method of Promise

The basic function of Promise is to transmit response data, and the following methods need to be supported, as shown in Table 3-1:

表格.jpg

Take the database access API in the previous section as an example to demonstrate the workflow of Promise, as shown in Figure 3-2:

a. The caller calls the writeAsync() API, submits the database access request and obtains the Promise object; then calls Promise.await(listener) to register the listener for the response data. Promise objects can also be passed to other places in the program, so that other codes that care about the response data can register more listeners.
b.writeAsync() creates a Promise object and associates it with this request, assuming it is identified by requestId.
c.writeAsync() has a limited number of threads resident at the bottom layer for sending requests and receiving responses. Take netty as an example. After receiving the response data from the network, one of the threads is notified and executes the channelRead() function for processing; the function takes out the response data and the corresponding Promise object, and calls Promise.signalAll() to notify. Note that this is pseudo code, which is slightly different from the actual signature of the callback function in netty.

图3-2a.png
Figure 3-2a Submit database access request

图3-2b.png
Figure 3-2b Create Promise object

图3-2c.png
Figure 3-2c Notify Promise Object

-Promise timing

The Promise method needs to guarantee the following timing. Here, "A is visible to B" is used to describe the timing, that is, if you perform operation A (register listener) first, it will produce a permanent effect (permanently record this listener), and then perform operation B (notify result) must be considered For this effect, perform the corresponding processing (trigger the listener recorded before).

await(listener) is visible to signalAll(result): after registering several listeners, each listener must be triggered when the result is notified, and omissions are not allowed.
signalAll(result) is visible to await(listener): after notifying the result, registering the listener will trigger immediately.
The first signalAll(result) is visible to subsequent signalAll(result). After the result is notified for the first time, the result is uniquely determined and will never change. After that, the result will be ignored and no side effects will be produced. Request timeout is a typical application of this feature: a timed task is created while submitting the request; if the response data can be correctly received within the timeout period, the Promise is notified that the Promise ends normally; otherwise, the timed task times out, and the Promise is notified that the Promise ends abnormally. Regardless of which event occurs first, it is guaranteed that only the first notification will be adopted, so that the request result is uniquely determined.

In addition, an await(listener) is best visible to subsequent await(listener) to ensure that the listener is triggered in strict accordance with the registration order.

-Non-thread-safe implementation of Promise

If thread safety is not considered, the following code listing can implement the basic features of Promise; see the next section for the implementation of thread safety. The code listing shows the realization of await(listener): void, signalAll(result), and await(): result in turn. Here are a few considerations for :

registered by await(listener). The field type is LinkedList to store any number of listeners while maintaining the trigger sequence of the listeners.
field isSignaled records whether the result has been notified. If isSignaled=true, the listener will be triggered immediately when await(listener) is called subsequently, and it will be ignored when signalAll(result) is called subsequently. In addition, we use isSignaled=true instead of result=null to judge whether result has been notified, because in some cases null itself can also be used as response data. For example, we use Promise<Exception> to indicate the result of the database write, to notify the null to indicate the success of the write, and to notify the Exception object (or a certain subclass) to indicate the reason for the failure.
signalAll(T result) calls listeners.clear() at the end to release the memory , because the listeners have already been triggered and no longer need to be stored in the memory.

public class Promise<T> {

    private boolean isSignaled = false;
    private T result;

    private final List<Consumer<T>> listeners = new LinkedList<>();

    public void await(Consumer<T> listener) {
        if (isSignaled) {
            listener.accept(result);
            return;
        }

        listeners.add(listener);
    }

    public void signalAll(T result) {
        if (isSignaled) {
            return;
        }

        this.result = result;
        isSignaled = true;
        for (Consumer<T> listener : listeners) {
            listener.accept(result);
        }
        listeners.clear();
    }

    public T await() {
        // 适当阻塞，直至signalAll()被调用；实际实现见3.3节
        return result;
    }
}

3.2.2 Thread safety features

Section 3.2.1 of the previous chapter explained the function of Promise and provided a non-thread-safe implementation. This section shows how to use concurrency tools to implement thread-safe Promises, as shown below. There are several considerations:

Thread safety. Each field is accessed by multiple threads, so all belong to the critical area, and need to use appropriate thread safety tools for locking, such as synchronized and Lock. One of the simplest implementations is to put all the code into the critical section, lock when entering the method, and release the lock when leaving the method. Note that when using return for early return, don't forget to put the lock.
Trigger the listener outside the critical area to reduce the length of stay in the critical area and reduce the potential risk of deadlock.
Synchronous await(). You can use any kind of synchronous waiting tool to achieve, such as CountDownLatch, Condition. The condition is implemented here. Note that according to the java grammar, the lock associated with the Condition must be acquired first when operating the Condition.

public class Promise<T> {

    private final ReentrantLock lock = new ReentrantLock();
    private final Condition resultCondition = lock.newCondition();

    private boolean isSignaled = false;
    private T result;

    private final List<Consumer<T>> listeners = new LinkedList<>();

    public void await(Consumer<T> listener) {
        lock.lock();
        if (isSignaled) {
            lock.unlock(); // 不要忘记放锁
            listener.accept(result); // 在临界区外触发listener
            return;
        }

        listeners.add(listener);
        lock.unlock();
    }

    public void signalAll(T result) {
        lock.lock();
        if (isSignaled) {
            lock.unlock(); // 不要忘记放锁
            return;
        }

        this.result = result;
        isSignaled = true;

        // this.listeners的副本
        List<Consumer<T>> listeners = new ArrayList<>(this.listeners);
        this.listeners.clear();
        lock.unlock();

        for (Consumer<T> listener : listeners) {
            listener.accept(result); // 在临界区外触发listener
        }

/* 操作Condition时须上锁*/
        lock.lock();
        resultCondition.signalAll();
        lock.unlock();
    }

    public T await() {
        lock.lock();
        if (isSignaled) {
            lock.unlock(); // 不要忘记放锁
            return result;
        }

        while (!isSignaled) {
            resultCondition.awaitUninterruptibly();
        }
        lock.unlock();

        return result;
    }
}

The above implementation is only for demonstration use, and there is still much room for improvement. For the realization principle of the production environment, readers can refer to jdk CompletableFutre and netty DefaultPromise. Areas that can be improved include:

uses CAS to set the response data . The fields isSignaled and result can be combined into one data object, and then set using CAS to further reduce the blocking time.
triggers the listener's timing . In the above code, Promise.signalAll() will trigger the listener in turn; during this period, if other threads call asynchronous await(listener), since the Promise response data has been notified, the thread will also trigger the listener. In the above process, two threads trigger the listener at the same time, so the trigger sequence is not strictly guaranteed. As an improvement, similar to netty DefaultPromise, a loop can be set inside Promise.signalAll() to continuously trigger listeners until listeners are empty, in case a new listener is registered during the period; during this period, the newly registered listener can be directly added to the listeners Instead of triggering immediately.
listener removal . Before notifying the response data, Promise holds a reference to the listener for a long time, causing the listener object to be unable to be gc. You can add the remove(listener) method, or allow only weak references to the listener.

3.2.3 Features to avoid

The previous section showed the characteristics and implementation principles of Promise. A pure Promise is a tool for asynchronously delivering response data. It should only implement necessary data delivery features and should not be mixed with logic such as request submission and data processing. Next, let's take a look at what features should be avoided when Promise is implemented in order not to limit the decisions the caller can make.

1. Asynchronous await() is blocked; this rule applies not only to Promises, but also to any asynchronous API. Asynchronous APIs are often used in delay-sensitive scenarios such as real-time communication. The function is to reduce thread blocking and avoid delaying other subsequent operations. Once congestion occurs, the response speed and throughput of the system will be severely impacted.

Take continuous submission of database requests as an example. As shown in Figure 3-3a, the caller calls an asynchronous API, submits three write requests in a row, and registers a callback on the returned Promise.

We examine writeAsync() and await() if blocking occurs, what effect will it have on the caller, as shown in Figure 3-3b. Submitting a request is a pure memory operation, and the thread is in the RUNNABLE state; if writeAsync() or await() is blocked, the thread is in the BLOCKED state, and the work is suspended and subsequent operations cannot be performed. When blocking occurs, the caller has to wait for a period of time every time a request is submitted, thereby reducing the frequency of submitting requests, and then delaying the server's response to these requests, causing the system's throughput to decrease and latency to increase. In particular, if the system uses a multiplexing mechanism, that is, one thread can handle multiple network connections or multiple requests, then thread blocking will seriously slow down the processing of subsequent requests and cause more difficult troubleshooting.

Common blocking reasons include:

Thread.sleep()
Submit tasks to the queue, call BlockingQueue.put() and take(); should be changed to non-blocking offer() and poll()
Submit a task to the thread pool, ExecutorService.submit(), if the thread pool rejection policy is CallerRunsPolicy, and the task itself is time-consuming.
The blocking functions are called, including: InputStream.read(), synchronous Promise.await(), KafkaProducer.send(). Note that KafkaProducer.send() is an asynchronous API in form, but when the underlying cache is full or the server (broker) information cannot be obtained, the send() method will still block.

图2-1a.png
Figure 3-3a Continuous submission of requests

图3-3b.png
Figure 3-3b Request processing timeline

2. Binding thread pool (ExecutorService), used to execute requests. As shown in Figure 3-4, thread pool is an optional model of asynchronous API, but it is not the only implementation.

thread pool model . In order not to block the caller, the API has a built-in thread pool to submit requests and process responses; the caller can continuously submit multiple requests to the thread pool, but does not need to wait for a response. After the caller submits a request, a thread in the thread pool will be monopolized, waiting to receive a response and process it, but no other requests can be processed before that; after the processing is completed, the thread becomes idle again and can continue Process subsequent requests.
responsive model . Similarly, the API has built-in sending and receiving threads to submit requests and process responses, and the caller does not need to wait synchronously. After the caller submits a request, the sending thread sends the request to the network; after finishing the sending, the thread becomes idle immediately and can send subsequent requests. When the response data is received, the receiving thread is notified to process the response; after the processing is completed, the thread becomes idle immediately and can process subsequent response data. In the above process, any thread will not be monopolized by a request, that is, the thread can process the request at any time without waiting for the previous request to be responded.

In summary, if the thread pool is bound, Promise will achieve compatibility with other models (such as the responsive model).

图3-4.png

Figure 3-4 Thread timeline: thread pool vs select

3. When creating a Promise object in the constructor, define how to submit a request. This method can only define how to process a single request, but cannot achieve batch processing of requests.

Taking database access as an example, modern databases generally support batch reads and writes. At the cost of slightly increasing the latency of a single access, the throughput is significantly improved; if the throughput is improved, the average latency will decrease instead. The following code snippet shows a batch request API: The data object BulkRequest can carry multiple ordinary request Requests to realize batch submission.

/* 提交单条请求*/
client.submit(new Request(1));
client.submit(new Request(2));
client.submit(new Request(3));

/* 提交批量请求*/
client.submit(new BulkRequest(
        new Request(1),
        new Request(2),
        new Request(3)
));

In order to take full advantage of the "batch request" feature, the caller needs to carry out "macro-control" across multiple requests. After a request is generated, it can be cached first; after waiting for a period of time, multiple cached requests are taken out, and a batch request is assembled and submitted together. Therefore, as shown in the following code snippet, it is meaningless to specify how to submit a single request when constructing a Promise. This part of the code (client.submit(new Request(...))) will not be executed; but the actual hope The code executed is actually to submit a bulk request (client.submit(new BulkRequest(...))).

/* Promise：提交单条请求*/
new Promise<>(() -> client.submit(new Request(1)));
new Promise<>(() -> client.submit(new Request(2)));
new Promise<>(() -> client.submit(new Request(3)));

4. When the construction method creates the Promise object, define how to process the response data, and not allow subsequent registration of callbacks to the response data. As shown in the following code snippets, when constructing the Promise object, the processing process (result) of the response data is registered; but in addition, other codes may also care about the response data and need to register the callback process1 (result), process2 (result) ). If a Promise can only register a unique callback when it is constructed, other followers cannot register the required callback function, that is, the Promise API degenerates back to the listener API.

/* 定义如何处理响应数据*/
Promise<String> promise = new Promise<>(result -> process(result));

/* 其他代码也关心响应数据*/
promise.await(result -> process1(result));
promise.await(result -> process2(result));

In summary, Promise should be a pure data object. Its responsibility is to store callback functions and store response data; at the same time, it should do a good job of timing control to ensure that there are no omissions in the trigger callback function and the trigger sequence. In addition, Promise should not be coupled with any implementation strategy, and the logic of submitting requests and processing responses should not be mixed.

4. Summary

This article explains the asynchronous non-blocking design pattern, and compares the synchronous API, asynchronous listener API, and asynchronous Promise API. Compared with the other two APIs, the Promise API has unparalleled flexibility. The caller can freely decide whether to return synchronously or asynchronously, and allows multiple callback functions to be registered for the response data. Finally, this article explains the implementation of the basic functions of Promise, and initially realizes the thread safety features.

There are 2 articles in this series, this article is the first "Principle". In the next article "Applications", we will see the rich application scenarios of Promise design patterns, combine or compare them with existing tools, and further transform and encapsulate the Promise API to provide features such as exception handling and scheduling strategies. .