Java JUC ThreadPoolExecutor parsing

Thread pool ThreadPoolExecutor

introduce

The thread pool mainly solves two problems: First, the thread pool can provide better performance when a large number of asynchronous tasks are executed. When the thread pool is not used, a new thread needs to be executed every time a task needs to be performed, and frequent creation and destruction consumes performance. The threads in the thread pool can be reused, and there is no need to recreate and destroy each time a task needs to be performed. Second, the thread pool provides means of resource limitation and management, such as limiting the number of threads and dynamically increasing threads.

In addition, the thread pool also provides many tunable parameters and extensibility interfaces to meet the needs of different situations. We can use the more convenient Executors factory method to create different types of thread pools, or we can customize our own threads. pool.

The working mechanism of the thread pool

1. When the thread pool is first created, there are no threads. When a new request comes, the core thread will be created to process the corresponding request.
2. When the processing is complete, the core thread will not be recycled
3. Each request creates a new core thread in the thread pool until the specified number of core threads is reached
4. When the core threads are all occupied, new requests are put into the work queue. The work queue is essentially a blocking queue
5. When the work queue is full, new requests will be handed over to temporary threads for processing
6. The temporary thread will continue to live for a period of time after it is used, and will not be destroyed until no request is processed

Class Diagram Introduction

As shown in the above class diagram, Executors is a tool class that provides a variety of static methods and provides different thread pool instances according to our choices.

ThreadPoolExecutor inherits the AbstractExecutorService abstract class. In ThreadPoolExecutor, the member variable ctl is an Integer atomic variable, which is used to record the state of the thread pool and the number of threads in the thread, similar to ReentrantReadWriteLock using a variable to store two kinds of information.

Assuming that the Integer type is a 32-bit binary representation, the upper 3 bits represent the status of the thread pool, and the last 29 bits represent the number of threads in the thread pool.

//默认RUNNING状态，线程个数为0
private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));

Get high 3 bits, running status

private static int runStateOf(int c)     { return c & ~CAPACITY; }

Get the lower 29 bits, the number of threads

private static int workerCountOf(int c)  { return c & CAPACITY; }

The meaning of thread pool status is as follows:

• RUNNING : accept new tasks and process tasks in blocking queue
• SHUTDOWN : reject new tasks but process tasks in blocking queue
• STOP : Reject new tasks and discard tasks in the blocking queue, while interrupting ongoing tasks
• TIDYING : After all tasks are executed (including tasks in the blocking queue), the number of active threads in the current thread pool is 0, and the terminated method will be called
• TERMINATED : Termination status. The state after the terminated method is called

The thread pool state transition is as follows:

• RUNNING -> SHUTDOWN: explicitly call the shutdown method, or implicitly call the shutdown method in the finalize method
• RUNNING or SHUTDOWN -> STOP : when the shutdownNow method is explicitly called
• SHUTDOWN -> TIDYING: when both the thread pool and task queue are empty
• STOP -> TIDYING: when the thread pool is empty
• TIDYING -> TERMINATED: when the terminated hook method is executed

The thread pool parameters are as follows:

parameter name	type	meaning
corePoolSize	int	number of core threads
maxPoolSize	int	maximum number of threads
keepAliveTime	long	keep alive time
workQueue	BlockingQueue	task storage queue
threadFactory	ThreadFactory	When the thread pool needs new threads, use ThreadFactory to create new threads
Handler	RejectedExecutionHandler	The rejection policy given by the thread pool cannot accept the submitted task

• corePoolSize : refers to the number of core threads. After the thread pool initialization is completed, by default, the thread pool does not have any threads. The thread pool will wait for the arrival of the task, and then create a new thread to execute the task.

• maxPoolSize : The thread pool may add some additional threads to the number of core threads, but these newly added threads have an upper limit, and the maximum cannot exceed maxPoolSize.

  • If the number of threads is less than corePoolSize, a new thread is created to run the task even if other worker threads are idle.
  • If the number of threads is greater than or equal to corePoolSize but less than maxPoolSize, the task is put into the work queue.
  • If the queue is full and the number of threads is less than maxPoolSize, a new thread is created to run the task.
  • If the queue is full and the number of threads is greater than or equal to maxPoolSize, the task is rejected using the rejection policy.
• keepAliveTime : If a thread is in an idle state and the current number of threads is greater than corePoolSize, the idle thread will be destroyed after the specified time. The specified time here is set by keepAliveTime.

• workQueue : After a new task is submitted, it will enter this work queue first, and then take out the task from the queue when the task is scheduled. There are four kinds of work queues provided in jdk:

  • ArrayBlockingQueue : Array-based bounded blocking queue, sorted by FIFO. When a new task comes in, it will be placed at the end of the queue. A bounded array can prevent resource exhaustion. When the number of threads in the thread pool reaches corePoolSize and a new task comes in, the task will be placed at the end of the queue, waiting to be scheduled. If the queue is already full, a new thread is created, and if the number of threads has reached maxPoolSize, the rejection policy is executed.
  • LinkedBlockingQueue : unbounded blocking queue based on linked list (in fact, the maximum capacity is Interger.MAX), sorted by FIFO. Due to the approximate unbounded nature of the queue, when the number of threads in the thread pool reaches corePoolSize, and new tasks come in, they will always be stored in the queue without creating new threads until maxPoolSize. Therefore, when using this work queue, the parameter maxPoolSize Actually it doesn't work.
  • SynchronousQueue : A blocking queue that does not cache tasks. The producer puts a task in and must wait until the consumer takes it out. That is to say, when a new task comes in, it will not be cached, but will be directly scheduled to execute the task. If there is no available thread, a new thread will be created, and if the number of threads reaches maxPoolSize, the rejection policy will be executed.
  • PriorityBlockingQueue : unbounded blocking queue with priority, the priority is realized by the parameter Comparator.
  • delayQueue : delay unbounded blocking queue with priority.
  • LinkedTransferQueue : unbounded blocking queue based on linked list.
  • LinkedBlockingDeque : double-ended blocking queue based on linked list.
• threadFactory : The factory used when creating a new thread, which can be used to set the thread name, whether it is a daemon thread, etc.
• handler : When the tasks in the work queue have reached the maximum limit, and the number of threads in the thread pool has also reached the maximum limit, if a new task is submitted, the rejection policy will be executed.

As shown in the ThreadPoolExecutor class diagram above, the mainLock is an exclusive lock, which is used to control the atomicity of the new Worker thread operation. termination is the condition queue corresponding to the lock, which is used to store the blocked thread when the thread calls awaitTermination.

private final ReentrantLock mainLock = new ReentrantLock();
private final HashSet<Worker> workers = new HashSet<Worker>();
private final Condition termination = mainLock.newCondition();

The Worker class inherits the AQS and Runnable interfaces, and is an object that carries specific tasks. Worker inherits AQS and implements a simple non-reentrant exclusive lock. state = 0 means the lock has not been acquired, state = 1 means the lock has been acquired, state = -1 is the default state when creating a Worker, and the state is set to -1 when creating it to avoid The thread was interrupted before running the runWorker method. The variable firstTask records the first task executed by the worker thread, and thread is the thread that executes the task.

private final class Worker extends AbstractQueuedSynchronizer implements Runnable{
    final Thread thread;
    Runnable firstTask;
    //...

DefaultThreadFactory is a thread factory, and the newThread method is a modification of the thread. Among them, poolNumber is a static atomic variable used to count the number of thread factories, and threadNumber is used to record how many threads are created by each thread factory. These two values are also used as part of the name of the thread pool and thread.

Source code analysis

execute method

The main function of the execute method is to submit the task command to the thread pool for execution.

As can be seen from the figure, the implementation of ThreadPoolExecutor is actually a producer-consumer model . When the user adds tasks to the thread pool, it is equivalent to the producer producing elements. The threads in the worker threads directly execute tasks or obtain tasks from the task queue. So consumers consume elements.

The specific code is as follows:

public void execute(Runnable command) {
        //1. 校验任务是否为null
        if (command == null)
            throw new NullPointerException();
        //2. 获取当前线程池的状态+线程个数的组合值
        int c = ctl.get();
        //3. 判断线程池中线程个数是否小于corePoolSize，小则开启新线程（core线程）运行
        if (workerCountOf(c) < corePoolSize) {
            if (addWorker(command, true))
                return;
            c = ctl.get();
        }
        //4. 如果线程池处于RUNNING状态，则添加任务到阻塞队列
        if (isRunning(c) && workQueue.offer(command)) {
            //4.1 二次检查
            int recheck = ctl.get();
            //4.2 如果当前线程池状态不是RUNNING则从队列中删除任务，并执行拒绝策略
            if (! isRunning(recheck) && remove(command))
                reject(command);
            //4.3 如果当前线程池为空，则添加一个线程
            else if (workerCountOf(recheck) == 0)
                addWorker(null, false);
        }
        //5. 如果队列满，则新增线程，新增失败则执行拒绝策略
        else if (!addWorker(command, false))
            reject(command);
    }

If the number of threads in the current thread pool is greater than or equal to corePoolSize, code (4) is executed, and if the current thread is in RUNNING, it is added to the task queue.

It should be noted that the thread pool status is judged here because it is possible that the thread pool is already in a non-RUNNING state, and new tasks will be discarded in a non-RUNNING state.

If the task is added successfully, execute the code (4.2) for secondary verification, because the state of the thread pool may be changed before executing 4.2. If the thread pool state is not RUNNING, remove the task from the task queue, and then execute the rejection policy ; If the second verification passes, then re-determine whether there are threads in the thread pool, and if not, add a new thread.

If the code (4) fails to add the task, it means that the queue is full, and then execute the code (5) to try to add a new thread, that is, thread3 and thread4 in the above figure, to execute the task. If the current number of thread pools > maximumPoolSize, the execution will be rejected. Strategy.

Let's look at the addWorker method next:

private boolean addWorker(Runnable firstTask, boolean core) {
    retry:
   // 每次for循环都需要获取最新的ctl值
    for (;;) {
        int c = ctl.get();
        int rs = runStateOf(c);
                //1. 检查队列是否只在必要时为空
        if (rs >= SHUTDOWN &&
            ! (rs == SHUTDOWN &&
               firstTask == null &&
               ! workQueue.isEmpty()))
            return false;
              //2. 循环CAS增加线程个数
        for (;;) {
            int wc = workerCountOf(c);
            //2.1 如果线程个数超限制则返回false
            if (wc >= CAPACITY ||
                wc >= (core ? corePoolSize : maximumPoolSize))
                return false;
            //2.2 通过CAS增加线程个数
            if (compareAndIncrementWorkerCount(c))
                break retry;
            //2.3 CAS失败后，查看线程状态是否发生变化，如果变化则跳转到外层循环重新尝试获取线程池状态，否则内层循环重新进行CAS
            c = ctl.get();
            if (runStateOf(c) != rs)
                continue retry;
        }
    }

      //3. 执行到这一步说明CAS成功
    boolean workerStarted = false;
    boolean workerAdded = false;
    Worker w = null;
    try {
        //3.1 创建Worker
        w = new Worker(firstTask);
        final Thread t = w.thread;
        if (t != null) {
            //3.2 增加独占锁，实现同步，因为可能多个线程同时调用线程池的execute方法
            final ReentrantLock mainLock = this.mainLock;
            mainLock.lock();
            try {
                //3.3 重新检查线程池状态，避免在获取锁前调用了shutdown
                int rs = runStateOf(ctl.get());
                if (rs < SHUTDOWN ||
                    (rs == SHUTDOWN && firstTask == null)) {
                    if (t.isAlive()) // precheck that t is startable
                        throw new IllegalThreadStateException();
                    //3.4 添加任务
                    workers.add(w);
                  // 更新当前最大线程数量 maximumPoolSize 和 corePoolSize可以在线程池创建之后动态修改的
                    int s = workers.size();
                    if (s > largestPoolSize)
                        largestPoolSize = s;
                    workerAdded = true;
                }
            } finally {
                mainLock.unlock();
            }
            //3.5 添加成功后启动任务
            if (workerAdded) {
                t.start();
                workerStarted = true;
            }
        }
    } finally {
       // 如果没有执行过t.start() 就要把这个woker从workers里面删除，并且ctl里面worker数量减1
        if (! workerStarted)
            addWorkerFailed(w);
    }
    return workerStarted;
}

First, it is clear that the purpose of the first part of the double loop is to add the number of threads through the CAS operation, and the second part is to safely add tasks to workers through ReentrantLock, and then start the task.

First look at the first part of the code (1).

if (rs >= SHUTDOWN &&
            ! (rs == SHUTDOWN && //(1)
               firstTask == null && //(2)
               ! workQueue.isEmpty())) //(3)

Code (1) will return false in the following three cases:

• (1) The current thread pool state is STOP, TIDYING or TERMINATED
• (2) The current thread pool state is SHUTDOWN and there is already the first task
• (3) The current thread pool state is SHUTDOWN and the task queue is empty

The role of the inner loop of code (2) is to use the CAS operation to increase the number of threads.

When the code (8) is executed, it means that the number of threads has been successfully increased through CAS, and now the task has not started to execute, so this part of the code adds Worker to the work set workers through global lock control.

Worker thread Worker execution

After the user thread is submitted to the thread pool, it is executed by the Worker. The following is the constructor of the Worker.

Worker(Runnable firstTask) {
    setState(-1); // inhibit interrupts until runWorker
    this.firstTask = firstTask;
    this.thread = getThreadFactory().newThread(this);//创建一个线程
}

In the constructor, first set the Worker's state to -1, in order to avoid the current Worker being interrupted before calling the runWorker method (when other threads call the shutdownNow of the thread pool, if the Worker's state >= 0, the thread will be interrupted). Here the thread's state is set to -1, so the thread will not be interrupted.

In the runWorker code, when the code (1) is run, the unlock method is called, which sets the status to 0, so calling shutdownNow at this time will interrupt the Worker thread.

final void runWorker(Worker w) {
    Thread wt = Thread.currentThread();
    Runnable task = w.firstTask;
    w.firstTask = null;
    w.unlock(); //1. 将state设置为0，允许中断
    boolean completedAbruptly = true;
    try {
        //2.
        while (task != null || (task = getTask()) != null) {
            //2.1
            w.lock();
            ////如果状态值大于等于STOP且当前线程还没有被中断，则主动中断线程
            if ((runStateAtLeast(ctl.get(), STOP) ||
                 (Thread.interrupted() &&
                  runStateAtLeast(ctl.get(), STOP))) &&
                !wt.isInterrupted())
                wt.interrupt();
            try {
                //2.2 执行任务前处理操作，默认是一个空实现；在子类中可以通过重写来改变任务执行前的处理行为
                beforeExecute(wt, task);
                Throwable thrown = null;
                try {
                    //2.3 执行任务
                    task.run();
                } catch (RuntimeException x) {
                    thrown = x; throw x;
                } catch (Error x) {
                    thrown = x; throw x;
                } catch (Throwable x) {
                    thrown = x; throw new Error(x);
                } finally {
                    //2.4 任务之后处理，同beforeExecute
                    afterExecute(task, thrown);
                }
            } finally {
                task = null;
                //2.5 统计当前worker完成了多少个任务
                w.completedTasks++;
                w.unlock();
            }
        }
        //设置为false，表示任务正常处理完成
        completedAbruptly = false;
    } finally {
        //3. 清理工作
        processWorkerExit(w, completedAbruptly);
    }
}

The purpose of locking during the execution of a specific task is to avoid the task being executed after other threads call shutdown from being interrupted during the task running (shutdown will only interrupt the currently blocked and suspended thread)

The cleanup working code is as follows:

private void processWorkerExit(Worker w, boolean completedAbruptly) {
    // 如果completedAbruptly为true则表示任务执行过程中抛出了未处理的异常
    // 所以还没有正确地减少worker计数，这里需要减少一次worker计数
    if (completedAbruptly)
        decrementWorkerCount();

      //1. 统计线程池中完成任务的个数，并从工作集合里面删除当前Worker
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        completedTaskCount += w.completedTasks;
        workers.remove(w);
    } finally {
        mainLock.unlock();
    }
        //1.2 尝试设置线程池状态为TERMINATED，在关闭线程池时等到所有worker都被回收后再结束线程池
    tryTerminate();
        //1.3 如果线程池状态 < STOP，即RUNNING或SHUTDOWN，则需要考虑创建新线程来代替被销毁的线程
    int c = ctl.get();
    if (runStateLessThan(c, STOP)) {
        // 如果worker是正常执行完的，则要判断一下是否已经满足了最小线程数要求
        // 否则直接创建替代线程
        if (!completedAbruptly) {
            int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
            if (min == 0 && ! workQueue.isEmpty())
                min = 1;
            if (workerCountOf(c) >= min)
                return; // replacement not needed
        }
        // 重新创建一个worker来代替被销毁的线程
        addWorker(null, false);
    }
}

In the above code, the code (1.1) counts the number of tasks completed by the thread pool, and adds a global lock before the statistics. Accumulates the tasks done in the current worker thread to a global counter, then removes the current worker from the working set.

The code (1.2) judges that if the current thread pool state is SHUTDOWN and the work queue is empty, or the current thread pool state is STOP and there are no active threads in the current thread pool, set the thread pool state to TERMINATED. If it is set to the TERMINATED state, you also need to call the signalAll method of the condition variable termination to activate all threads blocked by calling the awaitTermination method of the thread pool.

The code (1.3) determines whether the number of threads in the current thread pool is less than the number of core threads, and if so, a new thread is added.

shutdown method

After calling the shutdown method, the thread pool will no longer accept new tasks, but the tasks in the work queue still need to be executed. The method returns immediately, and does not wait for the queued task to complete before returning.

public void shutdown() {
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        //1. 权限检查
        checkShutdownAccess();
        //2. 设置当前线程池状态为SHUTDOWN，如果已经是SHUTDOWN则直接返回
        advanceRunState(SHUTDOWN);
        //3. 设置中断标志
        interruptIdleWorkers();
        onShutdown(); // hook for ScheduledThreadPoolExecutor
    } finally {
        mainLock.unlock();
    }
    //4. 尝试将状态改为TERMINATED
    tryTerminate();
}

First check whether the currently calling thread has permission to close the thread.

The code for subsequent code (2) is as follows. If the current thread pool state is >= SHUTDOWN, return directly, otherwise set to SHUTDOWN state.

private void advanceRunState(int targetState) {
    for (;;) {
        int c = ctl.get();
        if (runStateAtLeast(c, targetState) ||
            ctl.compareAndSet(c, ctlOf(targetState, workerCountOf(c))))
            break;
    }
}

The source code of code (3) is as follows, which sets the interrupt flag of all idle threads. The global lock is first added here, and only one thread can call the shutdown method to set the interrupt flag. Then try to acquire the worker's own lock, and set the interrupt flag if the acquisition is successful. Since the executing task has acquired the lock, the executing task is not interrupted. What is interrupted here is the thread blocking to the getTask method and trying to get the task from the queue, that is, the idle thread.

private void interruptIdleWorkers(boolean onlyOne) {
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        for (Worker w : workers) {
            Thread t = w.thread;
            if (!t.isInterrupted() && w.tryLock()) {
                try {
                    t.interrupt();
                } catch (SecurityException ignore) {
                } finally {
                    w.unlock();
                }
            }
            if (onlyOne)
                break;
        }
    } finally {
        mainLock.unlock();
    }
}

Finally, try to change the state to TERMINATED. First, use CAS to set the current thread pool state to TIDYING. If the setting is successful, execute the extended interface terminated to do something before the thread pool state becomes TERMINATED, and then set the current thread pool state to TERMINATED. Finally, call termination.signalAll to activate all threads blocked by calling the await series methods of the condition variable termination.

final void tryTerminate() {
    for (;;) {
        int c = ctl.get();
        if (isRunning(c) ||
            runStateAtLeast(c, TIDYING) ||
            (runStateOf(c) == SHUTDOWN && ! workQueue.isEmpty()))
            return;
        if (workerCountOf(c) != 0) { // Eligible to terminate
            interruptIdleWorkers(ONLY_ONE);
            return;
        }

        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            if (ctl.compareAndSet(c, ctlOf(TIDYING, 0))) {
                try {
                    terminated();
                } finally {
                    ctl.set(ctlOf(TERMINATED, 0));
                    termination.signalAll();
                }
                return;
            }
        } finally {
            mainLock.unlock();
        }
        // else retry on failed CAS
    }
}

shutdownNow method

After the shutdownNow method is called, the thread pool will no longer accept new tasks, and the tasks in the work queue will be discarded, the tasks being executed will be interrupted, and the method will return immediately without waiting for the completion of the activated tasks. The return value is the list of tasks discarded in the queue at this time.

public List<Runnable> shutdownNow() {
    List<Runnable> tasks;
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        //1. 权限检查
        checkShutdownAccess();
        //2. 设置线程池状态为STOP
        advanceRunState(STOP);
        //3. 中断所有线程
        interruptWorkers();
        //4. 将任务队列移动到tasks中
        tasks = drainQueue();
    } finally {
        mainLock.unlock();
    }
    tryTerminate();
    return tasks;
}

It should be noted that all interrupted threads include idle threads and threads that are executing tasks.

awaitTermination method

When the thread calls the awaitTermination method, the current thread will be blocked until the thread pool state changes to TERMINATED before returning, or the waiting time expires.

public boolean awaitTermination(long timeout, TimeUnit unit)
        throws InterruptedException {
        long nanos = unit.toNanos(timeout);
        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            for (;;) {
                if (runStateAtLeast(ctl.get(), TERMINATED))
                    return true;
                if (nanos <= 0)
                    return false;
                nanos = termination.awaitNanos(nanos);
            }
        } finally {
            mainLock.unlock();
        }
}

First obtain an exclusive lock, and then judge whether the current thread pool state is at least TERMINATED in the infinite loop. If so, return directly. Otherwise, it means that there are still threads executing in the current thread pool. Check whether the set timeout nanos is less than 0. If it is less than 0, it means that there is no need to wait, then return directly. If it is greater than 0, call the awaitNanos method of the condition variable termination to wait for the nanos time, and expect the thread pool state to become TERMINATED during this time.

Summarize

The thread pool cleverly uses an atomic variable of type Integer to record the thread pool state and the number of threads in the thread pool. The execution of tasks is controlled by the state of the thread pool, and each worker thread can process multiple tasks. The thread pool reduces the overhead of thread creation and destruction through the reuse of threads.