Reasonable use of thread pools and thread variables

This article starts from the principle and use of thread pools and thread variables, combined with examples to give best practices to help developers build stable and efficient java application services.

background

With the continuous development of computing technology, 3nm process chips have entered the trial production stage, and Moore's Law is gradually facing a huge physical bottleneck under the existing technology. Improving the performance of servers through multi-core processor technology has become the main direction for improving computing power.

In the server field, the back-end server based on java occupies a leading position. Therefore, mastering java concurrent programming technology and making full use of the concurrent processing capability of the CPU are the basic skills that a developer must learn. This article briefly introduces the thread pool source code and practice. Use of thread pools and thread variables.

Thread Pool Overview

▐ What is a thread pool

The thread pool is a "pooled" thread usage mode. By creating a certain number of threads, these threads are in a ready state to improve the system response speed. After the threads are used, they are returned to the thread pool to achieve the goal of reuse. Reduce the consumption of system resources.

▐ Why use thread pool

In general, thread pools have the following advantages:

Reduce resource consumption. Reduce the cost of thread creation and destruction by reusing already created threads.

Improve responsiveness. When a task arrives, the task can be executed immediately without waiting for the thread to be created.

Improve thread manageability. Threads are scarce resources. If they are created without restrictions, it will not only consume system resources, but also reduce the stability of the system. Using thread pools can be used for unified allocation, tuning, and monitoring.

Use of thread pool

▐ Thread pool creation & core parameter settings

In java, the implementation class of the thread pool is ThreadPoolExecutor, and the constructor is as follows:

 public ThreadPoolExecutor(int corePoolSize,
                              int maximumPoolSize,
                              long keepAliveTime,
                              TimeUnit timeUnit,
                              BlockingQueue<Runnable> workQueue,
                              ThreadFactory threadFactory,
                              RejectedExecutionHandler handler)

A thread pool can be created by new ThreadPoolExecutor(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler).

corePoolSize parameter

In the constructor, corePoolSize is the number of core threads in the thread pool. By default, core threads will always live, but when allowCoreThreadTimeout is set to true, core thread timeouts are also recycled.

maximumPoolSize parameter

In the constructor, maximumPoolSize is the maximum number of threads that the thread pool can hold.

keepAliveTime parameter

In the constructor, keepAliveTime represents the thread idle timeout period. If the thread is idle for longer than this time, the non-core thread will be recycled. If allowCoreThreadTimeout is set to true, core threads will also timeout for recycling.

timeUnit parameter

In the constructor, timeUnit represents the time unit for the thread idle timeout period. Commonly used are: TimeUnit.MILLISECONDS (milliseconds), TimeUnit.SECONDS (seconds), TimeUnit.MINUTES (minutes).

blockingQueue parameter

In the constructor, blockingQueue represents the task queue, and the common implementation classes of the thread pool task queue are:

ArrayBlockingQueue : A bounded blocking queue implemented by an array. This queue sorts the elements according to the FIFO principle and supports fair access to the queue.

LinkedBlockingQueue : An optional bounded blocking queue consisting of a linked list structure. If no size is specified, Integer.MAX_VALUE is used as the queue size, and elements are sorted according to the FIFO principle.

PriorityBlockingQueue : An unbounded blocking queue that supports priority sorting. By default, it is arranged in natural order. Comparator can also be specified.

DelayQueue: An unbounded blocking queue that supports delayed acquisition of elements. When creating an element, you can specify how long it will take to get the current element from the queue. It is often used in cache system design and timing task scheduling.

SynchronousQueue: A blocking queue that does not store elements. A deposit operation must wait for a get operation, and vice versa.

LinkedTransferQueue: An unbounded blocking queue composed of a linked list structure. Compared with LinkedBlockingQueue, there are more transfer and tryTranfer methods. This method will immediately deliver elements to consumers when there are consumers waiting to receive elements.

LinkedBlockingDeque: A double-ended blocking queue consisting of a linked list structure that can insert and delete elements from both ends of the queue.

threadFactory parameter

In the constructor, threadFactory represents the thread factory. Used to specify the way to create new threads for the thread pool, threadFactory can set parameters such as thread name, thread group, priority, etc. For example, the thread name in the thread pool can be set through the Google toolkit:

 new ThreadFactoryBuilder().setNameFormat("general-detail-batch-%d").build()

The RejectedExecutionHandler parameter is in the constructor, and rejectedExecutionHandler represents the rejection strategy. The rejection policy that needs to be executed when the maximum number of threads is reached and the queue tasks are full. Common rejection policies are as follows:

ThreadPoolExecutor.AbortPolicy: The default policy that throws RejectedExecutionException when the task queue is full.

ThreadPoolExecutor.DiscardPolicy: Discards new tasks that cannot be executed without throwing any exceptions.

ThreadPoolExecutor.CallerRunsPolicy: When the task queue is full, use the caller's thread to execute the task directly.

ThreadPoolExecutor.DiscardOldestPolicy: When the task queue is full, the task at the head of the blocking queue (ie, the oldest task) is discarded, and then the current task is added.

▐ Thread pool state transition diagram

The ThreadPoolExecutor thread pool has the following states:

RUNNING: Running state, accepting new tasks, and continuously processing tasks in the task queue;

SHUTDOWN: no longer accept new tasks, but process tasks in the task queue;

STOP: no longer accept new tasks, no longer process tasks in the task queue, and interrupt ongoing tasks;

TIDYING: Indicates that the thread pool is stopping, suspending all tasks, destroying all worker threads, and entering the TIDYING state when the thread pool executes the terminated() method;

TERMINATED: Indicates that the thread pool has stopped working, all worker threads have been destroyed, all tasks have been emptied or completed, and the terminated() method is completed;

▐ Thread pool task scheduling mechanism

The main steps of task scheduling when the thread pool submits a task are as follows:

When the number of surviving core threads in the thread pool is less than the value of the corePoolSize core thread number parameter, the thread pool will create a core thread to process the submitted task;

If the number of core threads in the thread pool is full, that is, the number of threads is equal to corePoolSize, the newly submitted task will be tried to be put into the task queue workQueue for execution;

When the number of surviving threads in the thread pool has been equal to corePoolSize, and the task queue workQueue is full, then determine whether the current number of threads has reached the maximumPoolSize, that is, whether the maximum number of threads is full, if not, create a non-core thread to execute the submitted Task;

If the current number of threads has reached the maximumPoolSize, and new tasks are submitted, the rejection policy will be implemented for processing.

The core code is as follows:

 public void execute(Runnable command) {
        if (command == null)
            throw new NullPointerException();
        /*
         * Proceed in 3 steps:
         *
         * 1. If fewer than corePoolSize threads are running, try to
         * start a new thread with the given command as its first
         * task.  The call to addWorker atomically checks runState and
         * workerCount, and so prevents false alarms that would add
         * threads when it shouldn't, by returning false.
         *
         * 2. If a task can be successfully queued, then we still need
         * to double-check whether we should have added a thread
         * (because existing ones died since last checking) or that
         * the pool shut down since entry into this method. So we
         * recheck state and if necessary roll back the enqueuing if
         * stopped, or start a new thread if there are none.
         *
         * 3. If we cannot queue task, then we try to add a new
         * thread.  If it fails, we know we are shut down or saturated
         * and so reject the task.
         */
        int c = ctl.get();
        if (workerCountOf(c) < corePoolSize) {
            if (addWorker(command, true))
                return;
            c = ctl.get();
        }
        if (isRunning(c) && workQueue.offer(command)) {
            int recheck = ctl.get();
            if (! isRunning(recheck) && remove(command))
                reject(command);
            else if (workerCountOf(recheck) == 0)
                addWorker(null, false);
        }
        else if (!addWorker(command, false))
            reject(command);
    }

▐ Tomcat thread pool analysis

Tomcat request processing process

The overall architecture of Tomcat consists of two parts: the connector and the container. The connector is responsible for communicating with the outside world, and the container is responsible for internal logic processing. In the connector:

Use the ProtocolHandler interface to encapsulate the difference between the I/O model and the application layer protocol. The I/O model can choose non-blocking I/O, asynchronous I/O or APR, and the application layer protocol can choose HTTP, HTTPS or AJP. ProtocolHandler combines the I/O model with the application layer protocol, so that EndPoint is only responsible for sending and receiving byte streams, and Processor is responsible for parsing byte streams into Tomcat Request/Response objects to achieve high cohesion and low coupling of functional modules. ProtocolHandler interface The inheritance relationship is as shown below.

Convert the Tomcat Request object to a standard ServletRequest object through the adapter Adapter.

In order to achieve fast response to requests, Tomcat uses thread pools to improve the processing capacity of requests. Below we take HTTP non-blocking I/O as an example to briefly analyze the Tomcat thread pool.

Tomcat thread pool creation

In Tomcat, the underlying network I/O processing is provided through the AbstractEndpoint class. If the user does not configure a custom public thread pool, the AbstractEndpoint creates the Tomcat default thread pool through the createExecutor method.

The core part of the code is as follows:

 public void createExecutor() {
        internalExecutor = true;
        TaskQueue taskqueue = new TaskQueue();
        TaskThreadFactory tf = new TaskThreadFactory(getName() + "-exec-", daemon, getThreadPriority());
        executor = new ThreadPoolExecutor(getMinSpareThreads(), getMaxThreads(), 60, TimeUnit.SECONDS,taskqueue, tf);
        taskqueue.setParent( (ThreadPoolExecutor) executor);
    }

Among them, TaskQueue and ThreadPoolExecutor are implemented for Tomcat's custom task queue and thread pool respectively.

Tomcat custom ThreadPoolExecutor

Tomcat's custom thread pool inherits from java.util.concurrent.ThreadPoolExecutor, and adds some member variables to more efficiently count the number of tasks that have been submitted but not yet completed (submittedCount), including tasks that are already in the queue and tasks that have been delivered A task that a worker thread has not yet started executing.

 /**
 * Same as a java.util.concurrent.ThreadPoolExecutor but implements a much more efficient
 * {@link #getSubmittedCount()} method, to be used to properly handle the work queue.
 * If a RejectedExecutionHandler is not specified a default one will be configured
 * and that one will always throw a RejectedExecutionException
 *
 */
public class ThreadPoolExecutor extends java.util.concurrent.ThreadPoolExecutor {

    /**
     * The number of tasks submitted but not yet finished. This includes tasks
     * in the queue and tasks that have been handed to a worker thread but the
     * latter did not start executing the task yet.
     * This number is always greater or equal to {@link #getActiveCount()}.
     */
    // 新增的submittedCount成员变量，用于统计已提交但还未完成的任务数
    private final AtomicInteger submittedCount = new AtomicInteger(0);
    private final AtomicLong lastContextStoppedTime = new AtomicLong(0L);
    // 构造函数
    public ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory,
            RejectedExecutionHandler handler) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
        // 预启动所有核心线程
        prestartAllCoreThreads();
    }

}

Tomcat rewrites the execute() method in the custom thread pool ThreadPoolExecutor, and adds one to the submittedCount of the tasks submitted for execution. In Tomcat's custom ThreadPoolExecutor, when the thread pool throws a RejectedExecutionException, it will call the force() method to try to add tasks to the TaskQueue again. If the addition fails, the submittedCount is decremented by one, and then RejectedExecutionException is thrown.

 @Override
    public void execute(Runnable command) {
        execute(command,0,TimeUnit.MILLISECONDS);
    }

    public void execute(Runnable command, long timeout, TimeUnit unit) {
        submittedCount.incrementAndGet();
        try {
            super.execute(command);
        } catch (RejectedExecutionException rx) {
            if (super.getQueue() instanceof TaskQueue) {
                final TaskQueue queue = (TaskQueue)super.getQueue();
                try {
                    if (!queue.force(command, timeout, unit)) {
                        submittedCount.decrementAndGet();
                        throw new RejectedExecutionException("Queue capacity is full.");
                    }
                } catch (InterruptedException x) {
                    submittedCount.decrementAndGet();
                    throw new RejectedExecutionException(x);
                }
            } else {
                submittedCount.decrementAndGet();
                throw rx;
            }

        }
    }

Tomcat custom task queue

A blocking queue TaskQueue is redefined in Tomcat, which inherits from LinkedBlockingQueue. In Tomcat, the default number of core threads is 10, and the default maximum number of threads is 200. In order to prevent the subsequent tasks from being queued after the number of threads reaches the number of core threads, Tomcat implements the core thread pool by rewriting the offer method of the custom task queue TaskQueue Thread creation after the number reaches the configured number.

Specifically, it can be known from the implementation of the thread pool task scheduling mechanism that when the offer method returns false, the thread pool will try to create a new thread, so as to achieve a fast response of the task. The core implementation code of TaskQueue is as follows:

 /**
 * As task queue specifically designed to run with a thread pool executor. The
 * task queue is optimised to properly utilize threads within a thread pool
 * executor. If you use a normal queue, the executor will spawn threads when
 * there are idle threads and you wont be able to force items onto the queue
 * itself.
 */
public class TaskQueue extends LinkedBlockingQueue<Runnable> {

    public boolean force(Runnable o, long timeout, TimeUnit unit) throws InterruptedException {
        if ( parent==null || parent.isShutdown() ) throw new RejectedExecutionException("Executor not running, can't force a command into the queue");
        return super.offer(o,timeout,unit); //forces the item onto the queue, to be used if the task is rejected
    }

    @Override
    public boolean offer(Runnable o) {
        // 1. parent为线程池，Tomcat中为自定义线程池实例
      //we can't do any checks
        if (parent==null) return super.offer(o);
        // 2. 当线程数达到最大线程数时，新提交任务入队
        //we are maxed out on threads, simply queue the object
        if (parent.getPoolSize() == parent.getMaximumPoolSize()) return super.offer(o);
        // 3. 当提交的任务数小于线程池中已有的线程数时，即有空闲线程，任务入队即可
        //we have idle threads, just add it to the queue
        if (parent.getSubmittedCount()<=(parent.getPoolSize())) return super.offer(o);
        // 4. 【关键点】如果当前线程数量未达到最大线程数，直接返回false，让线程池创建新线程
        //if we have less threads than maximum force creation of a new thread
        if (parent.getPoolSize()<parent.getMaximumPoolSize()) return false;
        // 5. 最后的兜底，放入队列
        //if we reached here, we need to add it to the queue
        return super.offer(o);
    }   
}

Tomcat custom task thread

Tomcat implements the recording of the creation time of each thread through the custom task thread TaskThread; uses the static inner class WrappingRunnable to wrap the Runnable to handle the StopPooledThreadException exception type.

 /**
 * A Thread implementation that records the time at which it was created.
 *
 */
public class TaskThread extends Thread {

    private final long creationTime;

    public TaskThread(ThreadGroup group, Runnable target, String name) {
        super(group, new WrappingRunnable(target), name);
        this.creationTime = System.currentTimeMillis();
    }


    /**
     * Wraps a {@link Runnable} to swallow any {@link StopPooledThreadException}
     * instead of letting it go and potentially trigger a break in a debugger.
     */
    private static class WrappingRunnable implements Runnable {
        private Runnable wrappedRunnable;
        WrappingRunnable(Runnable wrappedRunnable) {
            this.wrappedRunnable = wrappedRunnable;
        }
        @Override
        public void run() {
            try {
                wrappedRunnable.run();
            } catch(StopPooledThreadException exc) {
                //expected : we just swallow the exception to avoid disturbing
                //debuggers like eclipse's
                log.debug("Thread exiting on purpose", exc);
            }
        }

    }

}

Thinking & Summary

Why does Tomcat customize the thread pool and task queue implementation?

When the JUC native thread pool submits a task, when the number of worker threads reaches the number of core threads, continuing to submit the task will try to put the task into the blocking queue, only the current number of running threads does not reach the maximum set value and after the task queue task is full , will continue to create new worker threads to process tasks, so the JUC native thread pool cannot meet Tomcat's fast response requirements.

Why does Tomcat use unbounded queues?

Tomcat uses two parameters of acceptCount and maxConnections in EndPoint to avoid excessive request backlog. where maxConnections is the maximum number of connections that Tomcat receives and processes at any time. When the number of connections Tomcat receives reaches maxConnections, the Acceptor will not read the connections in the accept queue; at this time, the threads in the accept queue will be blocked until Tomcat The number of connections received is less than maxConnections (maxConnections defaults to 10000, if set to -1, the number of connections is unlimited). acceptCount is the length of the accept queue. When the number of connections in the accept queue reaches acceptCount, the queue is full. At this time, incoming requests will be rejected. The default value is 100 (based on Tomcat 8.5.43 version). Therefore, after the two parameters of acceptCount and maxConnections are used, Tomcat's default unbounded task queue usually does not cause OOM.

 /**
 * Allows the server developer to specify the acceptCount (backlog) that
 * should be used for server sockets. By default, this value
 * is 100.
 */
private int acceptCount = 100;

private int maxConnections = 10000;

▐ Best Practices

Avoid creating thread pools with Executors

The common methods of Executors are as follows:

newCachedThreadPool(): Creates a cacheable thread pool, calling execute will reuse previously constructed threads (if threads are available). If no thread is available, a new thread is created and added to the thread pool. Threads that have not been used for 60 seconds are terminated and removed from the cache. CachedThreadPool is suitable for concurrent execution of a large number of short-term and time-consuming tasks, or servers with light load;

newFiexedThreadPool(int nThreads): Create a thread pool with a fixed number of threads. When the number of threads is less than nThreads, a new thread will be created when a new task is submitted. When the number of threads is equal to nThreads, the task will be added to the blocking queue after the new task is submitted. After the executing thread is executed, the task is taken from the queue and executed. FiexedThreadPool is suitable for scenarios with a slightly heavy load but not too many tasks. In order to utilize resources reasonably, the number of threads needs to be limited;

newSingleThreadExecutor() creates a single-threaded Executor. SingleThreadExecutor is suitable for scenarios where tasks are executed serially. Each task is executed in sequence without concurrent execution;

newScheduledThreadPool(int corePoolSize) Creates a thread pool that supports scheduled and periodic task execution, and can be used to replace the Timer class in most cases. In ScheduledThreadPool, an instance of ScheduledThreadPoolExecutor is returned, and ScheduledThreadPoolExecutor actually inherits ThreadPoolExecutor. As can be seen from the code, ScheduledThreadPool is based on ThreadPoolExecutor, corePoolSize is the incoming corePoolSize, maximumPoolSize is Integer.MAX_VALUE, timeout is 0, and workQueue is DelayedWorkQueue. In fact, ScheduledThreadPool is a scheduling pool that implements three methods: schedule, scheduleAtFixedRate, and scheduleWithFixedDelay, which can implement operations such as delayed execution and periodic execution;

newSingleThreadScheduledExecutor() creates a ScheduledThreadPoolExecutor with a corePoolSize of 1;

newWorkStealingPool(int parallelism) returns a ForkJoinPool instance. ForkJoinPool is mainly used to implement the "divide and conquer" algorithm, which is suitable for computationally intensive tasks.

The Executors class looks more powerful and more convenient to use, but it has the following disadvantages:

FiexedThreadPool and SingleThreadPool task queue length is Integer.MAX_VALUE, which may accumulate a large number of requests, resulting in OOM;

CachedThreadPool and ScheduledThreadPool allow the number of threads to be created to be Integer.MAX_VALUE, which may create a large number of threads, resulting in OOM;

When using threads, you can directly call the constructor of ThreadPoolExecutor to create a thread pool, and set parameters such as corePoolSize, blockingQueue, and RejectedExecuteHandler according to the actual business scenario.

Avoid using local thread pools

When using the local thread pool, if the shutdown() method is not executed after the task is executed or there are other improper references, it is very easy to cause system resource exhaustion.

Reasonable setting of thread pool parameters

In engineering practice, the following formula is usually used to calculate the number of core threads:

nThreads=(w+c)/c n u=(w/c+1) n u

Among them, w is the waiting time, c is the calculation time, n is the number of CPU cores (usually obtained by the Runtime.getRuntime().availableProcessors() method), and u is the CPU target utilization (the value range is [0, 1] ); in the case of maximizing CPU utilization, when the processing task is a computationally intensive task, that is, the waiting time w is 0, and the number of core threads is equal to the number of CPU cores.

The above calculation formula is the recommended number of core threads in an ideal situation, and different systems/applications may have certain differences when running different tasks. Therefore, the optimal number of threads parameters also need to be determined according to the actual operation of the task and the performance of the stress test. Fine tune.

Add exception handling

In order to better discover, analyze and solve problems, it is recommended to increase the handling of exceptions when using multi-threading. Exception handling usually has the following solutions:

Add try...catch exception handling at the task code

If you use the Future method, you can receive the thrown exception through the get method of the Future object

Set the setUncaughtExceptionHandler for the worker thread and handle the exception in the uncaughtException method

Gracefully shut down the thread pool

 public void destroy() {
        try {
            poolExecutor.shutdown();
            if (!poolExecutor.awaitTermination(AWAIT_TIMEOUT, TimeUnit.SECONDS)) {
                poolExecutor.shutdownNow();
            }
        } catch (InterruptedException e) {
            // 如果当前线程被中断，重新取消所有任务
            pool.shutdownNow();
            // 保持中断状态
            Thread.currentThread().interrupt();
        }
    }

In order to achieve the goal of graceful shutdown, we should call the shutdown method first. Calling this method means that the thread pool will not receive any new tasks, but the tasks that have been submitted will continue to be executed. After that, we should also call the awaitTermination method. This method can set the maximum timeout period of the thread pool before closing. If the thread pool can be closed normally before the timeout period expires, it will return true, otherwise, the timeout will return false. Usually we need to estimate a reasonable timeout according to the business scenario, and then call this method.

If the awaitTermination method returns false, but you want to do other resource recycling work after the thread pool is closed as much as possible, you can consider calling the shutdownNow method again. At this time, all tasks that have not been processed in the queue will be discarded, and the thread pool will be set at the same time. The interrupt flag bit for each thread in . shutdownNow does not guarantee that a running thread will stop working unless the task submitted to the thread responds properly to the interrupt.

Hawkeye context parameter passing

 /**
* 在主线程中，开启鹰眼异步模式，并将ctx传递给多线程任务
**/
// 防止鹰眼链路丢失，需要传递
RpcContext_inner ctx = EagleEye.getRpcContext();
// 开启异步模式
ctx.setAsyncMode(true);


/**
* 在线程池任务线程中，设置鹰眼rpc环境
**/
private void runTask() {
    try {
        EagleEye.setRpcContext(ctx);
        // do something...

    } catch (Exception e) {
        log.error("requestError, params: {}", this.params, e);
    } finally {
        // 判断当前任务是否是主线程在运行，当Rejected策略为CallerRunsPolicy的时候，核对当前线程
        if (mainThread != Thread.currentThread()) {
            EagleEye.clearRpcContext();
        }
    }

}

ThreadLocal thread variable overview

▐ What is ThreadLocal

The ThreadLocal class provides thread-local variables. These variables are different from ordinary variables. Each thread that accesses a thread-local variable (through its get or set method) has its own independently initialized copy of the variable, so ThreadLocal does not have the problem of multi-thread competition and does not need to be locked separately.

▐ ThreadLocal usage scenarios

Each thread needs to have its own instance data (thread isolation);

Frame cross-layer data transfer;

Scenarios of complex call chains that require global parameter passing;

The management of database connections ensures transaction consistency in various nested calls of AOP;

The principle and practice of ThreadLocal

For ThreadLocal, the commonly used methods are get/set/initialValue 3 methods.

As we all know, SimpleDateFormat has thread safety issues in java. In order to use SimpleDateFormat safely, in addition to 1) creating SimpleDateFormat local variables; and 2) adding synchronization locks, we can also use 3) ThreadLocal:

 /**
* 使用 ThreadLocal 定义一个全局的 SimpleDateFormat
*/
private static ThreadLocal<SimpleDateFormat> simpleDateFormatThreadLocal = new
ThreadLocal<SimpleDateFormat>() {
@Override
protected SimpleDateFormat initialValue() {
return new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
}
};
// 用法
String dateString = simpleDateFormatThreadLocal.get().format(calendar.getTime());

▐ ThreadLocal principle

Thread maintains a ThreadLocal.ThreadLocalMap instance (threadLocals) internally, and the operations of ThreadLocal revolve around threadLocals.

threadLocal.get() method

 /**
     * Returns the value in the current thread's copy of this
     * thread-local variable.  If the variable has no value for the
     * current thread, it is first initialized to the value returned
     * by an invocation of the {@link #initialValue} method.
     *
     * @return the current thread's value of this thread-local
     */
public T get() {
    // 1. 获取当前线程
    Thread t = Thread.currentThread();
    // 2. 获取当前线程内部的ThreadLocalMap变量t.threadLocals;
    ThreadLocalMap map = getMap(t);
    // 3. 判断map是否为null
    if (map != null) {
        // 4. 使用当前threadLocal变量获取entry
        ThreadLocalMap.Entry e = map.getEntry(this);
        // 5. 判断entry是否为null
        if (e != null) {
            // 6.返回Entry.value
            @SuppressWarnings("unchecked")
            T result = (T)e.value;
            return result;
        }
    }
    // 7. 如果map/entry为null设置初始值
    return setInitialValue();
}

 /**
     * Variant of set() to establish initialValue. Used instead
     * of set() in case user has overridden the set() method.
     *
     * @return the initial value
     */
private T setInitialValue() {
    // 1. 初始化value，如果重写就用重写后的value，默认null
    T value = initialValue();
    // 2. 获取当前线程
    Thread t = Thread.currentThread();
    // 3. 获取当前线程内部的ThreadLocalMap变量
    ThreadLocalMap map = getMap(t);
    if (map != null)
        // 4. 不为null就set, key: threadLocal, value: value
        map.set(this, value);
    else
        // 5. map若为null则创建ThreadLocalMap对象
        createMap(t, value);
    return value;
}

/**
 * Create the map associated with a ThreadLocal. Overridden in
 * InheritableThreadLocal.
 *
 * @param t the current thread
 * @param firstValue value for the initial entry of the map
 */
void createMap(Thread t, T firstValue) {
    t.threadLocals = new ThreadLocalMap(this, firstValue);
}

/**
 * Construct a new map initially containing (firstKey, firstValue).
 * ThreadLocalMaps are constructed lazily, so we only create
 * one when we have at least one entry to put in it.
 */
ThreadLocalMap(ThreadLocal<?> firstKey, Object firstValue) {
    // 1. 初始化entry数组，size: 16
    table = new Entry[INITIAL_CAPACITY];
    // 2. 计算value的index
    int i = firstKey.threadLocalHashCode & (INITIAL_CAPACITY - 1);
    // 3. 在对应index位置赋值
    table[i] = new Entry(firstKey, firstValue);
    // 4. entry size
    size = 1;
    // 5. 设置threshold: threshold = len * 2 / 3;
    setThreshold(INITIAL_CAPACITY);
}
/**
 * Set the resize threshold to maintain at worst a 2/3 load factor.
 */
private void setThreshold(int len) {
    threshold = len * 2 / 3;
}

threadLocal.set() method

 /**
     * Sets the current thread's copy of this thread-local variable
     * to the specified value.  Most subclasses will have no need to
     * override this method, relying solely on the {@link #initialValue}
     * method to set the values of thread-locals.
     *
     * @param value the value to be stored in the current thread's copy of
     *        this thread-local.
     */
    public void set(T value) {
        // 1. 获取当前线程
        Thread t = Thread.currentThread();
        // 2. 获取当前线程内部的ThreadLocalMap变量
        ThreadLocalMap map = getMap(t);
        if (map != null)
            // 3. 设置value
            map.set(this, value);
        else
            // 4. 若map为null则创建ThreadLocalMap
            createMap(t, value);
    }

ThreadLocalMap

It can be seen from the JDK source code that the Entry in ThreadLocalMap is a weak reference type, which means that if the ThreadLocal is only referenced by this Entry and not strongly referenced by other objects, it will be recycled in the next GC.

 static class ThreadLocalMap {

        /**
         * The entries in this hash map extend WeakReference, using
         * its main ref field as the key (which is always a
         * ThreadLocal object).  Note that null keys (i.e. entry.get()
         * == null) mean that the key is no longer referenced, so the
         * entry can be expunged from table.  Such entries are referred to
         * as "stale entries" in the code that follows.
         */
        static class Entry extends WeakReference<ThreadLocal<?>> {
            /** The value associated with this ThreadLocal. */
            Object value;

            Entry(ThreadLocal<?> k, Object v) {
                super(k);
                value = v;
            }
        }
    
    // ...
}

▐ ThreadLocal example

The use of eagle eye link ThreadLocal

EagleEye is widely used within the group as a full-link monitoring system. Information such as traceId, rpcId, and pressure measurement markers are stored in EagleEye's ThreadLocal variables and passed between HSF/Dubbo service calls. EagleEye initializes data into ThreadLocal through Filter, and some related codes are as follows:

 EagleEyeHttpRequest eagleEyeHttpRequest = this.convertHttpRequest(httpRequest);
// 1. 初始化，将traceId、rpcId等数据存储到鹰眼的ThreadLocal变量中
EagleEyeRequestTracer.startTrace(eagleEyeHttpRequest, false);

try {
    chain.doFilter(httpRequest, httpResponse);
} finally {
    // 2. 清理ThreadLocal变量值
    EagleEyeRequestTracer.endTrace(this.convertHttpResponse(httpResponse));
}

In EagleEyeFilter, the EagleEyeRequestTracer.startTrace method is used for initialization. After the pre-input parameters are converted, the EagleEye context parameters are stored in ThreadLocal through the startTrace overload method. The relevant code is as follows:

In the finally code block, EagleEyeFilter ends the call chain through the EagleEyeRequestTracer.endTrace method, and clears the data in ThreadLocal through the clear method. The relevant code is implemented as follows:

Bad case: XX project rights claim failure problem

In an original link for claiming rights, the request for claiming rights can only be initiated after the first-level page is opened through the app. The request reaches the server after passing through the Taoyuan wireless gateway (Mtop), and the server obtains the current session information through the mtop sdk.

In the XX project, the rights and interests claiming link has been upgraded. When the first-level page is requested, the rights and interests claiming request is simultaneously initiated through the server. Specifically, when the server processes the first-level page request, it also calls the hsf/dubbo interface to claim rights. Therefore, when initiating the rpc call, it needs to carry the user's current session information, and the service provider extracts and injects the session information into the mtop context, so that information such as session id can be obtained through mtop sdk. During the implementation of a developer, the improper use of ThreadLocal caused the following problems:

Problem 1: Due to improper initialization timing of ThreadLocal, session information cannot be obtained, resulting in failure to claim rights;

Problem 2: When the request is completed, the variable value in ThreadLocal is not cleaned up, resulting in dirty data;

[Question 1: Analysis of failure to claim rights]

In the entitlement claiming service, the application builds a set of efficient and thread-safe dependency injection framework. The business logic modules based on this framework are usually abstracted in the form of xxxModule, and the modules are meshed with dependencies. The framework will automatically call init according to the dependencies. method (where the init method of the dependent module is executed first).

In the application, the main entrance of the benefit claim interface is the CommonXXApplyModule class, and the CommonXXApplyModule depends on the XXSessionModule. When the request comes, the init method will be called in sequence according to the dependencies, so the init method of XXSessionModule will be executed first; while the developers expect to restore the mtop context by calling the recoverMtopContext() method in the init method in the CommonXXApplyModule class, because recoverMtopContext() The method is called too late, so that the XXSessionModule module cannot obtain the correct session id and other information, resulting in the failure to claim the rights.

[Question 2: Dirty data analysis]

When the entitlement claiming service processes the request, if the current thread has processed the entitlement claiming request, because the value of the ThreadLocal variable has not been cleared, the XXSessionModule obtains the session information through the mtop SDK and obtains the session information of the previous request, resulting in dirty data. .

【solution】

At the entrance of the dependency injection framework, AbstractGate#visit (or in XXSessionModule) injects mtop context information through the recoverMtopContext method, and cleans up the threadlocal variable value of the current request in the finally code block of the entry method.

▐ Thinking & Summary

Why is Entry in ThreadLocalMap designed as a weak reference type?

If a strong reference type is used, the reference chain of threadlocal is: Thread -> ThreadLocal.ThreadLocalMap -> Entry[] -> Entry -> key (threadLocal object) and value; in this scenario, as long as the thread is still running ( Such as thread pool scenarios), if the remove method is not called, the object and all associated strongly referenced objects will not be reclaimed by the garbage collector.

What is the difference between using static and not using static to modify threadlocal variables?

If the static keyword is used for modification, a thread corresponds to only one thread variable; otherwise, the threadlocal semantics becomes perThread-perInstance, which is prone to memory leaks, as shown in the following example:

 public class ThreadLocalTest {
    public static class ThreadLocalDemo {
        private ThreadLocal<String> threadLocalHolder = new ThreadLocal();

        public void setValue(String value) {
            threadLocalHolder.set(value);
        }

        public String getValue() {
            return threadLocalHolder.get();
        }
    }

    public static void main(String[] args) {
        int count = 3;
        List<ThreadLocalDemo> list = new LinkedList<>();
        for (int i = 0; i < count; i++) {
            ThreadLocalDemo demo = new ThreadLocalDemo();
            demo.setValue("demo-" + i);
            list.add(demo);
        }
        System.out.println();
    }
}

In the debug line 22 of the above main method, it can be seen that there are 3 threadlocal instances in the threadLocals variable of the thread. In engineering practice, when using threadlocal, it is usually expected that a thread has only one instance of threadlocal. Therefore, if the static modification is not used, the expected semantics have changed, and it is easy to cause memory leaks.

▐ Best Practices

ThreadLocal variable value initialization and cleanup suggestions come in pairs

If you don't perform cleanup, you might get:

Memory leak: Because the key in ThreadLocalMap is a weak reference, and the Value is a strong reference. This leads to a problem. When ThreadLocal has no strong reference to external objects, the weak reference Key will be recycled when GC occurs, but Value will not be recycled, so the elements in the Entry appear <null, value>. If the thread that created the ThreadLocal keeps running, the value in the Entry object may not be recycled, which may lead to memory leaks.

Dirty data: Due to thread reuse, when user 1 requests, business data may be saved in ThreadLocal. If it is not cleaned up, user 1's data may be read when user 2's request comes in.

It is recommended to use try...finally for cleanup.

ThreadLocal variables are recommended to be modified with static

When we use ThreadLocal, we usually expect the semantics to be perThread. If we do not use static to modify the semantics, the semantics become perThread-perInstance; in the thread pool scenario, if we do not use static to modify, the created thread-related instances may reach M * N (where M is the number of threads, and N is the number of instances of the corresponding class), which can easily cause memory leaks ( https://errorprone.info/bugpattern/ThreadLocalUsage ).

Use ThreadLocal.withInitial with caution

In the application, carefully use the factory method ThreadLocal.withInitial(Supplier<? extends S> supplier) to create ThreadLocal objects. Once the ThreadLocals of different threads use the same Supplier object, isolation will be impossible, such as:

 // 反例，实际上使用了共享对象obj而并未隔离，
private static ThreadLocal<Obj> threadLocal = ThreadLocal.withIntitial(() -> obj);

Summarize

In java engineering practice, thread pools and thread variables are widely used. Improper use of thread pools and thread variables often causes safety production accidents. Therefore, the correct use of thread pools and thread variables is a basic skill that every developer must practice. Starting from the use of thread pools and thread variables, this article briefly introduces the principles and usage practices of thread pools and thread variables. Developers can use threads and thread variables correctly in combination with best practices and actual application scenarios to build stable, Efficient java application service.

team introduction

We come from Taobao Technology - Taobao Trading Platform, and are responsible for the basic link related business of Taobao business, such as product details, shopping cart, order placement, order, logistics, refund, etc. from pre-purchase, during purchase to post-purchase performance. There are tens of billions of data, high concurrent traffic of more than one million QPS, rich business scenarios, serving 1 billion consumers, supporting the basic business, gameplay, rules and various industries of Taobao and Tmall. business expansion. There are huge challenges waiting for you here. If you are interested, you can send your resume to [e-mail] . We look forward to your joining!