Java advanced usage: Thread-Affinity of CPU-bound threads

Introduction

In modern computer systems, there can be multiple CPUs, and each CPU can have multiple cores. In order to take full advantage of the capabilities of modern CPUs, JAVA introduces multithreading, and different threads can run on different CPUs or different CPU cores at the same time. But for JAVA programmers, how many threads are created can be controlled by themselves, but which CPU the thread runs on is a black box, which is generally difficult to know.

However, if different CPU cores schedule the same thread, performance loss caused by CPU switching may occur. Under normal circumstances, this loss is relatively small, but if your program is particularly concerned about the loss caused by this CPU switching, you can try the Java Thread Affinity that I will talk about today.

Introduction to Java Thread Affinity

Java thread Affinity is used to bind threads in JAVA code to specific CPU cores to improve program performance.

Obviously, in order to interact with the underlying CPU, java thread Affinity must use the method of interaction between JAVA and native methods. Although JNI is the official method of JAVA to interact with native methods, JNI is more cumbersome to use. . So java thread Affinity actually uses JNA, which is an improved library based on JNI that interacts with native methods.

Let's first introduce several concepts in CPU, namely CPU, CPU socket and CPU core.

The first is the CPU. The full name of the CPU is the central processing unit, also known as the central processing unit, which is the key core used for task processing.

So what is a CPU socket? The so-called socket is the socket where the CPU is inserted. If you have assembled a desktop computer, you should know that the CPU is installed on the socket.

CPU Core refers to the number of cores in the CPU. A long time ago, CPUs were single-core, but with the development of multi-core technology, a CPU can contain multiple cores, and the cores in the CPU are the real business processing. unit.

If you are on a linux machine, you can view the CPU status of the system by using the lscpu command, as follows:

 Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                1
On-line CPU(s) list:   0
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 94
Model name:            Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
Stepping:              3
CPU MHz:               2400.000
BogoMIPS:              4800.00
Hypervisor vendor:     KVM
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              4096K
L3 cache:              28160K
NUMA node0 CPU(s):     0
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 arat

From the above output we can see that this server has a socket, each socket has a core, and each core can process 1 thread at the same time.

The information of these CPUs can be called CPU layout. In Linux, the layout information of the CPU is stored in /proc/cpuinfo.

There is a CpuLayout interface in Java Thread Affinity to correspond to this information:

 public interface CpuLayout {
    
    int cpus();

    int sockets();

    int coresPerSocket();

    int threadsPerCore();

    int socketId(int cpuId);

    int coreId(int cpuId);

    int threadId(int cpuId);
}

According to the information of CPU layout, AffinityStrategies provides some basic Affinity strategies to arrange the distribution relationship between different threads, mainly including the following:

 SAME_CORE - 运行在同一个core中。
    SAME_SOCKET - 运行在同一个socket中，但是不在同一个core上。
    DIFFERENT_SOCKET - 运行在不同的socket中
    DIFFERENT_CORE - 运行在不同的core上
    ANY - 任何情况都可以

These strategies are also distinguished according to the socketId and coreId of CpuLayout. We take SAME_CORE as an example and press its specific implementation:

 SAME_CORE {
        @Override
        public boolean matches(int cpuId, int cpuId2) {
            CpuLayout cpuLayout = AffinityLock.cpuLayout();
            return cpuLayout.socketId(cpuId) == cpuLayout.socketId(cpuId2) &&
                    cpuLayout.coreId(cpuId) == cpuLayout.coreId(cpuId2);
        }
    }

The Affinity strategy can be ordered, the strategy in front will be matched first, if it does not match, the second strategy will be selected, and so on.

Use of AffinityLock

Next, let's look at the specific use of Affinity. The first is to obtain a CPU lock. Before JAVA7, we can write:

 AffinityLock al = AffinityLock.acquireLock();
try {
     // do some work locked to a CPU.
} finally {
     al.release();
}

After JAVA7, it can be written like this:

 try (AffinityLock al = AffinityLock.acquireLock()) {
    // do some work while locked to a CPU.
}

The acquireLock method can acquire any available cpu for the thread. This is a coarse-grained lock. If you want to get fine-grained core, you can use acquireCore:

 try (AffinityLock al = AffinityLock.acquireCore()) {
    // do some work while locked to a CPU.
}

acquireLock also has a bind parameter, indicating whether to bind the current thread to the acquired cpu lock. If the bind parameter = true, the current thread will run on the CPU obtained in acquireLock. If the bind parameter = false, it means that acquireLock will be bound at some point in the future.

We mentioned AffinityStrategy above, this AffinityStrategy can be used as a parameter of acquireLock:

 public AffinityLock acquireLock(AffinityStrategy... strategies) {
        return acquireLock(false, cpuId, strategies);
    }

By calling the acquireLock method of the current AffinityLock, the current thread can be allocated an AffinityLock related to the previous lock policy.

AffinityLock also provides a dumpLocks method to view the current binding state of CPU and thread. Let's take an example:

 private static final ExecutorService ES = Executors.newFixedThreadPool(4,
           new AffinityThreadFactory("bg", SAME_CORE, DIFFERENT_SOCKET, ANY));

for (int i = 0; i < 12; i++)
            ES.submit(new Callable<Void>() {
                @Override
                public Void call() throws InterruptedException {
                    Thread.sleep(100);
                    return null;
                }
            });
        Thread.sleep(200);
        System.out.println("\nThe assignment of CPUs is\n" + AffinityLock.dumpLocks());
        ES.shutdown();
        ES.awaitTermination(1, TimeUnit.SECONDS);

In the above code, we created a thread pool with 4 threads, the corresponding ThreadFactory is AffinityThreadFactory, named the thread pool bg, and allocated 3 AffinityStrategy. This means first assigning to the same core, then to a different socket, and finally to whatever CPU is available.

Then during the specific execution process, we submitted 12 threads, but our Thread pool has only 4 threads at most. It is foreseeable that only 4 threads will be bound to the CPU in the results returned by the AffinityLock.dumpLocks method. Let's take a look:

 The assignment of CPUs is
0: CPU not available
1: Reserved for this application
2: Reserved for this application
3: Reserved for this application
4: Thread[bg-4,5,main] alive=true
5: Thread[bg-3,5,main] alive=true
6: Thread[bg-2,5,main] alive=true
7: Thread[bg,5,main] alive=true

As you can see from the output, CPU0 is unavailable. The other 7 CPUs are available, but only 4 threads are bound, which matches our previous analysis.

Next, we modify the AffinityStrategy of AffinityThreadFactory as follows:

 new AffinityThreadFactory("bg", SAME_CORE)

Indicates that threads will only be bound to the same core, because in the current hardware, a core can only support the binding of one thread at the same time, so it is foreseeable that the final result will only be bound to one thread, and the running results are as follows:

 The assignment of CPUs is
0: CPU not available
1: Reserved for this application
2: Reserved for this application
3: Reserved for this application
4: Reserved for this application
5: Reserved for this application
6: Reserved for this application
7: Thread[bg,5,main] alive=true

It can be seen that only the first thread is CPU bound, which matches the previous analysis.

Allocate CPU directly using API

The acquireLock method of AffinityLock we mentioned above can actually accept a CPU id parameter, which is directly used to obtain the lock of the incoming CPU id. This allows subsequent threads to run on the specified CPU.

 public static AffinityLock acquireLock(int cpuId) {
        return acquireLock(true, cpuId, AffinityStrategies.ANY);
    }

In real time, this kind of Affinity is stored in BitSet. The index of BitSet is the id of the CPU, and the corresponding value is whether the lock is obtained.

First look at the definition of the setAffinity method:

 public static void setAffinity(int cpu) {
        BitSet affinity = new BitSet(Runtime.getRuntime().availableProcessors());
        affinity.set(cpu);
        setAffinity(affinity);
    }

Look at the use of setAffinity:

 long currentAffinity = AffinitySupport.getAffinity();
Affinity.setAffinity(1L << 5); // lock to CPU 5.

Note that because the bottom layer of BitSet uses Long for data storage, the index here is the bit index, so we need to convert the decimal CPU index.

Summarize

Java Thread Affinity can control the CPU used by Thread in the program from the JAVA code, which is very powerful and can be used by everyone.

This article has been included in http://www.flydean.com/01-java-thread-affinity/

The most popular interpretation, the most profound dry goods, the most concise tutorials, and many tricks you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", understand technology, understand you better!