Netty series: implementing thread and CPU binding in netty

Introduction

Earlier we introduced a very good library for fine-grained control of JAVA threads: java thread affinity. Using this library you can bind threads to specific CPUs or CPU cores, and improve thread execution efficiency by reducing thread switching between CPUs.

Although netty is good enough, who doesn't want to be even better? So an idea was born, that is, can the affinity library be used in netty?

The answer is yes, let's take a look.

Introduce affinity

affinity is provided in the form of a jar package. The latest official version is 3.20.0, so we need to introduce it like this:

 <!-- https://mvnrepository.com/artifact/net.openhft/affinity -->
<dependency>
    <groupId>net.openhft</groupId>
    <artifactId>affinity</artifactId>
    <version>3.20.0</version>
</dependency>

After the introduction of affinity, an affinity lib package will be added to the project's dependency library, so that we can happily use affinity in netty.

AffinityThreadFactory

With affinity, how to introduce affinity into netty?

We know that affinity is used to control threads, which means that affinity is related to threads. The thread-related in netty is EventLoopGroup. Let’s first look at the basic usage of EventLoopGroup in netty. Here is an example of NioEventLoopGroup. NioEventLoopGroup has many constructor parameters, one of which is to pass in a ThreadFactory:

 public NioEventLoopGroup(ThreadFactory threadFactory) {
        this(0, threadFactory, SelectorProvider.provider());
    }

This constructor indicates that the threads used in NioEventLoopGroup are created by threadFactory. In this way, we have found the correspondence between netty and affinity. Just construct the ThreadFactory of affinity.

It happens that there is an AffinityThreadFactory class in affinity, which is specially used to create threads corresponding to affinity.

Next, let's take a closer look at AffinityThreadFactory.

AffinityThreadFactory can create corresponding threads according to the different AffinityStrategy provided.

AffinityStrategy represents the relationship between threads. In affinity, there are 5 thread relationships, namely:

 SAME_CORE - 线程会运行在同一个CPU core中。
    SAME_SOCKET - 线程会运行在同一个CPU socket中，但是不在同一个core上。
    DIFFERENT_SOCKET - 线程会运行在不同的socket中。
    DIFFERENT_CORE - 线程会运行在不同的core上。
    ANY - 只要是可用的CPU资源都可以。

These relationships are implemented through the matches method in AffinityStrategy:

 boolean matches(int cpuId, int cpuId2);

matches passes in two parameters, which are the two cpuIds passed in. Let's take SAME_CORE as an example to see how this mathes method works:

 SAME_CORE {
        @Override
        public boolean matches(int cpuId, int cpuId2) {
            CpuLayout cpuLayout = AffinityLock.cpuLayout();
            return cpuLayout.socketId(cpuId) == cpuLayout.socketId(cpuId2) &&
                    cpuLayout.coreId(cpuId) == cpuLayout.coreId(cpuId2);
        }
    }

You can see that its logic is to first obtain the layout of the current CPU. CpuLayout contains basic information such as the number of CPUs, the number of sockets, and the number of CPU cores in each socket. And three methods are provided to return the corresponding socket, core and thread information according to the given cpuId:

 int socketId(int cpuId);

    int coreId(int cpuId);

    int threadId(int cpuId);

The matches method is to find the corresponding socket and core information according to the incoming cpuId for comparison, thereby generating 5 different strategies.

Let's take a look at the constructor of AffinityThreadFactory:

 public AffinityThreadFactory(String name, boolean daemon, @NotNull AffinityStrategy... strategies) {
        this.name = name;
        this.daemon = daemon;
        this.strategies = strategies.length == 0 ? new AffinityStrategy[]{AffinityStrategies.ANY} : strategies;
    }

You can pass in the name prefix of the thread, and whether it is a daemon thread. Finally, if strategies are not passed, the AffinityStrategies.ANY strategy is used by default, which means that any CPU that can be bound is allocated to the thread.

Next, let's see how this ThreadFactory creates new threads:

 public synchronized Thread newThread(@NotNull final Runnable r) {
        String name2 = id <= 1 ? name : (name + '-' + id);
        id++;
        Thread t = new Thread(new Runnable() {
            @Override
            public void run() {
                try (AffinityLock ignored = acquireLockBasedOnLast()) {
                    r.run();
                }
            }
        }, name2);
        t.setDaemon(daemon);
        return t;
    }

    private synchronized AffinityLock acquireLockBasedOnLast() {
        AffinityLock al = lastAffinityLock == null ? AffinityLock.acquireLock() : lastAffinityLock.acquireLock(strategies);
        if (al.cpuId() >= 0)
            lastAffinityLock = al;
        return al;
    }

As can be seen from the above code, the new thread created will be prefixed with the incoming name, followed by 1, 2, 3, and 4 suffixes. And according to whether the incoming flag is a daemon thread, the setDaemon method of the corresponding thread will be called.

The focus is on the Runnable content running inside the Thread. Inside the run method, first call the acquireLockBasedOnLast method to obtain the lock, and then run the corresponding thread method on the premise of obtaining the lock, which will bind the currently running Thread to the CPU.

From the acquireLockBasedOnLast method, we can see that AffinityLock is actually a chain structure. Each time a request is made, the acquireLock method of lastAffinityLock is called. If the lock is obtained, the lastAffinityLock will be replaced for the next lock. Obtain.

With AffinityThreadFactory, we only need to pass in AffinityThreadFactory in the use of netty.

Using AffinityThreadFactory in netty

As mentioned above, to use affinity in netty, you can pass AffinityThreadFactory into EventLoopGroup. For netty server, there can be two EventLoopGroups, namely acceptorGroup and workerGroup. In the following example, we pass AffinityThreadFactory into workerGroup, so that the threads allocated in subsequent work will follow the AffinityStrategies policy configured in AffinityThreadFactory to obtain the corresponding CPU :

 //建立两个EventloopGroup用来处理连接和消息
        EventLoopGroup acceptorGroup = new NioEventLoopGroup(acceptorThreads);
        //创建AffinityThreadFactory
        ThreadFactory threadFactory = new AffinityThreadFactory("affinityWorker", AffinityStrategies.DIFFERENT_CORE,AffinityStrategies.DIFFERENT_SOCKET,AffinityStrategies.ANY);
        //将AffinityThreadFactory加入workerGroup
        EventLoopGroup workerGroup = new NioEventLoopGroup(workerThreads,threadFactory);
        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(acceptorGroup, workerGroup)
                    .channel(NioServerSocketChannel.class)
                    .childHandler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        public void initChannel(SocketChannel ch) throws Exception {
                            ch.pipeline().addLast(new AffinityServerHandler());
                        }
                    })
                    .option(ChannelOption.SO_BACKLOG, 128)
                    .childOption(ChannelOption.SO_KEEPALIVE, true);

            // 绑定端口并开始接收连接
            ChannelFuture f = b.bind(port).sync();

            // 等待server socket关闭
            f.channel().closeFuture().sync();
        } finally {
            //关闭group
            workerGroup.shutdownGracefully();
            acceptorGroup.shutdownGracefully();
        }

In order to obtain better performance, Affinity can also isolate the CPU, and the isolated CPU is only allowed to execute the thread of this application, so as to obtain better performance.

To use this feature, you need to use linux's isolcpus. This function is mainly to separate one or more CPUs to perform specific Affinity tasks.

The isolcpus command can be followed by the ID of the CPU, or the /boot/grub/grub.conf file can be modified to add the CPU information to be isolated as follows:

 isolcpus=3，4，5

Summarize

Affinity can perform extreme control over threads, and friends who have strict performance requirements can try it, but you need to select the appropriate AffinityStrategies during use, otherwise you may not get the desired results.

Examples of this article can refer to: learn-netty4

For more information, please refer to http://www.flydean.com/51-netty-thread-affinity/

The most popular interpretation, the most profound dry goods, the most concise tutorials, and many tricks you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", understand technology, understand you better!