Netty series: NIO and netty in detail

Introduction

Why is netty fast? This is because the bottom layer of netty uses JAVA's NIO technology and optimizes its performance based on it. Although netty is not pure JAVA nio, the bottom layer of netty is still based on nio technology.

nio was introduced in JDK1.4 to distinguish it from traditional IO, so nio can also be called new io.

The three cores of nio are Selector, channel and Buffer. In this article, we will deeply explore the relationship between NIO and netty.

Common usage of NIO

Before explaining the NIO implementation in netty, let's review how the NIO selector and channel work in the JDK. For NIO, the selector is mainly used to accept connections from clients, so it is generally used on the server side. We take a NIO server-side and client-side chat room as an example to explain how NIO is used in JDK.

Because it is a simple chat room, we choose the ServerSocketChannel based on the Socket protocol, and the first is to open the Server channel:

ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.bind(new InetSocketAddress("localhost", 9527));
serverSocketChannel.configureBlocking(false);

Then register the selector with the server channel:

Selector selector = Selector.open();
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

Although it is NIO, for Selector, its select method is a blocking method, and it will return only after finding a matching channel. In order to perform the select operation multiple times, we need to perform the select operation of the selector in a while loop:

while (true) {
            selector.select();
            Set<SelectionKey> selectedKeys = selector.selectedKeys();
            Iterator<SelectionKey> iter = selectedKeys.iterator();
            while (iter.hasNext()) {
                SelectionKey selectionKey = iter.next();
                if (selectionKey.isAcceptable()) {
                    register(selector, serverSocketChannel);
                }
                if (selectionKey.isReadable()) {
                    serverResponse(byteBuffer, selectionKey);
                }
                iter.remove();
            }
            Thread.sleep(1000);
        }

There will be some SelectionKeys in the selector, and there are some OP Statuses in the SelectionKey that represent the operation status. According to the different OP Status, the selectionKey can have four statuses, namely isReadable, isWritable, isConnectable and isAcceptable.

When the SelectionKey is in the isAcceptable state, it means that the ServerSocketChannel can accept the connection. We need to call the register method to register the socketChannel generated by the serverSocketChannel accept in the selector to monitor its OP READ state, from which data can be read later:

    private static void register(Selector selector, ServerSocketChannel serverSocketChannel)
            throws IOException {
        SocketChannel socketChannel = serverSocketChannel.accept();
        socketChannel.configureBlocking(false);
        socketChannel.register(selector, SelectionKey.OP_READ);
    }

When selectionKey is in isReadable state, it means that data can be read from socketChannel and processed:

    private static void serverResponse(ByteBuffer byteBuffer, SelectionKey selectionKey)
            throws IOException {
        SocketChannel socketChannel = (SocketChannel) selectionKey.channel();
        socketChannel.read(byteBuffer);
        byteBuffer.flip();
        byte[] bytes= new byte[byteBuffer.limit()];
        byteBuffer.get(bytes);
        log.info(new String(bytes).trim());
        if(new String(bytes).trim().equals(BYE_BYE)){
            log.info("说再见不如不见！");
            socketChannel.write(ByteBuffer.wrap("再见".getBytes()));
            socketChannel.close();
        }else {
            socketChannel.write(ByteBuffer.wrap("你是个好人".getBytes()));
        }
        byteBuffer.clear();
    }

In the above serverResponse method, the corresponding SocketChannel is obtained from the selectionKey, and then the read method of the SocketChannel is called to read the data in the channel into the byteBuffer. If you want to reply the message to the channel, you still use the same socketChannel, and then call write The method writes back the message to the client side, and here a simple server side that writes back the client message is completed.

Next is the corresponding NIO client. The NIO client needs to use SocketChannel. First, establish a connection with the server:

socketChannel = SocketChannel.open(new InetSocketAddress("localhost", 9527));

Then you can use this channel to send and receive messages:

    public String sendMessage(String msg) throws IOException {
        byteBuffer = ByteBuffer.wrap(msg.getBytes());
        String response = null;
        socketChannel.write(byteBuffer);
        byteBuffer.clear();
        socketChannel.read(byteBuffer);
        byteBuffer.flip();
        byte[] bytes= new byte[byteBuffer.limit()];
        byteBuffer.get(bytes);
        response =new String(bytes).trim();
        byteBuffer.clear();
        return response;
    }

The write method can be used to write messages to the channel, and the read method can be used to read messages from the channel.

Such a NIO client is completed.

Although the above is the basic use of NIO's server and client, it basically covers all the points of NIO. Next, let's take a closer look at how NIO is used in netty.

NIO and EventLoopGroup

Taking Netty's ServerBootstrap as an example, you need to specify its group at startup. Let's take a look at the group method of ServerBootstrap:

public ServerBootstrap group(EventLoopGroup group) {
        return group(group, group);
    }

public ServerBootstrap group(EventLoopGroup parentGroup, EventLoopGroup childGroup) {
    ...
}

ServerBootstrap can accept one EventLoopGroup or two EventLoopGroups. EventLoopGroup is used to handle all events and IO. For ServerBootstrap, there can be two EventLoopGroups, and for Bootstrap, there is only one EventLoopGroup. The two EventLoopGroups represent the acceptor group and the worker group.

EventLoopGroup is just an interface. One of our commonly used implementations is NioEventLoopGroup. The following is a commonly used netty server-side code:

        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup)
                    .channel(NioServerSocketChannel.class)
                    .childHandler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        public void initChannel(SocketChannel ch) throws Exception {
                            ch.pipeline().addLast(new FirstServerHandler());
                        }
                    })
                    .option(ChannelOption.SO_BACKLOG, 128)
                    .childOption(ChannelOption.SO_KEEPALIVE, true);

            // 绑定端口并开始接收连接
            ChannelFuture f = b.bind(port).sync();
            // 等待server socket关闭
            f.channel().closeFuture().sync();

There are two classes related to NIO here, namely NioEventLoopGroup and NioServerSocketChannel. In fact, there are two similar classes at the bottom of them called NioEventLoop and NioSocketChannel. Next, we will explain some of their underlying implementation and logical relationship.

NioEventLoopGroup

NioEventLoopGroup and DefaultEventLoopGroup inherit from MultithreadEventLoopGroup:

public class NioEventLoopGroup extends MultithreadEventLoopGroup 

The difference between them lies in the newChild method. newChild is used to construct the actual object in the Group. For NioEventLoopGroup, newChild returns a NioEventLoop object. Let's first look at the newChild method of NioEventLoopGroup:

    protected EventLoop newChild(Executor executor, Object... args) throws Exception {
        SelectorProvider selectorProvider = (SelectorProvider) args[0];
        SelectStrategyFactory selectStrategyFactory = (SelectStrategyFactory) args[1];
        RejectedExecutionHandler rejectedExecutionHandler = (RejectedExecutionHandler) args[2];
        EventLoopTaskQueueFactory taskQueueFactory = null;
        EventLoopTaskQueueFactory tailTaskQueueFactory = null;

        int argsLength = args.length;
        if (argsLength > 3) {
            taskQueueFactory = (EventLoopTaskQueueFactory) args[3];
        }
        if (argsLength > 4) {
            tailTaskQueueFactory = (EventLoopTaskQueueFactory) args[4];
        }
        return new NioEventLoop(this, executor, selectorProvider,
                selectStrategyFactory.newSelectStrategy(),
                rejectedExecutionHandler, taskQueueFactory, tailTaskQueueFactory);
    }

In addition to the fixed executor parameters, this newChild method can also implement more functions according to the parameters passed in by the constructor of NioEventLoopGroup.

The parameters here are SelectorProvider, SelectStrategyFactory, RejectedExecutionHandler, taskQueueFactory and tailTaskQueueFactory, and the latter two EventLoopTaskQueueFactory are not necessary.

Finally, all parameters will be passed to the constructor of NioEventLoop to construct a new NioEventLoop.

Before explaining NioEventLoop in detail, let's study the actual function of the passed parameter types.

SelectorProvider

SelectorProvider is a class in JDK that provides a static provider() method to load the corresponding SelectorProvider class from Property or ServiceLoader and instantiate it.

In addition, practical NIO operation methods such as openDatagramChannel, openPipe, openSelector, openServerSocketChannel and openSocketChannel are provided.

SelectStrategyFactory

SelectStrategyFactory is an interface that defines only one method to return SelectStrategy:

public interface SelectStrategyFactory {

    SelectStrategy newSelectStrategy();
}

What is SelectStrategy?

Let's first look at which strategies are defined in SelectStrategy:

    int SELECT = -1;

    int CONTINUE = -2;

    int BUSY_WAIT = -3;

Three strategies are defined in SelectStrategy, namely SELECT, CONTINUE and BUSY_WAIT.

We know that in general, the select operation itself in NIO is a blocking operation, that is, a block operation. The strategy corresponding to this operation is SELECT, which is the select block state.

If we want to skip this block and re-enter the next event loop, then the corresponding strategy is CONTINUE.

BUSY_WAIT is a special strategy, which means that the IO loop polls for new events without blocking. This strategy is only supported in epoll mode. NIO and Kqueue modes do not support this strategy.

RejectedExecutionHandler

RejectedExecutionHandler is netty's own class, similar to java.util.concurrent.RejectedExecutionHandler, but specifically for SingleThreadEventExecutor. This interface defines a rejected method, which is used to indicate that the task addition fails due to the capacity limit of SingleThreadEventExecutor and is rejected:

void rejected(Runnable task, SingleThreadEventExecutor executor);

EventLoopTaskQueueFactory

EventLoopTaskQueueFactory is an interface used to create a taskQueue that stores submissions to EventLoop:

Queue<Runnable> newTaskQueue(int maxCapacity);

This Queue must be thread-safe and inherit from java.util.concurrent.BlockingQueue.

After explaining these parameters, we can view the specific NIO implementation of NioEventLoop in detail.

NioEventLoop

First of all, NioEventLoop, like DefaultEventLoop, inherits from SingleThreadEventLoop:

public final class NioEventLoop extends SingleThreadEventLoop

Represents an EventLoop that uses a single thread to execute tasks.

First of all, as an implementation of NIO, there must be a selector. Two selectors are defined in NioEventLoop, namely selector and unwrappedSelector:

    private Selector selector;
    private Selector unwrappedSelector;

In the constructor of NioEventLoop, they are defined like this:

        final SelectorTuple selectorTuple = openSelector();
        this.selector = selectorTuple.selector;
        this.unwrappedSelector = selectorTuple.unwrappedSelector;

First call the openSelector method, and then obtain the corresponding selector and unwrappedSelector through the returned SelectorTuple.

What is the difference between these two selectors?

In the openSelector method, first return a Selector by calling the provider's openSelector method, which is the unwrappedSelector:

final Selector unwrappedSelector;
unwrappedSelector = provider.openSelector();

Then check if DISABLE_KEY_SET_OPTIMIZATION is set, if not then unwrappedSelector and selector are actually the same Selector:

DISABLE_KEY_SET_OPTIMIZATION indicates whether to optimize the select key set:

if (DISABLE_KEY_SET_OPTIMIZATION) {
      return new SelectorTuple(unwrappedSelector);
   }

        SelectorTuple(Selector unwrappedSelector) {
            this.unwrappedSelector = unwrappedSelector;
            this.selector = unwrappedSelector;
        }

If DISABLE_KEY_SET_OPTIMIZATION is set to false, it means that we need to optimize the select key set. How to optimize it?

Let's take a look at the last return:

return new SelectorTuple(unwrappedSelector,
                                 new SelectedSelectionKeySetSelector(unwrappedSelector, selectedKeySet));

The second parameter of the SelectorTuple finally returned is the selector, where the selector is a SelectedSelectionKeySetSelector object.

SelectedSelectionKeySetSelector inherits from selector, the first parameter passed in by the constructor is a delegate, and all the methods defined in Selector are called by calling
The difference is that for the select method, the reset method of the selectedKeySet will be called first. The following is an example of the isOpen and select methods to observe the implementation of the code:

    public boolean isOpen() {
        return delegate.isOpen();
    }

    public int select(long timeout) throws IOException {
        selectionKeys.reset();
        return delegate.select(timeout);
    }

selectedKeySet is a SelectedSelectionKeySet object, which is a set collection used to store SelectionKey. In the openSelector() method, use new to instantiate this object:

final SelectedSelectionKeySet selectedKeySet = new SelectedSelectionKeySet();

Netty actually wants to use this SelectedSelectionKeySet class to manage the selectedKeys in the Selector, so then netty uses a highly skilled object replacement operation.

First determine whether there is an implementation of sun.nio.ch.SelectorImpl in the system:

        Object maybeSelectorImplClass = AccessController.doPrivileged(new PrivilegedAction<Object>() {
            @Override
            public Object run() {
                try {
                    return Class.forName(
                            "sun.nio.ch.SelectorImpl",
                            false,
                            PlatformDependent.getSystemClassLoader());
                } catch (Throwable cause) {
                    return cause;
                }
            }
        });

There are two Set fields in SelectorImpl:

    private Set<SelectionKey> publicKeys;
    private Set<SelectionKey> publicSelectedKeys;

These two fields are the objects we need to replace. If there is a SelectorImpl, first use the Unsafe class, call the objectFieldOffset method in PlatformDependent to get the offsets of these two fields relative to the object instance, and then call putObject to replace these two fields with the previously initialized selectedKeySet object:

Field selectedKeysField = selectorImplClass.getDeclaredField("selectedKeys");
Field publicSelectedKeysField = selectorImplClass.getDeclaredField("publicSelectedKeys");

if (PlatformDependent.javaVersion() >= 9 && PlatformDependent.hasUnsafe()) {
    // Let us try to use sun.misc.Unsafe to replace the SelectionKeySet.
    // This allows us to also do this in Java9+ without any extra flags.
    long selectedKeysFieldOffset = PlatformDependent.objectFieldOffset(selectedKeysField);
    long publicSelectedKeysFieldOffset =
            PlatformDependent.objectFieldOffset(publicSelectedKeysField);

    if (selectedKeysFieldOffset != -1 && publicSelectedKeysFieldOffset != -1) {
        PlatformDependent.putObject(
                unwrappedSelector, selectedKeysFieldOffset, selectedKeySet);
        PlatformDependent.putObject(
                unwrappedSelector, publicSelectedKeysFieldOffset, selectedKeySet);
        return null;
    }

If your system settings don't support Unsafe, then do it again with reflection:

 Throwable cause = ReflectionUtil.trySetAccessible(selectedKeysField, true);
 if (cause != null) {
     return cause;
 }
 cause = ReflectionUtil.trySetAccessible(publicSelectedKeysField, true);
 if (cause != null) {
     return cause;
 }
 selectedKeysField.set(unwrappedSelector, selectedKeySet);
 publicSelectedKeysField.set(unwrappedSelector, selectedKeySet);

A very important rewrite method that we need to pay attention to in NioEventLoop is the run method, in which the logic of how to execute the task is implemented.

Remember the selectStrategy we mentioned earlier? The run method returns the strategy of the select by calling selectStrategy.calculateStrategy, and then judges by
The value of strategy will be processed accordingly.

If strategy is CONTINUE, this skips this loop and goes to the next loop.

BUSY_WAIT is not supported in NIO. If it is in the SELECT state, the select operation will be performed again after curDeadlineNanos:

strategy = selectStrategy.calculateStrategy(selectNowSupplier, hasTasks());
  switch (strategy) {
  case SelectStrategy.CONTINUE:
      continue;
  case SelectStrategy.BUSY_WAIT:
      // fall-through to SELECT since the busy-wait is not supported with NIO
  case SelectStrategy.SELECT:
      long curDeadlineNanos = nextScheduledTaskDeadlineNanos();
      if (curDeadlineNanos == -1L) {
          curDeadlineNanos = NONE; // nothing on the calendar
      }
      nextWakeupNanos.set(curDeadlineNanos);
      try {
          if (!hasTasks()) {
              strategy = select(curDeadlineNanos);
          }
      } finally {
          // This update is just to help block unnecessary selector wakeups
          // so use of lazySet is ok (no race condition)
          nextWakeupNanos.lazySet(AWAKE);
      }
      // fall through
  default:

If strategy > 0, it means that SelectedKeys have been obtained, then you need to call the processSelectedKeys method to process SelectedKeys:

    private void processSelectedKeys() {
        if (selectedKeys != null) {
            processSelectedKeysOptimized();
        } else {
            processSelectedKeysPlain(selector.selectedKeys());
        }
    }

As mentioned above, there are two selectors in NioEventLoop and a selectedKeys property. This selectedKeys stores the Optimized SelectedKeys. If the value is not empty, the processSelectedKeysOptimized method is called, otherwise, the processSelectedKeysPlain method is called.

The two methods processSelectedKeysOptimized and processSelectedKeysPlain are not very different, but the selectedKeys to be processed are different.

The processing logic is to first get the key of selectedKeys, and then call its attachment method to get the attached object:

final SelectionKey k = selectedKeys.keys[i];
            selectedKeys.keys[i] = null;

            final Object a = k.attachment();

            if (a instanceof AbstractNioChannel) {
                processSelectedKey(k, (AbstractNioChannel) a);
            } else {
                NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;
                processSelectedKey(k, task);
            }

If the channel has not yet established a connection, then this object may be a NioTask, used to handle channelReady and channelUnregistered events.

If the channel is already connected, then this object may be an AbstractNioChannel.

For two different objects, different processSelectedKey methods will be called respectively.

For the first case, the task's channelReady method is called:

task.channelReady(k.channel(), k);

For the second case, various methods in ch.unsafe() will be called according to the various states of readyOps() of SelectionKey to perform operations such as read or close.

Summarize

Although NioEventLoop is also a SingleThreadEventLoop, by using NIO technology, existing resources can be better utilized to achieve better efficiency, which is why we use NioEventLoopGroup instead of DefaultEventLoopGroup in the project.

This article has been included in http://www.flydean.com/05-2-netty-nioeventloop/

The most popular interpretation, the most profound dry goods, the most concise tutorials, and many tricks you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", understand technology, understand you better!