Netty series: using native transport protocol in netty

Introduction

For IO, in addition to traditional block IO, NIO is the most used. Usually, NIO is the most commonly used in netty programs, such as NioEventLoopGroup, NioServerSocketChannel, etc.

We also know that there are faster IO methods than NIO in IO, such as kqueue and epoll, but these two methods require the support of native methods, which means that services need to be provided at the operating system level.

If we are on a server that supports Kqueue or epoll, can netty provide support for these excellent IOs?

The answer is yes. But first kqueue and epoll need JNI support, which means that JAVA programs need to call local native methods.

native transport protocol dependencies

To use the native transmission methods of kequeue and epoll, we need to add additional project dependencies. If it is a linux environment, you can add the following maven dependency environment:

 <dependencies>
    <dependency>
      <groupId>io.netty</groupId>
      <artifactId>netty-transport-native-epoll</artifactId>
      <version>${project.version}</version>
      <classifier>linux-x86_64</classifier>
    </dependency>
    ...
  </dependencies>

The version needs to match the version number of netty you are using, otherwise an abnormal call may occur.

The classifier represents the system architecture, and its value can be linux-x86_64 or linux-aarch_64.

If you are using a mac system, you can import it like this:

 <dependencies>
    <dependency>
      <groupId>io.netty</groupId>
      <artifactId>netty-transport-native-kqueue</artifactId>
      <version>${project.version}</version>
      <classifier>osx-x86_64</classifier>
    </dependency>
    ...
  </dependencies>

In addition to a separate individual package, netty also has an all in one netty-all package. If you use this all in one package, you do not need to add additional native dependencies.

If the system architecture provided by netty is not what you are using, then you need to compile it manually. The following are the packages that the compilation depends on. If it is in a RHEL/CentOS/Fedora system, use:

 sudo yum install autoconf automake libtool make tar \
                 glibc-devel \
                 libgcc.i686 glibc-devel.i686

If on a Debian/Ubuntu system, use:

 sudo apt-get install autoconf automake libtool make tar \
                     gcc

If you are on a MacOS/BSD system, use:

 brew install autoconf automake libtool

The use of netty local transport protocol

After installing the dependencies, we can use these native transport protocols in netty.

The use of the native transmission protocol is basically the same as the use of NIO, we only need to make the following substitutions.

If it is in the liunx system, make the following substitutions:

 NioEventLoopGroup → EpollEventLoopGroup
    NioEventLoop → EpollEventLoop
    NioServerSocketChannel → EpollServerSocketChannel
    NioSocketChannel → EpollSocketChannel

If it is on a mac system, make the following substitutions:

 NioEventLoopGroup → KQueueEventLoopGroup
    NioEventLoop → KQueueEventLoop
    NioServerSocketChannel → KQueueServerSocketChannel
    NioSocketChannel → KQueueSocketChannel

Here we still use the familiar chat service as an example. First, let's see how the netty server based on Kqueue should be written:

 EventLoopGroup bossGroup = new KQueueEventLoopGroup(1);
        EventLoopGroup workerGroup = new KQueueEventLoopGroup();
        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup)
             .channel(KQueueServerSocketChannel.class)
             .handler(new LoggingHandler(LogLevel.INFO))
             .childHandler(new NativeChatServerInitializer());

            Channel channel = b.bind(PORT).sync().channel();
            log.info("server channel:{}", channel);
            channel.closeFuture().sync();

Like NIO, on the server side we need to use KQueueEventLoopGroup to create two EventLoopGroups, one is bossGroup and the other is workerGroup.

Then pass these two groups into ServerBootstrap, and add KQueueServerSocketChannel as channel.

Other content is the same as that of NIO server.

Next, let's see how the netty client based on Kqueue changes to establish a connection with the server:

 EventLoopGroup group = new KQueueEventLoopGroup();
        try {
            Bootstrap b = new Bootstrap();
            b.group(group)
             .channel(KQueueSocketChannel.class)
             .handler(new NativeChatClientInitializer());

            // 建立连接
            Channel ch = b.connect(HOST, PORT).sync().channel();
            log.info("client channel: {}", ch);

Here, KQueueEventLoopGroup is used, and KQueueEventLoopGroup is placed in Bootstrap, and KQueueSocketChannel consistent with the server side is provided for Bootstrap.

Then the client writes messages to the channel, here we enter directly from the command line:

 // 从命令行输入
            ChannelFuture lastWriteFuture = null;
            BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
            for (;;) {
                String line = in.readLine();
                if (line == null) {
                    break;
                }
                // 将从命令行输入的一行字符写到channel中
                lastWriteFuture = ch.writeAndFlush(line + "\r\n");
                // 如果输入'再见'，则等待server端关闭channel
                if ("再见".equalsIgnoreCase(line)) {
                    ch.closeFuture().sync();
                    break;
                }
            }

The above code means to write the message received by the command line to the channel, if the input is 'goodbye', then close the channel.

To be able to process strings, three codecs are used here:

 // 添加行分割器
        pipeline.addLast(new DelimiterBasedFrameDecoder(8192, Delimiters.lineDelimiter()));
        // 添加String Decoder和String Encoder,用来进行字符串的转换
        pipeline.addLast(new StringEncoder());
        pipeline.addLast(new StringDecoder());

They are line splitter, character encoder and character decoder respectively.

Run it and see, the program runs fine, and the client and server can communicate.

Summarize

Here we only use Kqueue as an example to introduce the use of the native transmission protocol in netty. For the specific code, you can refer to:

learn-netty4

For more information, please refer to http://www.flydean.com/52-netty-native-transport-md/

The most popular interpretation, the most profound dry goods, the most concise tutorials, and many tricks you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", understand technology, understand you better!