Netty series: using protobuf protocol in netty

Introduction

There are many coding tools that adapt to different protocols in netty, and the popular protobuf produced by Google is no exception. Netty provides ProtobufDecoder and ProtobufEncoder with two tools and corresponding frame detection. Next, we will use an example to explain in detail how to use protobuf in netty.

Define protobuf

Let's take the simplest example. First, define a message that needs to be transmitted on the network. Here we define a student object, which has an age and a name attribute, as shown below:

syntax = "proto3";

package com.flydean17.protobuf;

option java_multiple_files = true;
option java_package = "com.flydean17.protobuf";
option java_outer_classname = "StudentWrapper";

message Student {
  optional int32 age = 1;
  optional string name =2;
}

Use the following command to compile it:

 protoc --experimental_allow_proto3_optional  -I=. --java_out=. student.proto

You can see that 3 files are generated, namely Student, StudentOrBuilder and StudentWrapper. Among them, Student and StudentOrBuilder are the objects we really need to use.

Define handler

In the handler, we mainly process the message. Here we build and send the message in the clientHandler. The StudentClientHandler inherits SimpleChannelInboundHandler and re-channelActive method. In this method, we use the protobuf syntax to construct a new Student instance, and Set up two attributes, age and name, for him.

Then use the ctx.write and ctx.flush methods to send it to the server:

    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        // channel活跃
        //构建一个Student，并将其写入到channel中
        Student student= Student.newBuilder().setAge(22).setName("flydean").build();
        log.info("client发送消息{}",student);
        ctx.write(student);
        ctx.flush();
    }

StudentServerHandler also inherits SimpleChannelInboundHandler and rewrites the channelRead0 method. When the server side reads the student message, log output and write it back to the channel for clientHandler to read:

    public void channelRead0(ChannelHandlerContext ctx, Student student) throws Exception {
        log.info("server收到消息{}",student);
        // 写入消息
        ChannelFuture future = ctx.write(student);
    }

When the client reads the message, it directly logs the output without further processing. At this point, a round of interaction between the client and the server is completed:

    public void channelRead0(ChannelHandlerContext ctx, Student student) throws Exception {
        log.info("client收到消息{}",student);
    }

Set up ChannelPipeline

In the previous section, whether in StudentClientHandler or StudentServerHandler, we assumed that the object passed in the channel is Student, not the original ByteBuf. How is this done?

Here we need to use the frame detection provided by netty. Netty provides ProtobufDecoder and ProtobufEncoder for the protobuf protocol, which are used to encode and decode protobuf objects.

But these two encoders and decoders are MessageToMessageEncoder and MessageToMessageDecoder, respectively. They are message-to-message encoders and decoders, so they need to be used in conjunction with frame detection.

Netty also provides frame detectors that work with protobuf, they are ProtobufVarint32FrameDecoder and ProtobufVarint32LengthFieldPrepender.

Varint32 refers to the encoding format of protobuf, and the first byte uses variable varint.

With the frame detector and codec, we only need to add them to the ChannelPipeline in order.

On the client side, StudentClientInitializer inherits from ChannelInitializer, we need to rewrite its initChannel method:

    public void initChannel(SocketChannel ch) {
        ChannelPipeline p = ch.pipeline();

        p.addLast(new ProtobufVarint32FrameDecoder());
        p.addLast(new ProtobufDecoder(Student.getDefaultInstance()));

        p.addLast(new ProtobufVarint32LengthFieldPrepender());
        p.addLast(new ProtobufEncoder());

        p.addLast(new StudentClientHandler());
    }

On the server side, the same StudentServerInitializer also inherits from ChannelInitializer, it also needs to rewrite its initChannel method:

    public void initChannel(SocketChannel ch) throws Exception {
        ChannelPipeline p = ch.pipeline();

        p.addLast(new ProtobufVarint32FrameDecoder());
        p.addLast(new ProtobufDecoder(Student.getDefaultInstance()));

        p.addLast(new ProtobufVarint32LengthFieldPrepender());
        p.addLast(new ProtobufEncoder());

        p.addLast(new StudentServerHandler());
    }

In this way, the ChannelPipeline is also set up.

Build and run the client and server

The last thing to do is to build the client and server and run them, which is no different from the normal netty client and server:

Build StudentClient:

   public static void main(String[] args) throws Exception {

        EventLoopGroup group = new NioEventLoopGroup();
        try {
            Bootstrap b = new Bootstrap();
            b.group(group)
             .channel(NioSocketChannel.class)
             .handler(new StudentClientInitializer());
            // 建立连接
            Channel ch = b.connect(HOST, PORT).sync().channel();
            // 等待关闭
            ch.closeFuture().sync();
        } finally {
            group.shutdownGracefully();
        }
    }

Build StudentServer:

   public static void main(String[] args) throws Exception {
        EventLoopGroup bossGroup = new NioEventLoopGroup(1);
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup)
             .channel(NioServerSocketChannel.class)
             .handler(new LoggingHandler(LogLevel.INFO))
             .childHandler(new StudentServerInitializer());

            b.bind(PORT).sync().channel().closeFuture().sync();
        } finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }

Run to get:

server端：
[nioEventLoopGroup-3-1] INFO  c.f.protobuf.StudentServerHandler - server收到消息age: 22
name: "flydean"

[nioEventLoopGroup-2-1] INFO  c.f.protobuf.StudentClientHandler - client发送消息age: 22
name: "flydean"

client端：
[nioEventLoopGroup-2-1] INFO  c.f.protobuf.StudentClientHandler - client收到消息age: 22
name: "flydean"

It can be seen that the Student message has been sent and received successfully.

Summarize

Netty provides a lot of tool classes adapted to the protocol, so that we can focus on business logic without considering specific encoding conversion issues, which is very easy to use.

For the examples in this article, please refer to: learn-netty4

This article has been included in http://www.flydean.com/17-netty-protobuf/

The most popular interpretation, the most profound dry goods, the most concise tutorial, and many tips you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", know technology, know you better!