Realize custom message communication protocol based on Netty (protocol design and analysis application actual combat)

The so-called protocol is a specification composed of three elements: grammar, semantics, and timing. Both parties in communication implement network data transmission according to the protocol specification, so that both parties in communication can achieve normal data communication and analysis.

Because different middlewares have certain differences in function, there should be no standardized protocol to meet different differentiated requirements. Therefore, many middlewares will define their own communication protocols. In addition, communication protocols can solve the problems of sticking and unpacking. .

In this article, we will implement a custom message protocol.

Elements of a custom agreement

Custom agreement, then this agreement must have constituent elements,

• Magic number: used to judge the validity of the data packet
• Version number: can support protocol upgrade
• Serialization algorithm: What serialization and deserialization method is used for the message body, such as json, protobuf, hessian, etc.
• Command type: what type of message is currently being sent, like in zookeeper, it passes a Type
• Request sequence number: Based on the duplex protocol, it provides asynchronous capability, that is, the received asynchronous message needs to find the previous communication request for response processing
• Message length
• Message body

Agreement definition

sessionId | reqType | Content-Length | Content |

Among them, Version , Content-Length , SessionId are Header information, and Content is the main body of interaction.

Define the project structure and import packages

<dependency>
    <groupId>io.netty</groupId>
    <artifactId>netty-all</artifactId>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-log4j12</artifactId>
</dependency>
<dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
</dependency>

The project structure is shown in Figure 4-1:

• netty-message-mic: Indicates the protocol module.
• netty-message-server: stands for nettyserver.

<center>Figure 4-1</center>

• Import log4j.properties

In nettyMessage-mic, the structure of the packet is as follows.

Define Header

Represents the message header

@Data
public class Header{
    private long sessionId; //会话id  : 占8个字节
    private byte type; //消息类型： 占1个字节

    private int length;     //消息长度 : 占4个字节
}

Define MessageRecord

Represents the message body

@Data
public class MessageRecord{

    private Header header;
    private Object body;
}

OpCode

Define operation type

public enum OpCode {

    BUSI_REQ((byte)0),
    BUSI_RESP((byte)1),
    PING((byte)3),
    PONG((byte)4);

    private byte code;

    private OpCode(byte code) {
        this.code=code;
    }

    public byte code(){
        return this.code;
    }
}

Define codec

Define the codec of the message protocol respectively

MessageRecordEncoder

@Slf4j
public class MessageRecordEncoder extends MessageToByteEncoder<MessageRecord> {

    @Override
    protected void encode(ChannelHandlerContext channelHandlerContext, MessageRecord record, ByteBuf byteBuf) throws Exception {
        log.info("===========开始编码Header部分===========");
        Header header=record.getHeader();
        byteBuf.writeLong(header.getSessionId()); //保存8个字节的sessionId
        byteBuf.writeByte(header.getType());  //写入1个字节的请求类型

        log.info("===========开始编码Body部分===========");
        Object body=record.getBody();
        if(body!=null){
            ByteArrayOutputStream bos=new ByteArrayOutputStream();
            ObjectOutputStream oos=new ObjectOutputStream(bos);
            oos.writeObject(body);
            byte[] bytes=bos.toByteArray();
            byteBuf.writeInt(bytes.length); //写入消息体长度:占4个字节
            byteBuf.writeBytes(bytes); //写入消息体内容
        }else{
            byteBuf.writeInt(0); //写入消息长度占4个字节，长度为0
        }
    }
}

MessageRecordDecode

@Slf4j
public class MessageRecordDecode extends ByteToMessageDecoder {

    @Override
    protected void decode(ChannelHandlerContext channelHandlerContext, ByteBuf byteBuf, List<Object> list) throws Exception {
        MessageRecord record=new MessageRecord();
        Header header=new Header();
        header.setSessionId(byteBuf.readLong());  //读取8个字节的sessionid
        header.setType(byteBuf.readByte()); //读取一个字节的操作类型
        record.setHeader(header);
        //如果byteBuf剩下的长度还有大于4个字节，说明body不为空
        if(byteBuf.readableBytes()>4){
            int length=byteBuf.readInt(); //读取四个字节的长度
            header.setLength(length);
            byte[] contents=new byte[length];
            byteBuf.readBytes(contents,0,length);
            ByteArrayInputStream bis=new ByteArrayInputStream(contents);
            ObjectInputStream ois=new ObjectInputStream(bis);
            record.setBody(ois.readObject());
            list.add(record);
            log.info("序列化出来的结果："+record);
        }else{
            log.error("消息内容为空");
        }
    }
}

Analysis and coding of test protocol

EmbeddedChannel is provided by netty to improve the unit test for ChannelHandler

public class CodesMainTest {
    public static void main( String[] args ) throws Exception {
        EmbeddedChannel channel=new EmbeddedChannel(
            new LoggingHandler(),
            new MessageRecordEncoder(),
            new MessageRecordDecode());
        Header header=new Header();
        header.setSessionId(123456);
        header.setType(OpCode.PING.code());
        MessageRecord record=new MessageRecord();
        record.setBody("Hello World");
        record.setHeader(header);
        channel.writeOutbound(record);

        ByteBuf buf= ByteBufAllocator.DEFAULT.buffer();
        new MessageRecordEncoder().encode(null,record,buf);
        channel.writeInbound(buf);
    }
}

Coding packet analysis

After running the above code, you will get the following message

         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 00 00 01 e2 40 03 00 00 00 12 ac ed 00 |.......@........|
|00000010| 05 74 00 0b 48 65 6c 6c 6f 20 57 6f 72 6c 64    |.t..Hello World |
+--------+-------------------------------------------------+----------------+

According to the agreement specification:

• The first 8 bytes represent sessionId
• One byte indicates the request type
• 4 bytes for length
• The latter part of the content represents the message body

Test sticky package and half package problem

Split through the slice method to get two packets.

ByteBuf provides a slice method, which can split the original ByteBuf without copying the data.

public class CodesMainTest {
    public static void main( String[] args ) throws Exception {
        //EmbeddedChannel是netty专门针对ChannelHandler的单元测试而提供的类。可以通过这个类来测试channel输入入站和出站的实现
        EmbeddedChannel channel=new EmbeddedChannel(
                //解决粘包和半包问题
//                new LengthFieldBasedFrameDecoder(2048,10,4,0,0),
                new LoggingHandler(),
                new MessageRecordEncoder(),
                new MessageRecordDecode());
        Header header=new Header();
        header.setSessionId(123456);
        header.setType(OpCode.PING.code());
        MessageRecord record=new MessageRecord();
        record.setBody("Hello World");
        record.setHeader(header);
        channel.writeOutbound(record);

        ByteBuf buf= ByteBufAllocator.DEFAULT.buffer();
        new MessageRecordEncoder().encode(null,record,buf);

       //*********模拟半包和粘包问题************//
        //把一个包通过slice拆分成两个部分
        ByteBuf bb1=buf.slice(0,7); //获取前面7个字节
        ByteBuf bb2=buf.slice(7,buf.readableBytes()-7); //获取后面的字节
        bb1.retain();

        channel.writeInbound(bb1);
        channel.writeInbound(bb2);
    }
}

Running the above code will get the following exception, readerIndex(0) +length(8) means that 8 bytes are to be read, but only 7 bytes are received, so an error is reported directly.

2021-08-31 15:53:01,385 [io.netty.handler.logging.LoggingHandler]-[DEBUG] [id: 0xembedded, L:embedded - R:embedded] READ: 7B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 00 00 01 e2                            |.......         |
+--------+-------------------------------------------------+----------------+
2021-08-31 15:53:01,397 [io.netty.handler.logging.LoggingHandler]-[DEBUG] [id: 0xembedded, L:embedded - R:embedded] READ COMPLETE
Exception in thread "main" io.netty.handler.codec.DecoderException: java.lang.IndexOutOfBoundsException: readerIndex(0) + length(8) exceeds writerIndex(7): UnpooledSlicedByteBuf(ridx: 0, widx: 7, cap: 7/7, unwrapped: PooledUnsafeDirectByteBuf(ridx: 0, widx: 31, cap: 256))

Solve the unpacking problem

LengthFieldBasedFrameDecoder is a length domain decoder. It is the most commonly used decoder to solve unpacking and sticking. It can basically cover most scenes based on length unpacking. Among them, the open source message middleware RocketMQ uses this decoder for decoding.

First, let me explain the core parameters of the decoder

• lengthFieldOffset, the offset of the length field, which is the starting position of storing the length data
• lengthFieldLength, the number of bytes occupied by the length field lock
• lengthAdjustment, in some more complex protocol designs, the length field not only contains the length of the message, but also contains other data such as version number, data type, data status, etc. At this time, we can use lengthAdjustment to modify, its value = package body The length value-the value of the length field
• initialBytesToStrip, the initial number of bytes that need to be skipped after decoding, that is, the starting position of the message content field
• lengthFieldEndOffset, the offset of the end of the length field, the value of this attribute=lengthFieldOffset+lengthFieldLength

public class CodesMainTest {
    public static void main( String[] args ) throws Exception {
        EmbeddedChannel channel=new EmbeddedChannel(
                //解决粘包和半包问题
                new LengthFieldBasedFrameDecoder(1024,
                        9,4,0,0),
                new LoggingHandler(),
                new MessageRecordEncoder(),
                new MessageRecordDecode());
        Header header=new Header();
        header.setSessionId(123456);
        header.setType(OpCode.PING.code());
        MessageRecord record=new MessageRecord();
        record.setBody("Hello World");
        record.setHeader(header);
        channel.writeOutbound(record);

        ByteBuf buf= ByteBufAllocator.DEFAULT.buffer();
        new MessageRecordEncoder().encode(null,record,buf);

       //*********模拟半包和粘包问题************//
        //把一个包通过slice拆分成两个部分
        ByteBuf bb1=buf.slice(0,7);
        ByteBuf bb2=buf.slice(7,buf.readableBytes()-7);
        bb1.retain();

        channel.writeInbound(bb1);
        channel.writeInbound(bb2);
    }
}

Adding a length decoder solves the problems caused by unpacking. The results of the operation are as follows

2021-08-31 16:09:35,115 [com.netty.example.codec.MessageRecordDecode]-[INFO] 序列化出来的结果：MessageRecord(header=Header(sessionId=123456, type=3, length=18), body=Hello World)
2021-08-31 16:09:35,116 [io.netty.handler.logging.LoggingHandler]-[DEBUG] [id: 0xembedded, L:embedded - R:embedded] READ COMPLETE

Communication based on custom message protocol

Below we complete the entire communication process, and the code structure is shown in Figure 4-2.

<center>Figure 4-2</center>

Server-side development

@Slf4j
public class ProtocolServer {

    public static void main(String[] args){
        EventLoopGroup boss = new NioEventLoopGroup();
        //2 用于对接受客户端连接读写操作的线程工作组
        EventLoopGroup work = new NioEventLoopGroup();
        ServerBootstrap b = new ServerBootstrap();
        b.group(boss, work)    //绑定两个工作线程组
            .channel(NioServerSocketChannel.class)    //设置NIO的模式
            // 初始化绑定服务通道
            .childHandler(new ChannelInitializer<SocketChannel>() {
                @Override
                protected void initChannel(SocketChannel sc) throws Exception {
                    sc.pipeline()
                        .addLast(
                        new LengthFieldBasedFrameDecoder(1024,
                                                         9,4,0,0))
                        .addLast(new MessageRecordEncoder())
                        .addLast(new MessageRecordDecode())
                        .addLast(new ServerHandler());
                }
            });
        ChannelFuture cf= null;
        try {
            cf = b.bind(8080).sync();
            log.info("ProtocolServer start success");
            cf.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }finally {
            work.shutdownGracefully();
            boss.shutdownGracefully();
        }
    }
}

ServerHandler

@Slf4j
public class ServerHandler extends ChannelInboundHandlerAdapter {

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        MessageRecord messageRecord=(MessageRecord)msg;
        log.info("server receive message:"+messageRecord);
        MessageRecord res=new MessageRecord();
        Header header=new Header();
        header.setSessionId(messageRecord.getHeader().getSessionId());
        header.setType(OpCode.BUSI_RESP.code());
        String message="Server Response Message!";
        res.setBody(message);
        header.setLength(message.length());
        ctx.writeAndFlush(res);
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        log.error("服务器读取数据异常");
        super.exceptionCaught(ctx, cause);
        ctx.close();
    }
}

Client development

public class ProtocolClient {

    public static void main(String[] args) {
        //创建工作线程组
        EventLoopGroup group = new NioEventLoopGroup();
        Bootstrap b = new Bootstrap();
        b.group(group).channel(NioSocketChannel.class)
            .handler(new ChannelInitializer<SocketChannel>() {
                @Override
                public void initChannel(SocketChannel ch) throws Exception {
                    ch.pipeline().addLast(new LengthFieldBasedFrameDecoder(1024,
                                                                           9,4,0,0))
                        .addLast(new MessageRecordEncoder())
                        .addLast(new MessageRecordDecode())
                        .addLast(new ClientHandler());

                }
            });
        // 发起异步连接操作
        try {
            ChannelFuture future = b.connect(new InetSocketAddress("localhost", 8080)).sync();
            Channel c = future.channel();
            for (int i = 0; i < 500; i++) {
                MessageRecord message = new MessageRecord();
                Header header = new Header();
                header.setSessionId(10001);
                header.setType((byte) OpCode.BUSI_REQ.code());
                message.setHeader(header);
                String context="我是请求数据"+i;
                header.setLength(context.length());
                message.setBody(context);
                c.writeAndFlush(message);
            }
            //closeFuture().sync()就是让当前线程(即主线程)同步等待Netty server的close事件，Netty server的channel close后，主线程才会继续往下执行。closeFuture()在channel close的时候会通知当前线程。
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }finally {
            group.shutdownGracefully();
        }
    }
}

ClientHandler

@Slf4j
public class ClientHandler extends ChannelInboundHandlerAdapter {
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        MessageRecord record=(MessageRecord)msg;
        log.info("Client Receive message:"+record);
        super.channelRead(ctx, msg);
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        super.exceptionCaught(ctx, cause);
        ctx.close();
    }
}

Copyright statement: All articles in this blog, except for special statements, adopt the CC BY-NC-SA 4.0 license agreement. Please indicate the reprint from Mic takes you to learn architecture!
If this article is helpful to you, please help me to follow and like. Your persistence is the motivation for my continuous creation. Welcome to follow the WeChat public account of the same name for more technical dry goods!