Netty series: the use of protobuf in UDP protocol

Introduction

The protobuf codec provided in netty allows us to pass protobuf objects directly in netty. At the same time, netty also provides a channel called NioDatagramChannel that supports the UDP protocol. If NioDatagramChannel is used directly, then we can read and write UDP objects directly from the channel: DatagramPacket.

But DatagramPacket encapsulates a ByteBuf object. If we want to write an object to a UDP channel, we need a method to convert the object into a ByteBuf. Obviously, the protobuf codec provided by netty is such a method.

So can NioDatagramChannel be combined with ProtobufDecoder and ProtobufEncoder?

The objects read and written by the channel in NioDatagramChannel are DatagramPacket. The ProtobufDecoder and ProtobufEncoder convert the protoBuf object MessageLiteOrBuilder and ByteBuf, so the two cannot be used in direct combination.

How can I use protobuf in UDP? Today I want to introduce to you the codecs DatagramPacketEncoder and DatagramPacketDecoder created by netty for UDP.

Representation of UDP in netty

How are UDP packets represented in netty?

Netty provides a class DatagramPacket to represent UDP packets. The UDP channel in netty uses DatagramPacket to transmit data. First look at the definition of DatagramPacket:

 public class DatagramPacket
        extends DefaultAddressedEnvelope<ByteBuf, InetSocketAddress> implements ByteBufHolder

DatagramPacket inherits from DefaultAddressedEnvelope and implements the ByteBufHolder interface.

Among them, ByteBuf is the data to be transmitted in the data packet, and InetSocketAddress is the address to which the data packet needs to be sent.

And this DefaultAddressedEnvelope inherits from AddressedEnvelope:

 public class DefaultAddressedEnvelope<M, A extends SocketAddress> implements AddressedEnvelope<M, A>

There are three properties in DefaultAddressedEnvelopee, which are message, sender and recipient:

 private final M message;
    private final A sender;
    private final A recipient;

These three properties represent the message to be sent, the sender's address, and the receiver's address, respectively.

DatagramPacketEncoder

DatagramPacketEncoder is an encoder of DatagramPacket, so the object to be encoded is DatagramPacket. In the previous section we also mentioned that DatagramPacket actually inherits from AddressedEnvelope. All DatagramPackets are an AddressedEnvelope object, so for generality, the object to be encoded by the DatagramPacketEncoder is the AddressedEnvelope.

Let's first look at the definition of DatagramPacketEncoder:

 public class DatagramPacketEncoder<M> extends MessageToMessageEncoder<AddressedEnvelope<M, InetSocketAddress>> {

DatagramPacketEncoder is a MessageToMessageEncoder that accepts a generic type of AddressedEnvelope, which is the object type we want to encode.

So what does the DatagramPacketEncoder encode the AddressedEnvelope into?

An encoder is defined in DatagramPacketEncoder, which can be passed in when DatagramPacketEncoder is initialized:

 private final MessageToMessageEncoder<? super M> encoder;

    public DatagramPacketEncoder(MessageToMessageEncoder<? super M> encoder) {
        this.encoder = checkNotNull(encoder, "encoder");
    }

In fact, the encode method implemented in DatagramPacketEncoder, the bottom layer is to call the encode method of the encoder, let's take a look at his implementation:

 protected void encode(
            ChannelHandlerContext ctx, AddressedEnvelope<M, InetSocketAddress> msg, List<Object> out) throws Exception {
        assert out.isEmpty();

        encoder.encode(ctx, msg.content(), out);
        if (out.size() != 1) {
            throw new EncoderException(
                    StringUtil.simpleClassName(encoder) + " must produce only one message.");
        }
        Object content = out.get(0);
        if (content instanceof ByteBuf) {
            // Replace the ByteBuf with a DatagramPacket.
            out.set(0, new DatagramPacket((ByteBuf) content, msg.recipient(), msg.sender()));
        } else {
            throw new EncoderException(
                    StringUtil.simpleClassName(encoder) + " must produce only ByteBuf.");
        }
    }

The above logic is to call the msg.content() method from the AddressedEnvelope to get the content in the AddressedEnvelope, and then call the encode method of the encoder to encode it and write it into out.

Finally, call the get method of out to take out the encoded content, and then encapsulate it into DatagramPacket.

So no matter what object the encoder finally returns, it will be encapsulated in the DatagramPacket and returned.

To sum up, the DatagramPacketEncoder passes in an AddressedEnvelope object, calls the encoder to encode the content of the AddressedEnvelope, and finally encapsulates it into a DatagramPacket and returns it.

Given the excellent object serialization capabilities of protoBuf, we can pass the ProtobufEncoder into the DatagramPacketEncoder as a real encoder:

 ChannelPipeline pipeline = ...;
pipeline.addLast("udpEncoder", new DatagramPacketEncoder(new ProtobufEncoder(...));

This combines ProtobufEncoder and DatagramPacketEncoder.

DatagramPacketDecoder

DatagramPacketDecoder is the opposite operation to DatagramPacketEncoder. It decodes the received DatagramPacket object. As for the object to be decoded, it is also determined by the decoder property passed in:

 public class DatagramPacketDecoder extends MessageToMessageDecoder<DatagramPacket> {

    private final MessageToMessageDecoder<ByteBuf> decoder;

    public DatagramPacketDecoder(MessageToMessageDecoder<ByteBuf> decoder) {
        this.decoder = checkNotNull(decoder, "decoder");
    }

The object to be decoded by DatagramPacketDecoder is DatagramPacket, and the object to be decoded by the incoming decoder is ByteBuf.

So we need a decoder implementation that can decode ByteBuf, and ProtobufDecoder corresponds to protoBuf.

Let's first look at how the decoder method of DatagramPacketDecoder is implemented:

 protected void decode(ChannelHandlerContext ctx, DatagramPacket msg, List<Object> out) throws Exception {
        decoder.decode(ctx, msg.content(), out);
    }

It can be seen that the decoder method of DatagramPacketDecoder is very simple, that is, to get the content content from DatagramPacket, and then hand it over to the decoder to decode.

If you use ProtobufDecoder as the built-in decoder, you can decode the ByteBuf object into a ProtoBuf object, which just echoes the encode mentioned earlier.

Passing the ProtobufDecoder into the DatagramPacketDecoder is also very simple, we can do this:

 ChannelPipeline pipeline = ...;
   pipeline.addLast("udpDecoder", new DatagramPacketDecoder(new ProtobufDecoder(...));

Such a DatagramPacketDecoder is completed.

Summarize

It can be seen that if you directly use DatagramPacketEncoder and DatagramPacketDecoder plus ProtoBufEncoder and ProtoBufDecoder, then the direct conversion between DatagramPacket and ByteBuf is achieved.

Of course, the ProtoBufEncoder and ProtoBufDecoder here can be replaced with different codecs according to user needs.

The ability to freely combine encoding and decoding methods is the greatest charm of the netty encoder.

This article has been included in http://www.flydean.com/17-1-netty-protobuf-udp/

The most popular interpretation, the most profound dry goods, the most concise tutorials, and many tricks you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", understand technology, understand you better!