Netty series: the core MessageToMessage encoder in netty

Introduction

In netty, we need to pass various types of messages. These messages can be strings, arrays, or custom objects. Different objects may need to be converted to each other, so a converter that can be converted freely is required. In order to unify coding rules and facilitate user expansion, netty provides a set of frameworks for conversion between messages. This article will explain the specific implementation of this framework.

Introduction to the framework

Netty provides three classes for the conversion between messages and messages, these three classes are abstract classes, namely MessageToMessageDecoder, MessageToMessageEncoder and MessageToMessageCodec.

Let's take a look at their definitions:

 public abstract class MessageToMessageEncoder<I> extends ChannelOutboundHandlerAdapter

 public abstract class MessageToMessageDecoder<I> extends ChannelInboundHandlerAdapter

 public abstract class MessageToMessageCodec<INBOUND_IN, OUTBOUND_IN> extends ChannelDuplexHandler

MessageToMessageEncoder inherits from ChannelOutboundHandlerAdapter and is responsible for writing messages to the channel.

MessageToMessageDecoder inherits from ChannelInboundHandlerAdapter and is responsible for reading messages from the channel.

MessageToMessageCodec inherits from ChannelDuplexHandler, which is a bidirectional handler that can read messages from the channel and write messages to the channel.

With these three abstract classes, let's look at the concrete implementation of these three classes.

MessageToMessageEncoder

Let's take a look at the message encoder MessageToMessageEncoder. The most important method in the encoder is write. Let's take a look at the implementation of write:

 public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
        CodecOutputList out = null;
        try {
            if (acceptOutboundMessage(msg)) {
                out = CodecOutputList.newInstance();
                @SuppressWarnings("unchecked")
                I cast = (I) msg;
                try {
                    encode(ctx, cast, out);
                } finally {
                    ReferenceCountUtil.release(cast);
                }

                if (out.isEmpty()) {
                    throw new EncoderException(
                            StringUtil.simpleClassName(this) + " must produce at least one message.");
                }
            } else {
                ctx.write(msg, promise);
            }
        } catch (EncoderException e) {
            throw e;
        } catch (Throwable t) {
            throw new EncoderException(t);
        } finally {
            if (out != null) {
                try {
                    final int sizeMinusOne = out.size() - 1;
                    if (sizeMinusOne == 0) {
                        ctx.write(out.getUnsafe(0), promise);
                    } else if (sizeMinusOne > 0) {
                        if (promise == ctx.voidPromise()) {
                            writeVoidPromise(ctx, out);
                        } else {
                            writePromiseCombiner(ctx, out, promise);
                        }
                    }
                } finally {
                    out.recycle();
                }
            }
        }
    }

The write method accepts a raw object msg that needs to be converted, and a ChannelPromise that represents the progress of channel read and write.

First, a type judgment will be made on msg, which is implemented in acceptOutboundMessage.

 public boolean acceptOutboundMessage(Object msg) throws Exception {
        return matcher.match(msg);
    }

The matcher here is a TypeParameterMatcher object, which is a property initialized in the MessageToMessageEncoder constructor:

 protected MessageToMessageEncoder() {
        matcher = TypeParameterMatcher.find(this, MessageToMessageEncoder.class, "I");
    }

The I here is the msg type to match.

If it does not match, continue to call ctx.write(msg, promise); to write the message to the channel without any conversion for the next handler to call.

If the match is successful, the core encode method will be called: encode(ctx, cast, out);

Note that the encode method is an abstract method in MessageToMessageEncoder and needs to be extended by the user in the inherited class.

The encode method actually converts the msg object into the object to be converted, and then adds it to out. This out is a list object, specifically a CodecOutputList object. As a list, out is a list that can store multiple objects.

So when is out written to the channel?

Don't worry, there is a finally code block at the end of the write method. In this code block, the out will be written to the channel.

Because out is a List, the object part in out may be written successfully, so special handling is required here.

First, determine whether there is only one object in out. If it is an object, write it directly to the channel. If there is more than one object in out, then it is divided into two cases. The first case is that the incoming promise is a voidPromise, then the writeVoidPromise method is called.

What is voidPromise?

We know that promises have multiple states, and we can learn about data writing through the state changes of promises. For voidPromise, it only cares about one failed state, and none of the other states.

If the user cares about other states of the promise, the writePromiseCombiner method will be called to combine the states of multiple objects into one promise and return.

In fact, in writeVoidPromise and writePromiseCombiner, the objects in out are taken out one by one and written into the channel, so multiple promises are generated and the promises need to be merged:

 private static void writeVoidPromise(ChannelHandlerContext ctx, CodecOutputList out) {
        final ChannelPromise voidPromise = ctx.voidPromise();
        for (int i = 0; i < out.size(); i++) {
            ctx.write(out.getUnsafe(i), voidPromise);
        }
    }

    private static void writePromiseCombiner(ChannelHandlerContext ctx, CodecOutputList out, ChannelPromise promise) {
        final PromiseCombiner combiner = new PromiseCombiner(ctx.executor());
        for (int i = 0; i < out.size(); i++) {
            combiner.add(ctx.write(out.getUnsafe(i)));
        }
        combiner.finish(promise);
    }

MessageToMessageDecoder

Corresponding to the encoder is the decoder, and the logic of MessageToMessageDecoder is similar to that of MessageToMessageEncoder.

First of all, it is also necessary to judge the type of message to be read. A TypeParameterMatcher object is also defined here to detect the type of incoming message:

 protected MessageToMessageDecoder() {
        matcher = TypeParameterMatcher.find(this, MessageToMessageDecoder.class, "I");
    }

The important method in the decoder is the channelRead method. Let's take a look at its implementation:

 public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        CodecOutputList out = CodecOutputList.newInstance();
        try {
            if (acceptInboundMessage(msg)) {
                @SuppressWarnings("unchecked")
                I cast = (I) msg;
                try {
                    decode(ctx, cast, out);
                } finally {
                    ReferenceCountUtil.release(cast);
                }
            } else {
                out.add(msg);
            }
        } catch (DecoderException e) {
            throw e;
        } catch (Exception e) {
            throw new DecoderException(e);
        } finally {
            try {
                int size = out.size();
                for (int i = 0; i < size; i++) {
                    ctx.fireChannelRead(out.getUnsafe(i));
                }
            } finally {
                out.recycle();
            }
        }
    }

First detect the type of msg, only the accepted type will be decoded, otherwise msg will be added to CodecOutputList.

Finally, in the finally code block, the objects in out are taken out one by one, and ctx.fireChannelRead is called to read.

The key method of message conversion is decode, which is also an abstract method and needs to implement specific functions in the inherited class.

MessageToMessageCodec

An encoder and a decoder were explained earlier, and they are both unidirectional. The last codec to be explained is called MessageToMessageCodec. This codec is bidirectional, that is, it can receive messages and send messages.

Let's take a look at its definition:

 public abstract class MessageToMessageCodec<INBOUND_IN, OUTBOUND_IN> extends ChannelDuplexHandler

MessageToMessageCodec inherits from ChannelDuplexHandler and receives two generic parameters, INBOUND_IN and OUTBOUND_IN.

It defines two TypeParameterMatcher, used to filter inboundMsg and outboundMsg respectively:

 protected MessageToMessageCodec() {
        inboundMsgMatcher = TypeParameterMatcher.find(this, MessageToMessageCodec.class, "INBOUND_IN");
        outboundMsgMatcher = TypeParameterMatcher.find(this, MessageToMessageCodec.class, "OUTBOUND_IN");
    }

The channelRead and write methods are implemented respectively to read and write messages:

 @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        decoder.channelRead(ctx, msg);
    }

    @Override
    public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
        encoder.write(ctx, msg, promise);
    }

The decoder and encoder here are actually the MessageToMessageDecoder and MessageToMessageEncoder we mentioned earlier:

 private final MessageToMessageEncoder<Object> encoder = new MessageToMessageEncoder<Object>() {

        @Override
        public boolean acceptOutboundMessage(Object msg) throws Exception {
            return MessageToMessageCodec.this.acceptOutboundMessage(msg);
        }

        @Override
        @SuppressWarnings("unchecked")
        protected void encode(ChannelHandlerContext ctx, Object msg, List<Object> out) throws Exception {
            MessageToMessageCodec.this.encode(ctx, (OUTBOUND_IN) msg, out);
        }
    };

    private final MessageToMessageDecoder<Object> decoder = new MessageToMessageDecoder<Object>() {

        @Override
        public boolean acceptInboundMessage(Object msg) throws Exception {
            return MessageToMessageCodec.this.acceptInboundMessage(msg);
        }

        @Override
        @SuppressWarnings("unchecked")
        protected void decode(ChannelHandlerContext ctx, Object msg, List<Object> out) throws Exception {
            MessageToMessageCodec.this.decode(ctx, (INBOUND_IN) msg, out);
        }
    };

It can be seen that MessageToMessageCodec is actually the encapsulation of MessageToMessageDecoder and MessageToMessageEncoder. If you need to extend MessageToMessageCodec, you need to implement the following two methods:

 protected abstract void encode(ChannelHandlerContext ctx, OUTBOUND_IN msg, List<Object> out)
            throws Exception;

    protected abstract void decode(ChannelHandlerContext ctx, INBOUND_IN msg, List<Object> out)
            throws Exception;

Summarize

The encoding framework of MessageToMessage provided in netty is the basis for extending the codec later. Only by deeply understanding the principle, we can use the new codec with ease.

This article has been included in http://www.flydean.com/14-0-1-netty-codec-msg-to-msg/

The most popular interpretation, the most profound dry goods, the most concise tutorials, and many tricks you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", understand technology, understand you better!