Netty series: automatic decoder ReplayingDecoder in netty

Introduction

Netty provides a decoder from ByteBuf to user-defined message called ByteToMessageDecoder. To use this decoder, we need to inherit this decoder and implement the decode method, so as to implement the content in ByteBuf to the user-defined message object in this method conversion.

So what problems will be encountered in the process of using ByteToMessageDecoder? Why is there another ReplayingDecoder? Let's take a look at this question.

Possible problems with ByteToMessageDecoder

If you want to implement your own decoder to convert ByteBuf into your own message object, you can inherit ByteToMessageDecoder, and then implement the decode method. Let's first look at the definition of the decode method:

 protected void decode(ChannelHandlerContext ctx,
                             ByteBuf buf, List<Object> out) throws Exception

In the input parameters, buf is the ByteBuf to be decoded, and out is the decoded object list. We need to convert the data in the ByteBuf into our own object and add it to the out list.

Then there may be a problem here, because the data in buf may not be ready when we call the decode method. For example, we need an Integer, but the data in buf is not enough for an integer, so we need some data logic in buf Judgment, we describe this process with a Buf object with message length.

The so-called Buf object with message length means that the first 4 bits in the Buf message constitute an integer, which represents the length of the subsequent message in the buf.

Therefore, the process of reading the message for conversion is to first read the first 4 bytes to get the length of the message, and then read the bytes of this length. This is the content of the message we really want to get.

Let's see how to implement this logic if it is inherited from ByteToMessageDecoder?

 public class IntegerHeaderFrameDecoder extends ByteToMessageDecoder {
  
      @Override
     protected void decode(ChannelHandlerContext ctx,
                             ByteBuf buf, List<Object> out) throws Exception {
  
       if (buf.readableBytes() < 4) {
          return;
       }
  
       buf.markReaderIndex();
       int length = buf.readInt();
  
       if (buf.readableBytes() < length) {
          buf.resetReaderIndex();
          return;
       }
  
       out.add(buf.readBytes(length));
     }
   }

In decoding, we first need to determine whether there are 4 readable bytes in buf, if not, return directly. If there is, read the length of these 4 bytes first, and then judge whether the readable bytes in buf are less than the length that should be read. If it is less than that, it means that the data is not ready, and you need to call resetReaderIndex to reset .

Finally, if all the conditions are met, the actual read work is performed.

Is there a way to read the buf directly according to the content you want without making a judgment in advance? The answer is ReplayingDecoder.

Let's first take a look at what happens when the above example is rewritten with ReplayingDecoder:

 public class IntegerHeaderFrameDecoder
        extends ReplayingDecoder<Void> {
  
     protected void decode(ChannelHandlerContext ctx,
                             ByteBuf buf, List<Object> out) throws Exception {
  
       out.add(buf.readBytes(buf.readInt()));
     }
   }

Using ReplayingDecoder, we can ignore whether buf has received enough readable data and just read it directly.

In contrast, ReplayingDecoder is very simple. Next, let's explore the implementation principle of ReplayingDecoder.

Implementation principle of ReplayingDecoder

ReplayingDecoder is actually a subclass of ByteToMessageDecoder, which is defined as follows:

 public abstract class ReplayingDecoder<S> extends ByteToMessageDecoder

In ByteToMessageDecoder, the most important method is channelRead. In this method, callDecode(ctx, cumulation, out); is actually called to realize the decoding process from accumulation to out.

The secret of ReplayingDecoder is to rewrite this method. Let's take a look at the specific implementation of this method:

 protected void callDecode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) {
        replayable.setCumulation(in);
        try {
            while (in.isReadable()) {
                int oldReaderIndex = checkpoint = in.readerIndex();
                int outSize = out.size();
                if (outSize > 0) {
                    fireChannelRead(ctx, out, outSize);
                    out.clear();
                    if (ctx.isRemoved()) {
                        break;
                    }
                    outSize = 0;
                }
                S oldState = state;
                int oldInputLength = in.readableBytes();
                try {
                    decodeRemovalReentryProtection(ctx, replayable, out);
                    if (ctx.isRemoved()) {
                        break;
                    }
                    if (outSize == out.size()) {
                        if (oldInputLength == in.readableBytes() && oldState == state) {
                            throw new DecoderException(
                                    StringUtil.simpleClassName(getClass()) + ".decode() must consume the inbound " +
                                    "data or change its state if it did not decode anything.");
                        } else {
                            continue;
                        }
                    }
                } catch (Signal replay) {
                    replay.expect(REPLAY);
                    if (ctx.isRemoved()) {
                        break;
                    }

                    // Return to the checkpoint (or oldPosition) and retry.
                    int checkpoint = this.checkpoint;
                    if (checkpoint >= 0) {
                        in.readerIndex(checkpoint);
                    } else {
                    }
                    break;
                }
                if (oldReaderIndex == in.readerIndex() && oldState == state) {
                    throw new DecoderException(
                           StringUtil.simpleClassName(getClass()) + ".decode() method must consume the inbound data " +
                           "or change its state if it decoded something.");
                }
                if (isSingleDecode()) {
                    break;
                }
            }
        } catch (DecoderException e) {
            throw e;
        } catch (Exception cause) {
            throw new DecoderException(cause);
        }
    }

The implementation here is different from ByteToMessageDecoder in that ReplayingDecoder defines a checkpoint, which is set at the beginning of trying to decode data:

 int oldReaderIndex = checkpoint = in.readerIndex();

If an exception occurs during decoding, use checkpoint to reset the index:

 int checkpoint = this.checkpoint;
         if (checkpoint >= 0) {
            in.readerIndex(checkpoint);
        } else {
    }

The exception caught here is Signal, what is Signal?

Signal is an Error object:

 public final class Signal extends Error implements Constant<Signal>

This exception is thrown from the replayable.

replayable is a special ByteBuf object called ReplayingDecoderByteBuf:

 final class ReplayingDecoderByteBuf extends ByteBuf

The Signal property is defined in ReplayingDecoderByteBuf:

 private static final Signal REPLAY = ReplayingDecoder.REPLAY;

This Signal exception is thrown from the get method in ReplayingDecoderByteBuf. Take getInt as an example to see how the exception is thrown:

 public int getInt(int index) {
        checkIndex(index, 4);
        return buffer.getInt(index);
    }

The getInt method will first call the checkIndex method to detect the length in the buff. If it is less than the length to be read, an exception REPLAY will be thrown:

 private void checkIndex(int index, int length) {
        if (index + length > buffer.writerIndex()) {
            throw REPLAY;
        }
    }

This is where the Signal exception comes in.

Summarize

The above is the introduction of ReplayingDecoder. Although ReplayingDecoder is easy to use, it can be seen from its implementation that ReplayingDecoder continuously retrying by throwing exceptions, so in some special cases, it will cause performance degradation.

That is to say, while reducing the amount of our code, the execution efficiency of the program is reduced. It seems that it is impossible for a horse to run and not to graze.

This article has been included in http://www.flydean.com/14-4-netty-replayingdecoder/

The most popular interpretation, the most profound dry goods, the most concise tutorials, and many tricks you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", understand technology, understand you better!