Netty series: the core encoder base64 in netty

Introduction

We know that data transmission in netty is carried out in the form of ByteBuf. It can be said that ByteBuf is the data transmission basis of netty. But for modern applications, often we need to use other data structures or types.

In order to facilitate our writing in the program, one way is to convert the data format by the programmer himself when passing the data into netty, and then call the system method of netty. Another way is to define some codecs, and the internal encoding mechanism of netty will automatically convert the data format used in the program to ByteBuf.

Obviously, the way to use codec is simpler and more in line with the development rules of the program.

In order to facilitate the development of the program, netty itself defines some core codec plug-ins, we can directly select them when needed.

This article will explain how netty internally implements codec and a core encoder base64.

Implementation logic of netty codec

The purpose of all netty codecs is to convert data types during data transmission, in other words, to process data. We know that there are two classes in netty that handle data, namely ChannelInboundHandlerAdapter and ChannelOutboundHandlerAdapter, which respectively handle inbound messages and outbound messages in the channel.

So naturally, our codec logic only needs to be added in these two places.

Netty provides us with two inherited classes of the HandlerAdapter class, MessageToMessageDecoder and MessageToMessageEncoder:

 public abstract class MessageToMessageEncoder<I> extends ChannelOutboundHandlerAdapter 

public abstract class MessageToMessageDecoder<I> extends ChannelInboundHandlerAdapter

It can be seen from the name that these two classes are used for encoding and decoding respectively, so our codec only needs to implement these two classes respectively.

The following is an example of StringToIntegerDecoder and IntegerToStringEncoder:

 public class StringToIntegerDecoder extends
               MessageToMessageDecoder<String> {
  
            @Override
           public void decode(ChannelHandlerContext ctx, String message,
                              List<Object> out) throws Exception {
               out.add(message.length());
           }
       }

 public class IntegerToStringEncoder extends
               MessageToMessageEncoder<Integer> {
  
            @Override
           public void encode(ChannelHandlerContext ctx, Integer message, List<Object> out)
                   throws Exception {
               out.add(message.toString());
           }
       }

The simplest implementation is to refactor the decode and encode methods of these two classes respectively.

Implementation of Base64 in netty

We know that there is already a tool class for Base64 implementation in JDK called java.util.Base64. But a new implementation class is also used in netty called Base64, and its full name is io.netty.handler.codec.base64.Base64.

This Base64 class uses a Base64Dialect class, which is the Base64 encoding method supported by Base64 in netty. The following types are provided in Base64Dialect:

 STANDARD
URL_SAFE
ORDERED

Among them, STANDARD corresponds to RFC3548 and the standard Base64 in JDK, URL_SAFE corresponds to the base64url version in RFC3548, and corresponds to getUrlEncode in JDK.

The last one is ORDERED, which represents RFC1940, which is not implemented in JDK.

Why is there already a Base64 tool class in JDK, and a new class needs to be created in netty?

We can consider the use of Base64 in netty. Generally speaking, we add custom encoding to handlers, and these handlers are mainly for data stream processing.

The Base64 implementation that comes with JDK can be used on fixed-length data, but if it is used in data stream processing, the efficiency will be relatively low. So Netty needs to reimplement a Base64 class for base64 in the case of streaming data.

The implementation in netty uses Robert Harder's Public Domain Base64 Encoder/Decoder. The implementation logic of this algorithm will not be discussed here. Interested friends can explore on their own.

Base64 provides a method for base64 encoding and decoding ByteBuf. We choose the method with the longest parameter to observe, as shown below:

 public static ByteBuf encode(
            ByteBuf src, int off, int len, boolean breakLines, Base64Dialect dialect, ByteBufAllocator allocator)

    public static ByteBuf decode(
            ByteBuf src, int off, int len, Base64Dialect dialect, ByteBufAllocator allocator)

For the encode method, the following parameters are required:

1. src of type ByteBuf, this is the source we need to encode.
2. The off and len of the int type indicate the position of the data to be encoded in the ByteBuf.
3. breakLines of type boolean, indicating whether to add line breaks.
4. A dialect of type Base64Dialect, indicating the selected base64 encoding type.
5. The allocator of ByteBufAllocator indicates how the returned ByteBuf is generated.

The same Decode method requires the following parameters:

1. src of type ByteBuf, this is the source we need to decode.
2. The off and len of the int type indicate the position of the data to be decoded in the ByteBuf.
3. A dialect of type Base64Dialect, indicating the selected base64 encoding type.
4. The allocator of ByteBufAllocator indicates how the returned ByteBuf is generated.

base64 encode and decoder in netty

Just now we introduced the new Base64 tool class provided in netty, which provides methods for encoding and decoding data in ByteBuf. Next, let's take a look at how netty uses this tool class to implement base64 encoding and decoding in netty.

Netty provides the encoding and decoding of Base64, namely Base64Encoder and Base64Decoder. Let's first look at the basic use of Base64 codec:

 ChannelPipeline pipeline = ...;
  
   // Decoders
   pipeline.addLast("frameDecoder", new DelimiterBasedFrameDecoder(80, Delimiters.nulDelimiter()));
   pipeline.addLast("base64Decoder", new Base64Decoder());
  
   // Encoder
   pipeline.addLast("base64Encoder", new Base64Encoder());

It is very simple to use, just add Base64Decoder and Base64Encoder to the pipeline.

Sometimes Base64Decoder needs to be used together with DelimiterBasedFrameDecoder, especially in TCP/IP protocol, because we need to judge ByteBuf should be divided into several frames according to specific Delimiters. This ensures the validity of the data.

Base64Encoder

First look at the base64 encoder, the implementation of Base64Encoder is relatively simple, first look at the definition of Base64Encoder:

 public class Base64Encoder extends MessageToMessageEncoder<ByteBuf>

Base64Encoder inherits from MessageToMessageEncoder. The generic ByteBuf it passes in indicates that ByteBuf is encoded as ByteBuf. Although the external ByteBuf type has not changed, the data in ByteBuf has been encoded into Base64.

Next is the constructor of Base64Encoder:

 public Base64Encoder(boolean breakLines, Base64Dialect dialect) {
        this.dialect = ObjectUtil.checkNotNull(dialect, "dialect");
        this.breakLines = breakLines;
    }

Base64Encoder can accept two parameters, breakLines with newline characters and Base64Dialect with base64 encoding.

Its encode method is also very simple:

 protected void encode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
        out.add(Base64.encode(msg, msg.readerIndex(), msg.readableBytes(), breakLines, dialect));
    }

Directly use the encode method of the Base64 tool class we mentioned above, and then add the return value to the out object.

Base64Decoder

Base64Decoder is used to decode the base64 encoded content in ByteBuf into original content. Let's first look at the definition of Base64Decoder:

 public class Base64Decoder extends MessageToMessageDecoder<ByteBuf>

Base64Decoder inherits MessageToMessageDecoder, and the generic type passed in is ByteBuf.

First look at the constructor of Base64Decoder:

 public Base64Decoder(Base64Dialect dialect) {
        this.dialect = ObjectUtil.checkNotNull(dialect, "dialect");
    }

The constructor of Base64Decoder is very simple. Compared with Base64Encoder, it only needs one parameter, which is the dialect of Base64Dialect type, which indicates the selected base64 decoding method.

Next is its decoding method:

 protected void decode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
        out.add(Base64.decode(msg, msg.readerIndex(), msg.readableBytes(), dialect));
    }

The decoding method also calls the decode method of the Base64 tool class, and then adds it to the returned out list.

Summarize

This chapter introduces Base64, the core encoder in netty, which is responsible for encoding the message in ByteBuf into base64 format, and provides the corresponding decoder, which you can use when needed.

This article has been included in http://www.flydean.com/14-1-netty-codec-base64/

The most popular interpretation, the most profound dry goods, the most concise tutorials, and many tricks you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", understand technology, understand you better!