Netty series: the core encoder bytes array in netty

Introduction

We know that the core of data transmission in netty is ByteBuf. ByteBuf provides a variety of data read and write methods, including basic types and byte array read and write methods. If you want to transfer these data in netty, you need to build ByteBuf, then call the corresponding method in ByteBuf to write the corresponding data, and then apply the standard template in netty to use.

For byte arrays, if it is encapsulated into ByteBuf every time, it is a bit troublesome to transmit. So netty provides a bytes-based core codec.

what is byte

So what is byte? Byte represents a byte, which is 8bits. In binary, it is the range -128-127. byte is the fundamental type in JAVA.

It also has a wrap type called Byte.

First look at the definition of Byte:

 public final class Byte extends Number implements Comparable<Byte>

The value access of byte is defined in Byte:

 public static final byte   MIN_VALUE = -128;

    public static final byte   MAX_VALUE = 127;

And also provides some basic tools and methods.

Because byte represents an 8bits binary, if not counting bit operations, byte is basically the smallest data storage unit in JAVA. So all objects in JAVA can be converted into bytes.

The conversion of basic types is not discussed here. Here we mainly look at the conversion between the string String and the object Object and byte array.

Let's first look at the conversion between String and byte array, that is, the conversion between String and binary.

The basic conversion idea is to encode the characters in the String, and then store the encoded characters.

The String class itself provides a getBytes method, which can accept the encoding type. For UTF-8, let's look at the call of the conversion method:

 public static byte[] stringToBytes(String str) throws UnsupportedEncodingException {
       return str.getBytes("utf-8");
    }

    public static String bytesToString(byte[] bs) throws UnsupportedEncodingException {
       return new String(bs, "utf-8");
    }

Just call the method in String directly.

If it is an Object object, because Object itself does not provide a conversion method, we need to use the toByteArray method of ByteArrayOutputStream and the readObject method of ByteArrayInputStream to realize the conversion between byte array and Object, as shown below:

 //对象转数组
    public byte[] toByteArray (Object obj) throws IOException {
        try(ByteArrayOutputStream bos = new ByteArrayOutputStream();
            ObjectOutputStream oos = new ObjectOutputStream(bos)) {
            oos.writeObject(obj);
            oos.flush();
            return  bos.toByteArray();
        }
    }

    //数组转对象
    public Object toObject (byte[] bytes) throws IOException, ClassNotFoundException {
        try (
            ByteArrayInputStream bis = new ByteArrayInputStream (bytes);
            ObjectInputStream ois = new ObjectInputStream (bis)) {
            return ois.readObject();
        }
    }

Tool class for byte arrays in netty

The core of netty is ByteBuf, which provides read and write methods for most basic data types. Of course, if you want to read the object, you still need to convert the object to byte and then write or read it from ByteBuf.

Of course, netty does not need to be so complicated. Netty provides an Unpooled tool class to easily convert byte arrays to ByteBuf.

Let's first look at the ByteBuff construction method provided by the Unpooled method:

 ByteBuf heapBuffer    = buffer(128);
   ByteBuf directBuffer  = directBuffer(256);
   ByteBuf wrappedBuffer = wrappedBuffer(new byte[128], new byte[256]);
   ByteBuf copiedBuffer  = copiedBuffer(ByteBuffer.allocate(128));

These are several ByteBuf construction methods provided by Unpooled, where heapBuffer represents the buff built in user space, and directBuffer represents the buff built directly in system space. wrappedBuffer is a view built on top of the existing byte array and ByteBuf, while copiedBuffer is a copy of the byte array, byteBuf and string.

Here we need to use the wrappedBuffer method to encapsulate the byte array into a ByteBuf:

 public static ByteBuf wrappedBuffer(byte[] array) {
        if (array.length == 0) {
            return EMPTY_BUFFER;
        }
        return new UnpooledHeapByteBuf(ALLOC, array, array.length);
    }

wrappedBuffer returns an UnpooledHeapByteBuf object, which itself is a ByteBuf. Here, the byte array is passed into UnpooledHeapByteBuf as a constructor.

The array here is a private variable in UnpooledHeapByteBuf:

 byte[] array;

In addition to the constructor, UnpooledHeapByteBuf also provides a setArray method to set the byte array:

 private void setArray(byte[] initialArray) {
        array = initialArray;
        tmpNioBuf = null;
    }

Here's how to build a ByteBuf from an array:

 public ByteBuf setBytes(int index, ByteBuffer src) {
        ensureAccessible();
        src.get(array, index, src.remaining());
        return this;
    }

To read the byte array from ByteBuf, you can call the getBytes method of ByteBufUtil:

 public static byte[] getBytes(ByteBuf buf) {
        return getBytes(buf,  buf.readerIndex(), buf.readableBytes());
    }

Byte encoder in netty

Everything is ready and only due to the east wind. With the tools provided by netty above, we can use these tools to build byte-based encoders.

The byte-based codecs in netty are called ByteArrayEncoder and ByteArrayDecoder respectively.

Let's take a look at how these two classes are used. Here is an example of a typical TCP/IP application:

 ChannelPipeline pipeline = ...;
  
   // Decoders
   pipeline.addLast("frameDecoder",
                    new LengthFieldBasedFrameDecoder(1048576, 0, 4, 0, 4));
   pipeline.addLast("bytesDecoder",
                    new ByteArrayDecoder());
  
   // Encoder
   pipeline.addLast("frameEncoder", new LengthFieldPrepender(4));
   pipeline.addLast("bytesEncoder", new ByteArrayEncoder());

The LengthFieldBasedFrameDecoder and LengthFieldPrepender here are frame splitters with message length as the split standard. Here we mainly focus on the ByteArrayDecoder and ByteArrayEncoder added in ChannelPipeline.

After adding the byte encoding and decoding, you can directly use the byte array directly in the handler, as shown below:

 void channelRead(ChannelHandlerContext ctx, byte[] bytes) {
       ...
   }

Let's first look at ByteArrayEncoder, which is an encoder whose implementation is very simple:

 public class ByteArrayEncoder extends MessageToMessageEncoder<byte[]> {
    @Override
    protected void encode(ChannelHandlerContext ctx, byte[] msg, List<Object> out) throws Exception {
        out.add(Unpooled.wrappedBuffer(msg));
    }
}

Specifically, use the Unpooled.wrappedBuffer method to encapsulate the byte array into a ByteBuf, and then add it to the out list.

Similarly, let's take a look at ByteArrayDecoder, which is a decoder, and the implementation is relatively simple:

 public class ByteArrayDecoder extends MessageToMessageDecoder<ByteBuf> {
    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
         // copy the ByteBuf content to a byte array
        out.add(ByteBufUtil.getBytes(msg));
    }
}

The specific implementation is to call the ByteBufUtil.getBytes method, convert the ByteBuf into a byte array, and then add it to the list object.

Summarize

If you want to transmit binary data in netty, the byte encoder and decoder provided by netty have encapsulated the tedious details, and you can use it with confidence.

This article has been included in http://www.flydean.com/14-2-netty-codec-bytes/

The most popular interpretation, the most profound dry goods, the most concise tutorials, and many tricks you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", understand technology, understand you better!