Java handwritten RPC from scratch (04)-serialization

Serialization

java handwritten RPC from scratch (01) based on socket to achieve

java handwritten RPC from scratch (02)-netty4 to achieve client and server

java Handwritten RPC from scratch (03) How to implement the client to call the server?

In the previous sections, we have implemented the most basic client-side calling server. In this section, we will learn about object serialization in communication.

Why serialization is needed

The bottom layer of netty is based on ByteBuf for communication.

Earlier, we used the encoder/decoder to process the calculated input/output parameters, so that it is convenient for us to use pojo directly.

But there is a problem. If we want to abstract our project as a framework, we need to write encoders/decoders for all objects.

Obviously, it is unrealistic to write a pair directly through each object, and how the user uses it is also unknown.

Serialization method

The byte-based implementation has good performance and low readability.

String-based implementations, such as json serialization, have good readability and relatively poor performance.

ps: You can choose according to your personal preference. For related serialization, please refer to the following, which will not be expanded here.

json serialization framework

Realization ideas

We can convert all our Pojo to byte, and then convert Byte to ByteBuf.

vice versa.

Code

maven

Introduce the serialization package:

<dependency>
    <groupId>com.github.houbb</groupId>
    <artifactId>json</artifactId>
    <version>0.1.1</version>
</dependency>

Server

core

The server code can be greatly simplified:

serverBootstrap.group(workerGroup, bossGroup)
    .channel(NioServerSocketChannel.class)
    // 打印日志
    .handler(new LoggingHandler(LogLevel.INFO))
    .childHandler(new ChannelInitializer<Channel>() {
        @Override
        protected void initChannel(Channel ch) throws Exception {
            ch.pipeline()
                    .addLast(new RpcServerHandler());
        }
    })
    // 这个参数影响的是还没有被accept 取出的连接
    .option(ChannelOption.SO_BACKLOG, 128)
    // 这个参数只是过一段时间内客户端没有响应，服务端会发送一个 ack 包，以判断客户端是否还活着。
    .childOption(ChannelOption.SO_KEEPALIVE, true);

Only one implementation class is needed here.

RpcServerHandler

The serialization/deserialization of the server is adjusted to directly use JsonBs.

package com.github.houbb.rpc.server.handler;

import com.github.houbb.json.bs.JsonBs;
import com.github.houbb.log.integration.core.Log;
import com.github.houbb.log.integration.core.LogFactory;
import com.github.houbb.rpc.common.model.CalculateRequest;
import com.github.houbb.rpc.common.model.CalculateResponse;
import com.github.houbb.rpc.common.service.Calculator;
import com.github.houbb.rpc.server.service.CalculatorService;

import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.SimpleChannelInboundHandler;

/**
 * @author binbin.hou
 * @since 0.0.1
 */
public class RpcServerHandler extends SimpleChannelInboundHandler {

    private static final Log log = LogFactory.getLog(RpcServerHandler.class);

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        final String id = ctx.channel().id().asLongText();
        log.info("[Server] channel {} connected " + id);
    }

    @Override
    protected void channelRead0(ChannelHandlerContext ctx, Object msg) throws Exception {
        final String id = ctx.channel().id().asLongText();

        ByteBuf byteBuf = (ByteBuf)msg;
        byte[] bytes = new byte[byteBuf.readableBytes()];
        byteBuf.readBytes(bytes);
        CalculateRequest request = JsonBs.deserializeBytes(bytes, CalculateRequest.class);
        log.info("[Server] receive channel {} request: {} from ", id, request);

        Calculator calculator = new CalculatorService();
        CalculateResponse response = calculator.sum(request);

        // 回写到 client 端
        byte[] responseBytes = JsonBs.serializeBytes(response);
        ByteBuf responseBuffer = Unpooled.copiedBuffer(responseBytes);
        ctx.writeAndFlush(responseBuffer);
        log.info("[Server] channel {} response {}", id, response);
    }

}

Client

core

The client can be simplified as follows:

channelFuture = bootstrap.group(workerGroup)
    .channel(NioSocketChannel.class)
    .option(ChannelOption.SO_KEEPALIVE, true)
    .handler(new ChannelInitializer<Channel>(){
        @Override
        protected void initChannel(Channel ch) throws Exception {
            channelHandler = new RpcClientHandler();
            ch.pipeline()
                    .addLast(new LoggingHandler(LogLevel.INFO))
                    .addLast(channelHandler);
        }
    })
    .connect(RpcConstant.ADDRESS, port)
    .syncUninterruptibly();

RpcClientHandler

The serialization/deserialization of the client is adjusted to directly use JsonBs.

package com.github.houbb.rpc.client.handler;

import com.github.houbb.json.bs.JsonBs;
import com.github.houbb.log.integration.core.Log;
import com.github.houbb.log.integration.core.LogFactory;
import com.github.houbb.rpc.client.core.RpcClient;
import com.github.houbb.rpc.common.model.CalculateResponse;

import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.SimpleChannelInboundHandler;

/**
 * <p> 客户端处理类 </p>
 *
 * <pre> Created: 2019/10/16 11:30 下午  </pre>
 * <pre> Project: rpc  </pre>
 *
 * @author houbinbin
 * @since 0.0.2
 */
public class RpcClientHandler extends SimpleChannelInboundHandler {

    private static final Log log = LogFactory.getLog(RpcClient.class);

    /**
     * 响应信息
     * @since 0.0.4
     */
    private CalculateResponse response;

    @Override
    protected void channelRead0(ChannelHandlerContext ctx, Object msg) throws Exception {
        ByteBuf byteBuf = (ByteBuf)msg;
        byte[] bytes = new byte[byteBuf.readableBytes()];
        byteBuf.readBytes(bytes);

        this.response = JsonBs.deserializeBytes(bytes, CalculateResponse.class);
        log.info("[Client] response is :{}", response);
    }

    @Override
    public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
        // 每次用完要关闭，不然拿不到response，我也不知道为啥（目测得了解netty才行）
        // 个人理解：如果不关闭，则永远会被阻塞。
        ctx.flush();
        ctx.close();
    }

    public CalculateResponse getResponse() {
        return response;
    }

}

summary

In order to facilitate everyone to learn, the above source code has been open source:

https://github.com/houbb/rpc

I hope this article is helpful to you. If you like it, please like, collect and forward a wave.

I am an old horse, and I look forward to meeting you next time.