Tars-Java client source code analysis

1. Introduction to the basic RPC framework

In distributed computing, remote procedure call (Remote Procedure Call, abbreviated RPC) allows a program running on one computer to call a program in another address space computer, just like calling a local program, without additional involvement in this interaction Proxy object construction, network protocol, etc. for programming.

The general RPC architecture has at least three structures, namely the registry, the service provider and the service consumer. As shown in Figure 1.1, the registry provides registration services and notification services of registration information changes. The service provider runs on the server to provide services, and the service consumers use the services of the service provider.

The service provider (RPC Server) runs on the server side, provides service interface definitions and service implementation classes, and exposes service interfaces to the outside. The Registry, which runs on the server side, is responsible for recording the service objects of the service provider, and providing remote service information query services and change notification services. The service consumer (RPC Client) runs on the client and calls remote services through remote proxy objects.

1.1 RPC call flow

As shown in the following figure, it describes the call flow of RPC, where IDL (Interface Description Language) is an interface description language, so that programs running on different platforms and programs written in different languages can communicate with each other.

1) The client calls the client stub module. The call is a local procedure call, where the parameters are pushed onto the stack in the normal way.

2) The client stub module packs the parameters into the message and makes a system call to send the message. Packing parameters are called marshalling.

3) The local operating system of the client sends the message from the client computer to the server computer.

4) The local operating system on the server computer passes the incoming data packet to the server stub module.

5) The server stub module unpacks the parameters from the message. The unpacking parameter is called unpacking.

6) Finally, the server stub module executes the server program flow. The reply is to perform the same steps in the opposite direction.

2. Introduction to Tars Java Client Design

The overall design of the Tars Java client is basically the same as the mainstream RPC framework. We first introduce the initialization process of the Tars Java client.

2.1 Tars Java client initialization process

As shown in Figure 2.1, it describes the initialization process of Tars Java.

1) Create a CommunicatorConfig configuration item first, and name it communicatorConfig, in which parameters such as locator, moduleName, connections, etc. are set as needed.

2) Through the above CommunicatorConfig configuration item, named config, then call CommunicatorFactory.getInstance().getCommunicator(config) to create a Communicator object, named communicator.

3) Assuming objectName="MESSAGE.ControlCenter.Dispatcher", the proxy interface that needs to be generated is Dispatcher.class, call the communicator.stringToProxy(objectName, Dispatcher.class) method to generate the implementation class of the proxy object.

4) In the stringToProxy() method, first initialize the QueryHelper proxy object, call the getServerNodes() method to obtain the list of remote service objects, and set the return value to the objectName field of the communicatorConfig. For specific code analysis of proxy objects, see the section "2.3 Proxy Generation" below.

5) Determine whether the LoadBalance parameter is set before calling stringToProxy. If not, generate the default DefaultLoadBalance object using the RR rotation algorithm.

6) Create a TarsProtocolInvoker protocol call object, the process of which is to obtain a list of URLs by parsing the objectName and simpleObjectName in the communicatorConfig. One URL corresponds to a remote service object. TarsProtocolInvoker initializes the ServantClient object corresponding to each URL. One of the URLs is configured according to the connections of the communicatorConfig. The item confirms how many ServantClient objects are generated. Then use ServantClients and other parameters to initialize the TarsInvoker object, and set these TarsInvoker object collections to the allInvokers member variable of TarsProtocolInvoker, where each URL corresponds to a TarsInvoker object. The above analysis shows that a remote service node corresponds to a TarsInvoker object, and a TarsInvoker object contains connections and ServantClient objects. For the TCP protocol, then one ServantClient object corresponds to a TCP connection.

7) Use the api, objName, servantProxyConfig, loadBalance, protocolInvoker, this.communicator parameters to generate an ObjectProxy object that implements the JDK proxy interface InvocationHandler.

8) Initialize while generating the ObjectProxy object. First, the loadBalancer.refresh() method will be executed to refresh the remote service node to the load balancer to facilitate subsequent tars remote calls for routing.

9) Then register the statistical information reporting device, where the reporting method uses the ScheduledThreadPoolExecutor of the JDK for periodic training and reporting.

10) Registered service list refresher, the technical method adopted is basically the same as the above statistical information reporting device.

2.2 Usage example

The following code is the most simplified example. The configuration in CommunicatorConfig uses default values. After the communicator is configured through CommunicatorConfig, it directly specifies the specific service object name, IP and port of the remote service object to generate a remote service proxy object.

Tars Java code usage example // First initialize the basic Tars configuration CommunicatorConfig cfg = new CommunicatorConfig(); // Generate a Communicator object through the above CommunicatorConfig configuration. Communicator communicator = CommunicatorFactory.getInstance().getCommunicator(cfg);// Specify the service object name, IP and port of Tars remote service to generate a remote service proxy object.

// 先初始化基本Tars配置
    CommunicatorConfig cfg = new CommunicatorConfig();
    // 通过上述的CommunicatorConfig配置生成一个Communicator对象。
    Communicator communicator = CommunicatorFactory.getInstance().getCommunicator(cfg);
    // 指定Tars远程服务的服务对象名、IP和端口生成一个远程服务代理对象。
    HelloPrx proxy = communicator.stringToProxy(HelloPrx.class, "TestApp.HelloServer.HelloObj@tcp -h 127.0.0.1 -p 18601 -t 60000");
    //同步调用，阻塞直到远程服务对象的方法返回结果
    String ret = proxy.hello(3000, "Hello World");
    System.out.println(ret);
    //异步调用，不关注异步调用最终的情况
    proxy.async_hello(null, 3000, "Hello World");
      //异步调用,注册一个实现TarsAbstractCallback接口的回执处理对象，该实现类分别处理调用成功，调用超时和调用异常的情况。
    proxy.async_hello(new HelloPrxCallback() {
        @Override
        public void callback_expired() { //超时事件处理
        }
        @Override
        public void callback_exception(Throwable ex) { //异常事件处理
        }
        @Override
        public void callback_hello(String ret) { //调用成功事件处理
            Main.logger.info("invoke async method successfully {}", ret);
       }
    }, 1000, "Hello World");

In the above example, two common calling methods are demonstrated, namely synchronous calling and asynchronous calling. Among them, asynchronous call, if the caller wants to capture the final result of the asynchronous call, you can register an implementation class that implements the TarsAbstractCallback interface to handle exceptions, timeouts and success events of the tars call.

2.3 Proxy generation

The remote proxy object of the client stub module of Tars Java uses the JDK native Proxy method. As shown in the source code below, ObjectProxy implements the interface method of java.lang.reflect.InvocationHandler, which is the proxy interface that comes with the JDK.

proxy implementation

public final class ObjectProxy<T> implements ServantProxy, InvocationHandler {
    public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
        String methodName = method.getName();
        Class<?>[] parameterTypes = method.getParameterTypes();
        InvokeContext context = this.protocolInvoker.createContext(proxy, method, args);
        try {
            if ("toString".equals(methodName) && parameterTypes.length == 0) {
                return this.toString();
            } else if
                //***** 省略代码 *****
            } else {
                // 在负载均衡器选取一个远程调用类，进行应用层协议的封装，最后调用TCP传输层进行发送。
                Invoker invoker = this.loadBalancer.select(context);
                return invoker.invoke(context);
            }
        } catch (Throwable var8) {
            // ***** 省略代码 *****
        }
    }
}

Of course, the generation of the above remote service proxy class involves auxiliary classes. Tars Java uses ServantProxyFactory to generate the above ObjectProxy, and stores the ObjectProxy object in the Map structure, so that the caller can directly reuse the existing remote service proxy object when it is used for the second time.

The specific related logic is shown in the source code. ObjectProxyFactory is an auxiliary factory class that generates ObjectProxy. Unlike ServantProxyFactory, it does not cache generated proxy objects.

class ServantProxyFactory {
    private final ConcurrentHashMap<String, Object> cache = new ConcurrentHashMap();
    // ***** 省略代码 *****
    public <T> Object getServantProxy(Class<T> clazz, String objName, ServantProxyConfig servantProxyConfig, LoadBalance loadBalance, ProtocolInvoker<T> protocolInvoker) {
        Object proxy = this.cache.get(objName);
        if (proxy == null) {
            this.lock.lock(); // 加锁，保证只生成一个远程服务代理对象。
            try {
                proxy = this.cache.get(objName);
                if (proxy == null) {
                    // 创建实现JDK的java.lang.reflect.InvocationHandler接口的对象
                    ObjectProxy<T> objectProxy = this.communicator.getObjectProxyFactory().getObjectProxy(clazz, objName, servantProxyConfig, loadBalance, protocolInvoker);
                    // 使用JDK的java.lang.reflect.Proxy来生成实际的代理对象
                    this.cache.putIfAbsent(objName, this.createProxy(clazz, objectProxy));
                    proxy = this.cache.get(objName);
                }
            } finally {
                this.lock.unlock();
            }
        }
        return proxy;
    }
    /** 使用JDK自带的Proxy.newProxyInstance生成代理对象 */
    private <T> Object createProxy(Class<T> clazz, ObjectProxy<T> objectProxy) {
        return Proxy.newProxyInstance(Thread.currentThread().getContextClassLoader(), new Class[]{clazz, ServantProxy.class}, objectProxy);
    }
    // ***** 省略代码 *****
}

From the above source code, you can see that createProxy uses the JDK's Proxy.newProxyInstance method to generate remote service proxy objects.

2.4 Remote service addressing method

As an RPC remote framework, in a distributed system, calling remote services involves the issue of how to route, that is, how to select a service node from multiple remote service nodes to call. Of course, Tars Java supports direct connection to specific nodes. Invoke the remote service, as described in the 2.2 usage example above.

As shown in the figure below, a call of ClientA at a certain time uses the Service3 node for remote service calls, and a call of ClientB at a certain time uses the Service2 node. Tars Java provides a variety of load balancing algorithm implementation classes, including RoundRobinLoadBalance using RR round training algorithm, ConsistentHashLoadBalance using consistent hashing algorithm, and HashLoadBalance using common hashing algorithm.

As shown in the following source code, if you want to customize the load balancer to define the routing rules for remote calls, you need to implement the com.qq.tars.rpc.common.LoadBalance interface, where the LoadBalance.select() method is responsible for selecting according to the routing rules The corresponding Invoker object is then called remotely. For the specific logic, see the source code proxy implementation. Since the remote service node may change, such as offline remote service node, the routing information of the local load balancer needs to be refreshed, then the logic of this information update is implemented in the LoadBalance.refresh() method.

load balancing interface

public interface LoadBalance<T> {
    /** 根据负载均衡策略，挑选invoker */
    Invoker<T> select(InvokeContext invokeContext) throws NoInvokerException;
    /** 通知invoker列表的更新 */
    void refresh(Collection<Invoker<T>> invokers);
}

2.5 Network model

The IO mode of Tars Java adopts the Selector mode of JDK's NIO. The TCP protocol is used here to describe network processing. As shown in the following source code, Reactor is a thread. In the run() method, the selector.select() method is called, which means that unless an event is generated by the network at this time, it will The thread has been blocked.

If a network event occurs at this time, the thread will be awakened at this time and the subsequent code will be executed. One of the codes is dispatcheEvent(key), which means that the event will be distributed.

According to the corresponding conditions, the acceptor.handleConnectEvent(key) method is called to handle the client connection success event, or the acceptor.handleAcceptEvent(key) method is used to handle the server accepts the connection success event, or the acceptor.handleReadEvent(key) method is called from the Socket. Read data, or acceptor.handleWriteEvent(key) method to write data to Socket.

Reactor event handling

public final class Reactor extends Thread {
    protected volatile Selector selector = null;
    private Acceptor acceptor = null;
    //***** 省略代码 *****
    public void run() {
        try {
            while (!Thread.interrupted()) {
                // 阻塞直到有网络事件发生。
                selector.select();
                //***** 省略代码 *****
                while (iter.hasNext()) {
                    SelectionKey key = iter.next();
                    iter.remove();
                    if (!key.isValid()) continue;
                    try {
                        //***** 省略代码 *****
                        // 分发传输层协议TCP或UDP网络事件
                        dispatchEvent(key);
                //***** 省略代码 *****
            }
        }
        //***** 省略代码 *****
    }
        //***** 省略代码 *****
    private void dispatchEvent(final SelectionKey key) throws IOException {
        if (key.isConnectable()) {
            acceptor.handleConnectEvent(key);
        } else if (key.isAcceptable()) {
            acceptor.handleAcceptEvent(key);
        } else if (key.isReadable()) {
            acceptor.handleReadEvent(key);
        } else if (key.isValid() && key.isWritable()) {
            acceptor.handleWriteEvent(key);
        }
    }
}

Network processing adopts the Reactor event-driven model. Tars defines a Reactor object corresponding to a Selector object, and creates 2 Reactor objects by default for each remote service (overall service cluster, not a single node program) for processing, by modifying com.qq.tars .net .client.selectorPoolSize This JVM startup parameter value determines how many Reactor objects are created by a remote service.

The thread pool that handles the implementation of Read Event (Read Event) and Write Event (Write Event) in the above figure is configured when Communicator is initialized. The specific logic is shown in the source code, where the thread pool parameter configuration is determined by the corePoolSize, maxPoolSize, keepAliveTime and other parameters of CommunicatorConfig.

Read and write event thread pool initialization

private void initCommunicator(CommunicatorConfig config) throws CommunicatorConfigException {
    //***** 省略代码 *****
    this.threadPoolExecutor = ClientPoolManager.getClientThreadPoolExecutor(config);
    //***** 省略代码 *****
}
​
public class ClientPoolManager {
    public static ThreadPoolExecutor getClientThreadPoolExecutor(CommunicatorConfig communicatorConfig) {
        //***** 省略代码 *****
        clientThreadPoolMap.put(communicatorConfig, createThreadPool(communicatorConfig));
        //***** 省略代码 *****
        return clientPoolExecutor;
    }    
     
    private static ThreadPoolExecutor createThreadPool(CommunicatorConfig communicatorConfig) {
        int corePoolSize = communicatorConfig.getCorePoolSize();
        int maxPoolSize = communicatorConfig.getMaxPoolSize();
        int keepAliveTime = communicatorConfig.getKeepAliveTime();
        int queueSize = communicatorConfig.getQueueSize();
        TaskQueue taskqueue = new TaskQueue(queueSize);
​
        String namePrefix = "tars-client-executor-";
        TaskThreadPoolExecutor executor = new TaskThreadPoolExecutor(corePoolSize, maxPoolSize, keepAliveTime, TimeUnit.SECONDS, taskqueue, new TaskThreadFactory(namePrefix));
        taskqueue.setParent(executor);
        return executor;
    }
}

2.6 Remote call interaction model

Call the method of the proxy class, then it will enter the invoke method in the ObjectProxy that implements the InvocationHandler interface.

The following figure describes the process of remote service invocation. Here are a few points to focus on, one is how to write data to network IO. The second is how Tars Java makes synchronous or asynchronous calls, and what technologies are used at the bottom.

2.6.1 Write IO process

As shown in the figure (low-level code writing IO process), ServantClient will call the underlying network write operation. In the invokeWithSync method, the ServantClient's own member variable TCPSession is obtained, and the TCPSession.write() method is called, as shown in the figure (low-level code writing IO process) and As shown in the following source code (read and write event thread pool initialization), first obtain Encode to encode the request content into an IoBuffer object, and finally put the java.nio.ByteBuffer content of IoBuffer into the queue member variable of TCPSession, and then call key.selector() .wakeup(), wake up Selector.select() in the run() method of Reactor, and perform subsequent write operations.

For the specific Reactor logic, see the 2.5 network model above. If Reactor checks the conditions and finds that IO can be written, that is, key.isWritable() is true, then it will eventually take out the ByteBuffer object from TCPSession.queue in a loop and call SocketChannel.write( byteBuffer) performs the actual write network Socket operation, and the code logic is shown in the doWrite() method in the source code.

Read and write event thread pool initialization

public class TCPSession extends Session {
    public void write(Request request) throws IOException {
        try {
            IoBuffer buffer = selectorManager.getProtocolFactory().getEncoder().encodeRequest(request, this);
            write(buffer);
        //***** 省略代码 *****
    }
    protected void write(IoBuffer buffer) throws IOException {
        //***** 省略代码 *****
        if (!this.queue.offer(buffer.buf())) {
            throw new IOException("The session queue is full. [ queue size:" + queue.size() + " ]");
        }
        if (key != null) {
            key.interestOps(key.interestOps() | SelectionKey.OP_WRITE);
            key.selector().wakeup();
        }
    }
    protected synchronized int doWrite() throws IOException {
        int writeBytes = 0;
        while (true) {
            ByteBuffer wBuf = queue.peek();
            //***** 省略代码 *****
            int bytesWritten = ((SocketChannel) channel).write(wBuf);
            //***** 省略代码 *****
        return writeBytes;
    }
}

2.6.2 Implementation of the underlying technology of synchronous and asynchronous calls

For synchronous method invocation, as shown in the figure (remote invocation process) and source code (synchronous invocation of ServantClient), ServantClient calls the underlying network write operation and creates a Ticket object in the invokeWithSync method. Ticket as the name implies is the meaning of a ticket. This ticket is the only one Identifies the current network call situation.

ServantClient's synchronous call

public class ServantClient {
    public <T extends ServantResponse> T invokeWithSync(ServantRequest request) throws IOException {
            //***** 省略代码 *****
            ticket = TicketManager.createTicket(request, session, this.syncTimeout);
            Session current = session;
            current.write(request);
            if (!ticket.await(this.syncTimeout, TimeUnit.MILLISECONDS)) {
            //***** 省略代码 *****
            response = ticket.response();
            //***** 省略代码 *****
            return response;
            //***** 省略代码 *****
        return response;
    }
}

As shown in the code, after the session.write() operation is executed, the ticket.await() method is executed immediately. The thread of this method waits until the remote service replies and returns the result to the client. After the ticket.await() is awakened, it will Perform subsequent operations, and finally the invokeWithSync method returns the response object. Among them, the waiting for wake-up function of Ticket is implemented internally by java.util.concurrent.CountDownLatch.

For asynchronous method calls, the ServantClient.invokeWithAsync method will be executed, a Ticket will also be created, and the Session.write() operation will be executed. Although ticket.await() will not be called, when the Reactor receives a remote reply, it will first Parse the Tars protocol header to get the Response object, and then put the Response object into the IO read and write thread pool shown in the figure (Tars-Java's network event processing model) for further processing, as shown in the following source code (asynchronous callback event processing) , And finally call the WorkThread.run() method, and execute ticket.notifyResponse(resp) in the run() method. This method will execute the method similar to the above code 2.1 that implements the call success callback method of the TarsAbstractCallback interface.

asynchronous callback event processing

public final class WorkThread implements Runnable {
    public void run() {
        try {
            //***** 省略代码 *****
                Ticket<Response> ticket = TicketManager.getTicket(resp.getTicketNumber());
            //***** 省略代码 *****
                ticket.notifyResponse(resp);
                ticket.countDown();
                TicketManager.removeTicket(ticket.getTicketNumber());
            }
            //***** 省略代码 *****
    }
}

As shown in the following source code, TicketManager will have a timing task to check whether all calls have timed out. If the condition of (currentTime-t.startTime)> t.timeout is established, then t.expired() will be called to inform the callback object, this call time out.

call timeout event handling

public class TicketManager {
            //***** 省略代码 *****
    static {
        executor.scheduleAtFixedRate(new Runnable() {
            long currentTime = -1;
            public void run() {
                Collection<Ticket<?>> values = tickets.values();
                currentTime = System.currentTimeMillis();
                for (Ticket<?> t : values) {
                    if ((currentTime - t.startTime) > t.timeout) {
                        removeTicket(t.getTicketNumber());
                        t.expired();
                    }
                }
            }
        }, 500, 500, TimeUnit.MILLISECONDS);
    }
}

Three, summary

Code calls are generally recursive calls at various levels, and the depth and breadth of code calls are very large. It is easier to understand the meaning and design concept of the source code by learning the source code step by step by debugging the code.

There is no essential difference between Tars and other RPC frameworks. By analogy with the design concepts of other frameworks, you can have a deeper understanding of Tars Java design concepts.

Four, references

1.Remote procedure call

2. Tars Java source code Github repository

3. RPC framework Introduction and principles

Author: vivo Internet server team-Ke Shengkai