This time I designed a distributed, high-performance, and scalable RPC framework with TPS millions of levels

Author: Glacier

Blog address: https://binghe001.github.io

Hello everyone, I'm Glacier~~

That's right, this time Glacier is going to do something again. This time, the one who is going to start is RPC框架项目 . Why should you start with RPC框架项目 , because in the process of continuous development of distributed, micro-service and even cloud native, RPC, as an essential communication component at the bottom, is widely used in distributed, micro-service and cloud-native in cloud native projects.

Why develop an RPC framework

The thing is, before developing this RPC framework, I spent a lot of time thoroughly researching the Dubbo framework.

After binging through the source code of Dubbo2.x and Dubbo3.x, Binghe originally wanted to write a column on Dubbo source code analysis for everyone. To this end, I actually prepared privately for more than a month: drawing flow charts, analyzing source code, and writing test demos. When I was looking at the source code of Dubbo, I also added very detailed comments to the source code of Dubbo. Here, the source code of Dubbo2.x and Dubbo3.x is included.

When I stayed up late for more than a month, I suddenly found a problem: after years of continuous iterative development, Dubbo has a lot of source code. Know what year and month to write. When I wrote an article to analyze Dubbo's latest version 3.x, I may have written that Dubbo has been updated to 4.x and 5.x in the middle and late stages of the column, and the settings may be 6.x and 7.x.

Instead of analyzing the source code so laboriously, it is better to start from scratch with everyone to create a distributed, high-performance, and scalable RPC framework that can be used in the actual production environment. In this way, everyone can intuitively feel how an RPC framework that can be used in actual scenarios is developed step by step.

I believe that it is relatively simple to read the source code of Dubbo after learning "RPC Hands-on Column". Are you saying this is the case?

What can you learn?

Since it is the beginning of the entire column, it must tell you what practical techniques you can learn in this column. Here, I will draw a picture to intuitively tell you what technologies can be learned in the "RPC Hand Rolling Column".

The overall technical overview of the overall framework technology of "RPC Hand Lunching Column" is shown in the figure. After joining the planet and working with Glacier to realize it from scratch, get it done. When you follow the rhythm of Glacier to get this RPC framework, you will find: what Dubbo, what gRPC , what BRPC, what Hessian, what Tars, what Thrift, what motan, what hprose, etc., etc. The mainstream RPC frameworks on the market are not a problem for you. Keep up with the rhythm of Glacier, you can of.

I believe that after seeing the knowledge points involved in the "RPC Hand Rolling Column", you should be able to understand that our "RPC Hand Rolling Column" from scratch is still relatively hard-core, right?

In addition, our RPC project supports synchronous calls, asynchronous calls, callbacks and one-way calls.

• Synchronous call
  
• Asynchronous call

• callback

• one-way call

Yes, that's right, the positioning of the RPC framework finally implemented by our "RPC Hands-on Column" is to be able to use it in the actual environment as much as possible. Through the study of this column, you can deeply understand how the RPC framework that can be used in actual scenarios is developed step by step.

code structure

I position this bhrpc项目 as a distributed, high-performance, and scalable RPC framework that can be used in actual scenarios. At present, the functions that have been developed and improved in general have reached 60+ sub-projects. Bar.

The project uses a lot of custom SPI technology benchmarking Dubbo to achieve high scalability. You can add your own custom plug-ins according to your own needs and SPI design requirements.

Demonstration effect

Having said so much, let's take a look at the effect of using this RPC framework, because the calling methods supported by our RPC framework include: native RPC calling, integrating Spring (XML/annotation), integrating SpringBoot, integrating SpringCloud, integrating SpringCloud Alibaba , Integrate Docker and integrate K8S in seven ways.

Here, we will demonstrate this RPC framework by integrating Spring annotations .

RPC core annotation description

In order to give you a better understanding of this RPC framework, let me show you two core annotations of the RPC framework, one is the RPC service provider annotation @RpcService @RpcReference and the other is the RPC service caller annotation- @RpcReference .

(1) The core source code of the service provider annotation @RpcService is shown below.

 /**
 * @author binghe
 * @version 1.0.0
 * @description bhrpc服务提供者注解
 */
@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Component
public @interface RpcService {

    /**
     * 接口的Class
     */
    Class<?> interfaceClass() default void.class;

    /**
     * 接口的ClassName
     */
    String interfaceClassName() default "";

    /**
     * 版本号
     */
    String version() default "1.0.0";

    /**
     * 服务分组，默认为空
     */
    String group() default "";

    /**
     * 延迟发布，预留
     */
    int delay() default 0;

    /**
     * 是否导出rpc服务，预留
     */
    boolean export() default true;
}

(2) The core source code of the service caller's annotation @RpcReference is as follows.

 /**
 * @author binghe
 * @version 1.0.0
 * @description bhrpc服务消费者
 */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
@Autowired
public @interface RpcReference {

    /**
     * 版本号
     */
    String version() default "1.0.0";

    /**
     * 注册中心类型, 目前的类型包含：zookeeper、nacos、etcd、consul
     */
    String registryType() default "zookeeper";

    /**
     * 注册地址
     */
    String registryAddress() default "127.0.0.1:2181";

    /**
     * 负载均衡类型，默认基于ZK的一致性Hash
     */
    String loadBalanceType() default "zkconsistenthash";

    /**
     * 序列化类型，目前的类型包含：protostuff、kryo、json、jdk、hessian2、fst
     */
    String serializationType() default "protostuff";

    /**
     * 超时时间，默认5s
     */
    long timeout() default 5000;

    /**
     * 是否异步执行
     */
    boolean async() default false;

    /**
     * 是否单向调用
     */
    boolean oneway() default false;

    /**
     * 代理的类型，jdk：jdk代理， javassist: javassist代理, cglib: cglib代理
     */
    String proxy() default "jdk";

    /**
     * 服务分组，默认为空
     */
    String group() default "";
}

Here, I only list part of the source code of the service provider annotation @RpcService and the service caller annotation @RpcReference , you can take a look at it later in the process of continuous improvement of the RPC framework. To the full picture of the source code and the functions implemented by each of its annotations. Here, I will not go into details.

Of course, in the native calling method implemented by this RPC framework, remote calls can be implemented without these annotations.

Effect demonstration

Interface definition

Define two interfaces, HelloService and HelloPersonService. The source code is as follows.

• HelloService interface source code

 public interface HelloService {
    String hello(String name);
    String hello(Person person);
}

• HelloPersonService interface source code

 public interface HelloPersonService {
    List<Person> getTestPerson(String name,int num);
}

Implement the service provider demo

(1) Create the implementation classes HelloServiceImpl and HelloPersonServiceImpl of the HelloService interface and the HelloPersonService interface, as shown below.

• HelloServiceImpl class source code

 @RpcService(interfaceClass = HelloService.class, version = "1.0.0")
public class HelloServiceImpl implements HelloService {

    @Override
    public String hello(String name) {
        return "Hello! " + name;
    }

    @Override
    public String hello(Person person) {
        return "Hello! " + person.getFirstName() + " " + person.getLastName();
    }
}

It can be seen that the RPC service provider annotation @RpcService is added to the HelloServiceImpl class, indicating that it is published as an RPC service.

• HelloPersonServiceImpl class source code

 @RpcService(interfaceClass = HelloPersonService.class, version = "1.0.0")
public class HelloPersonServiceImpl implements HelloPersonService {
    @Override
    public List<Person> getTestPerson(String name, int num) {
        List<Person> persons = new ArrayList<>(num);
        for (int i = 0; i < num; ++i) {
            persons.add(new Person(Integer.toString(i), name));
        }
        return persons;
    }
}

It can be seen that the RPC service provider annotation @RpcService is added to the HelloPersonServiceImpl class, indicating that it is published as an RPC service.

(2) Create the configuration class ServerConfig of the service provider demo, inject the implementation class of the RegistryService registry interface into the ServerConfig class, and the core class RpcServer of the RPC service provider, as shown below.

 /**
 * @author binghe
 * @version 1.0.0
 * @description 基于注解的配置类
 */
@Configuration
@ComponentScan(value = {"io.binghe.rpc.demo"})
@PropertySource(value = {"classpath:rpc.properties"})
public class SpringAnnotationProviderConfig {

    @Value("${registry.address}")
    private String registryAddress;

    @Value("${registry.type}")
    private String registryType;

    @Value("${registry.loadbalance.type}")
    private String registryLoadbalanceType;

    @Value("${server.address}")
    private String serverAddress;

    @Value("${reflect.type}")
    private String reflectType;

    @Bean
    public RpcSpringServer rpcSpringServer(){
        return new RpcSpringServer(serverAddress, registryAddress, registryType, registryLoadbalanceType, reflectType);
    }
}

(3) Create the startup class ServerTest of the service provider demo, as shown below.

 /**
 * @author binghe
 * @version 1.0.0
 * @description RPC整合Spring注解，服务提供者demo启动类
 */
public class ServerTest {
    public static void main(String[] args){
        new AnnotationConfigApplicationContext(ServerConfig.class);
    }
}

Implement the service caller demo

(1) Create the TestService interface of the test service caller, as shown below.

 public interface TestService {
    void printResult();
}

(2) Create the implementation class TestServiceImpl of the TestService interface, mark Spring's @Service annotation on the TestServiceImpl class, and inject the implementation class of the HelloService interface and the HelloPersonService interface through the @RpcReference annotation in the TestServiceImpl class , and implement the printResult() method of the TestService interface. The source code is as follows.

 /**
 * @author binghe
 * @version 1.0.0
 * @description 测试RPC服务调用者
 */
@Service
public class TestServiceImpl implements TestService {

    @RpcReference(version = "1.0.0", timeout = 3000, proxy = "javassist", isAsync = true)
    private HelloService helloService;
    
    @RpcReference(proxy = "cglib")
    private HelloPersonService helloPersonService;

    @Override
    public void printResult() {
        String result = helloService.hello("binghe");
        System.out.println(result);
        result = helloService.hello(new Person("binghe001", "binghe002"));
        System.out.println(result);
        System.out.println("=================================");
        List<Person> personList = helloPersonService.getTestPerson("binghe", 2);
        personList.stream().forEach(System.out::println);
    }
}

From the source code of the TestServiceImpl class, we can see that the javassist dynamic proxy is used when calling the methods of the HelloService interface remotely, and the cglib dynamic proxy is used when calling the HelloPersonService interface remotely.

(3) Create the configuration class ClientConfig of the service caller demo, as shown below.

 @Configuration
@ComponentScan(value = {"io.binghe.rpc.*"})
@PropertySource(value = {"classpath:rpc.properties"})
public class ClientConfig {
}

(4) Create the startup class ClientTest of the service caller demo, as shown below.

 public class ClientTest {

    public static void main(String[] args){
        AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(ClientConfig.class);
        TestService testService = context.getBean(TestService.class);
        testService.printResult();
        context.close();
    }
}

Start service test

(1) Start Zookeeper. Here, for the simplicity of the demonstration, just start the stand-alone Zookeeper on my local machine. The effect after startup is shown in the figure below.

(2) Start the service provider ServerTest class, and the log information output after startup is as follows.

 13:43:36,876  INFO ConnectionStateManager:228 - State change: CONNECTED
13:43:36,905  INFO RpcClient:79 - use cglib dynamic proxy...
13:43:36,942  INFO CuratorFrameworkImpl:235 - Starting
13:43:36,943  INFO ZooKeeper:868 - Initiating client connection, connectString=127.0.0.1:2181

As you can see, the service provider has registered the published service with Zookeeper.

(3) Log in to the Zookeeper client to view the services registered in Zookeeper, as shown below.

• View the service information published by the HelloService interface

 [zk: localhost:2181(CONNECTED) 5] get /binghe_rpc/io.binghe.rpc.test.client.HelloService#1.0.0/65eb0d7f-4bf7-4a0a-bafc-1b7e0e030353

{"name":"io.binghe.rpc.test.client.HelloService#1.0.0","id":"65eb0d7f-4bf7-4a0a-bafc-1b7e0e030353","address":"127.0.0.1","port":18866,"sslPort":null,"payload":{"@class":"io.binghe.rpc.center.meta.ServiceMeta","serviceName":"io.binghe.rpc.test.client.HelloService","serviceVersion":"1.0.0","serviceAddr":"127.0.0.1","servicePort":18866},"registrationTimeUTC":1656135817627,"serviceType":"DYNAMIC","uriSpec":null,"enabled":true}

• View the service information published by the HelloPersonService interface

 [zk: localhost:2181(CONNECTED) 7] get /binghe_rpc/io.binghe.rpc.test.client.HelloPersonService#1.0.0/882a5cdb-f581-4a83-8d56-800a8f14e831

{"name":"io.binghe.rpc.test.client.HelloPersonService#1.0.0","id":"882a5cdb-f581-4a83-8d56-800a8f14e831","address":"127.0.0.1","port":18866,"sslPort":null,"payload":{"@class":"io.binghe.rpc.center.meta.ServiceMeta","serviceName":"io.binghe.rpc.test.client.HelloPersonService","serviceVersion":"1.0.0","serviceAddr":"127.0.0.1","servicePort":18866},"registrationTimeUTC":1656135817274,"serviceType":"DYNAMIC","uriSpec":null,"enabled":true}

It can be seen from the Zookeeper client that the services published by the HelloService interface and the HelloPersonService interface have been registered with Zookeeper.

(4) Start the service provider ClientTest class to implement RPC calls, and the output log information is as follows.

 13:56:47,391  INFO ConnectionStateManager:228 - State change: CONNECTED
13:56:47,488  INFO RpcClient:76 - use javassist dynamic proxy...
13:56:47,518  INFO ConnectionStateManager:228 - State change: CONNECTED
13:56:47,545  INFO RpcClient:79 - use cglib dynamic proxy...
13:56:48,253  INFO RpcConsumer:85 - connect rpc server 127.0.0.1 on port 18866 success.
Hello! binghe
Hello! binghe001 binghe002
=================================
0 binghe
1 binghe

As you can see, the result information of the remote call is output on the command line of the ClientTest class. And output the remote method calling the HelloService interface using the javassist dynamic proxy. The remote method that calls the HelloPersonService interface uses the cglib dynamic proxy.

The RPC framework we built together actually has many very powerful functions. Here, we will not demonstrate them one by one. We will implement it together later.

little advice

This column of ours is a column with a strong practical type, and the RPC framework we hand-built from scratch will involve many knowledge points. As the saying goes, what you get on paper is shallow, and you never know that you have to do it. Glacier hopes that everyone will be diligent in learning this column and implement the code together with the column. During the period, you must use your brain and summarize more, so that you can deepen your understanding of various knowledge points. Don't overthink things, learn for a long time but learn nothing in the end.

Well, that's it for today's opening article. If the article is a little helpful to you, remember to give Binghe one-click three links. Welcome to forward the article to more friends, Binghe will be very grateful~~

set off together

I will put the source code acquisition method of "RPC Hand Lunching Column" in the knowledge planet, and at the same time, I will create a special knowledge planet group on WeChat. Glacier will answer the questions of golfers on the knowledge planet and in the planet group, and pay attention to Glacier technology. Reply to Planet to get the coupon.

Everyone is welcome to forward articles or planets to the group or circle of friends. These contents will be continuously improved in the time of get off work, weekends and holidays. Through video + article + knowledge booklet + live broadcast + homework, I will learn, improve and progress with you. The ultimate goal is to improve your technical strength, let you go further in the workplace, and make more money by the way.

Okay, let's stop here today, I'm Glacier, see you next time~~