Java version of gRPC combat six: the client dynamically obtains the server address

Welcome to my GitHub

https://github.com/zq2599/blog_demos

Content: Classification and summary of all original articles and supporting source code, involving Java, Docker, Kubernetes, DevOPS, etc.;

"Java version of gRPC combat" full series of links

1. Generate code with proto
2. service release and call
3. server stream
4. client stream
5. Bidirectional flow
6. client dynamically obtains the server address
7. Based on eureka registration discovery

Why the client needs to dynamically obtain the server address

This article is the sixth in the "java version of gRPC combat" series. When we were developing client applications, the required server address was set according to the following steps:

• Configure in application.yml, as shown below:

• In the bean that uses gRPC, use the annotation <font color="blue">GrpcClient</font> to inject the Stub class into the member variable:

• The advantages of the above operation methods are simple and easy to use and good configuration, but the disadvantages are also obvious: once the IP address or port of the server changes, you must modify the application.yml and restart the client application;

Why not use the registration center

• You will surely think that the easiest way to solve the above problems is to use a registry, such as nacos, eureka, etc. In fact, I think so, until one day, due to work reasons, I will deploy myself in an existing gRPC microservice environment The application of this microservice environment is not a java technology stack, but is based on golang. They all use the go-zero framework (it’s too old). This go-zero framework does not provide a Java language SDK, so I can only Obey the rules of the go-zero framework and obtain the address information of other microservices from etcd before calling other gRPC servers, as shown in the following figure:

• In this way, our previous method of configuring server information in application.yml is not available. In this article, we will develop a new gRPC client application to meet the following requirements:
• When creating a Stub object, the server information no longer comes from the annotation <font color="blue">GrpcClient</font>, but from the result of querying etcd;
• When the server information on etcd changes, the client can update it in time without restarting the application;

Overview of this article

• In this article, we will develop a springboot application named <font color="blue">get-service-addr-from-etcd</font>, which is obtained from etcd <font color="blue">local-server</font >The IP and port of the application, and then call the <font color="red">sayHello</font> interface of the local-server, as shown below:

• The <font color="blue">local-server</font> application is a simple gRPC server. For details, please refer to "Java version of gRPC combat 2: Service release and invocation"
• This chapter consists of the following chapters:
• Develop client applications;
• Deploy gRPC server application;
• Deploy etcd;
• Imitate the rules of go-zero, write the IP address and port of the server application into etcd;
• Start the client application and verify that the service on the server can be called normally;
• Restart the server and modify the port when restarting;
• Modify the port information of the server in etcd;
• Call the interface to trigger the client to re-instantiate the Stub object;
• Verify that the client can normally call the server service with the modified port;

Source download

• The complete source code in this actual combat can be downloaded on GitHub. The address and link information are shown in the following table ( https://github.com/zq2599/blog_demos):

name	Link	Remark
Project homepage	https://github.com/zq2599/blog_demos	The project's homepage on GitHub
git warehouse address (https)	https://github.com/zq2599/blog_demos.git	The warehouse address of the source code of the project, https protocol
git warehouse address (ssh)	[e-mail]:zq2599/blog_demos.git	The warehouse address of the source code of the project, ssh protocol

• There are multiple folders in this git project. The source code of the "java version of gRPC combat" series is under the <font color="blue">grpc-tutorials</font> folder, as shown in the red box below:

• There are multiple directories under the <font color="blue">grpc-tutorials</font> folder. The client code corresponding to this article is in <font color="blue">get-service-addr-from-etcd< /font> directory, as shown below:

Develop client applications

• Add a new line to the build.gradle file of the parent project, which is the etcd-related library, as shown in the red box in the following figure:

• Create a new module named <font color="blue">get-service-addr-from-etcd</font> under the parent project grpc-turtorials, and its build.gradle content is as follows:

plugins {
    id 'org.springframework.boot'
}

dependencies {
    implementation 'org.projectlombok:lombok'
    implementation 'org.springframework.boot:spring-boot-starter'
    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'net.devh:grpc-client-spring-boot-starter'
    implementation 'io.etcd:jetcd-core'
    implementation project(':grpc-lib')
}

• The configuration file application.yml, set your own web port number and application name, and grpc.etcdendpoints is the address information of etcd cluster:

server:
  port: 8084
spring:
  application:
    name: get-service-addr-from-etcd

grpc:
  # etcd的地址，从此处取得gRPC服务端的IP和端口
  etcdendpoints: 'http://192.168.72.128:2379,http://192.168.50.239:2380,http://192.168.50.239:2381'

• The code of the startup class DynamicServerAddressDemoApplication.java is not posted, just the ordinary springboot startup class;
• Added the StubWrapper.java file, which is a spring bean. The simpleBlockingStub method should be focused on. When the bean is registered in spring, the simpleBlockingStub method will be executed, so that whenever the bean is registered in spring, the gRPC server information will be queried from etcd , And then create a SimpleBlockingStub object:

package com.bolingcavalry.dynamicrpcaddr;

import com.bolingcavalry.grpctutorials.lib.SimpleGrpc;
import io.etcd.jetcd.ByteSequence;
import io.etcd.jetcd.Client;
import io.etcd.jetcd.KV;
import io.etcd.jetcd.kv.GetResponse;
import io.grpc.Channel;
import io.grpc.ManagedChannelBuilder;
import lombok.Data;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import java.util.Arrays;
import static com.google.common.base.Charsets.UTF_8;

/**
 * @author will ([e-mail])
 * @version 1.0
 * @description: 包装了SimpleBlockingStub实例的类，发起gRPC请求时需要用到SimpleBlockingStub实例
 * @date 2021/5/8 19:34
 */
@Component("stubWrapper")
@Data
@Slf4j
@ConfigurationProperties(prefix = "grpc")
public class StubWrapper {

    /**
     * 这是etcd中的一个key，该key对应的值是grpc服务端的地址信息
     */
    private static final String GRPC_SERVER_INFO_KEY = "/grpc/local-server";

    /**
     * 配置文件中写好的etcd地址
     */
    private String etcdendpoints;

    private SimpleGrpc.SimpleBlockingStub simpleBlockingStub;

    /**
     * 从etcd查询gRPC服务端的地址
     * @return
     */
    public String[] getGrpcServerInfo() {
        // 创建client类
        KV kvClient = Client.builder().endpoints(etcdendpoints.split(",")).build().getKVClient();

        GetResponse response = null;

        // 去etcd查询/grpc/local-server这个key的值
        try {
            response = kvClient.get(ByteSequence.from(GRPC_SERVER_INFO_KEY, UTF_8)).get();
        } catch (Exception exception) {
            log.error("get grpc key from etcd error", exception);
        }

        if (null==response || response.getKvs().isEmpty()) {
            log.error("empty value of key [{}]", GRPC_SERVER_INFO_KEY);
            return null;
        }

        // 从response中取得值
        String rawAddrInfo = response.getKvs().get(0).getValue().toString(UTF_8);

        // rawAddrInfo是“192.169.0.1:8080”这样的字符串，即一个IP和一个端口，用":"分割，
        // 这里用":"分割成数组返回
        return null==rawAddrInfo ? null : rawAddrInfo.split(":");
    }

    /**
     * 每次注册bean都会执行的方法，
     * 该方法从etcd取得gRPC服务端地址，
     * 用于实例化成员变量SimpleBlockingStub
     */
    @PostConstruct
    public void simpleBlockingStub() {
        // 从etcd获取地址信息
        String[] array = getGrpcServerInfo();

        log.info("create stub bean, array info from etcd {}", Arrays.toString(array));

        // 数组的第一个元素是gRPC服务端的IP地址，第二个元素是端口
        if (null==array || array.length<2) {
            log.error("can not get valid grpc address from etcd");
            return;
        }

        // 数组的第一个元素是gRPC服务端的IP地址
        String addr = array[0];
        // 数组的第二个元素是端口
        int port = Integer.parseInt(array[1]);

        // 根据刚才获取的gRPC服务端的地址和端口，创建channel
        Channel channel = ManagedChannelBuilder
                .forAddress(addr, port)
                .usePlaintext()
                .build();

        // 根据channel创建stub
        simpleBlockingStub = SimpleGrpc.newBlockingStub(channel);
    }
}

• GrpcClientService is a service class that encapsulates StubWrapper:

package com.bolingcavalry.dynamicrpcaddr;

import com.bolingcavalry.grpctutorials.lib.HelloReply;
import com.bolingcavalry.grpctutorials.lib.HelloRequest;
import com.bolingcavalry.grpctutorials.lib.SimpleGrpc;
import io.grpc.StatusRuntimeException;
import lombok.Setter;
import net.devh.boot.grpc.client.inject.GrpcClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
public class GrpcClientService {

    @Autowired(required = false)
    @Setter
    private StubWrapper stubWrapper;

    public String sendMessage(final String name) {
        // 很有可能simpleStub对象为null
        if (null==stubWrapper) {
            return "invalid SimpleBlockingStub, please check etcd configuration";
        }

        try {
            final HelloReply response = stubWrapper.getSimpleBlockingStub().sayHello(HelloRequest.newBuilder().setName(name).build());
            return response.getMessage();
        } catch (final StatusRuntimeException e) {
            return "FAILED with " + e.getStatus().getCode().name();
        }
    }
}

• A new controller class GrpcClientController is added to provide an http interface, which will call the method of GrpcClientService, and finally complete the remote gRPC call:

package com.bolingcavalry.dynamicrpcaddr;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class GrpcClientController {

    @Autowired
    private GrpcClientService grpcClientService;

    @RequestMapping("/")
    public String printMessage(@RequestParam(defaultValue = "will") String name) {
        return grpcClientService.sendMessage(name);
    }
}

• Next, add a new controller class, RefreshStubInstanceController, which provides an http interface refreshstub to the outside. The function is to delete the stubWrapper bean and register again, so that whenever the refreshstub interface is called externally, the server information can be obtained from etcd and then the SimpleBlockingStub can be re-instantiated Member variable, so as to achieve the effect of the client dynamically obtaining the server address:

package com.bolingcavalry.dynamicrpcaddr;

import com.bolingcavalry.grpctutorials.lib.SimpleGrpc;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.support.AbstractBeanDefinition;
import org.springframework.beans.factory.support.BeanDefinitionBuilder;
import org.springframework.beans.factory.support.BeanDefinitionRegistry;
import org.springframework.beans.factory.support.DefaultListableBeanFactory;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class RefreshStubInstanceController implements ApplicationContextAware {

    private ApplicationContext applicationContext;

    @Autowired
    private GrpcClientService grpcClientService;

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        this.applicationContext = applicationContext;
    }

    @RequestMapping("/refreshstub")
    public String refreshstub() {

        String beanName = "stubWrapper";

        //获取BeanFactory
        DefaultListableBeanFactory defaultListableBeanFactory = (DefaultListableBeanFactory) applicationContext.getAutowireCapableBeanFactory();

        // 删除已有bean
        defaultListableBeanFactory.removeBeanDefinition(beanName);

        //创建bean信息.
        BeanDefinitionBuilder beanDefinitionBuilder = BeanDefinitionBuilder.genericBeanDefinition(StubWrapper.class);

        //动态注册bean.
        defaultListableBeanFactory.registerBeanDefinition(beanName, beanDefinitionBuilder.getBeanDefinition());

        // 更新引用关系(注意，applicationContext.getBean方法很重要，会触发StubWrapper实例化操作)
        grpcClientService.setStubWrapper(applicationContext.getBean(StubWrapper.class));

        return "Refresh success";
    }
}

• Encoding is complete, start verification;

Deploy gRPC server application

Deploying the gRPC server application is very simple, just start the <font color="blue">local-server</font> application:

Deploy etcd

• In order to simplify the operation, my etcd cluster here is deployed with docker, and the corresponding docker-compose.yml file content is as follows:

version: '3'
services:
  etcd1:
    image: "quay.io/coreos/etcd:v3.4.7"
    entrypoint: /usr/local/bin/etcd
    command:
      - '--name=etcd1'
      - '--data-dir=/etcd_data'
      - '--initial-advertise-peer-urls=http://etcd1:2380'
      - '--listen-peer-urls=http://0.0.0.0:2380'
      - '--listen-client-urls=http://0.0.0.0:2379'
      - '--advertise-client-urls=http://etcd1:2379'
      - '--initial-cluster-token=etcd-cluster'
      - '--heartbeat-interval=250'
      - '--election-timeout=1250'
      - '--initial-cluster=etcd1=http://etcd1:2380,etcd2=http://etcd2:2380,etcd3=http://etcd3:2380'
      - '--initial-cluster-state=new'
    ports:
      - 2379:2379
    volumes:
      - ./store/etcd1/data:/etcd_data
  etcd2:
    image: "quay.io/coreos/etcd:v3.4.7"
    entrypoint: /usr/local/bin/etcd
    command:
      - '--name=etcd2'
      - '--data-dir=/etcd_data'
      - '--initial-advertise-peer-urls=http://etcd2:2380'
      - '--listen-peer-urls=http://0.0.0.0:2380'
      - '--listen-client-urls=http://0.0.0.0:2379'
      - '--advertise-client-urls=http://etcd2:2379'
      - '--initial-cluster-token=etcd-cluster'
      - '--heartbeat-interval=250'
      - '--election-timeout=1250'
      - '--initial-cluster=etcd1=http://etcd1:2380,etcd2=http://etcd2:2380,etcd3=http://etcd3:2380'
      - '--initial-cluster-state=new'
    ports:
      - 2380:2379
    volumes:
      - ./store/etcd2/data:/etcd_data
  etcd3:
    image: "quay.io/coreos/etcd:v3.4.7"
    entrypoint: /usr/local/bin/etcd
    command:
      - '--name=etcd3'
      - '--data-dir=/etcd_data'
      - '--initial-advertise-peer-urls=http://etcd3:2380'
      - '--listen-peer-urls=http://0.0.0.0:2380'
      - '--listen-client-urls=http://0.0.0.0:2379'
      - '--advertise-client-urls=http://etcd3:2379'
      - '--initial-cluster-token=etcd-cluster'
      - '--heartbeat-interval=250'
      - '--election-timeout=1250'
      - '--initial-cluster=etcd1=http://etcd1:2380,etcd2=http://etcd2:2380,etcd3=http://etcd3:2380'
      - '--initial-cluster-state=new'
    ports:
      - 2381:2379
    volumes:
      - ./store/etcd3/data:/etcd_data

• After preparing the above files, execute <font color="blue">docker-compose up -d</font> to create a cluster;

Write the IP address and port of the server application to etcd

• The IP of the server where my <font color="blue">local-server</font> is located is <font color="red">192.168.50.5</font>, port 9898, so execute the following command to change the local-server information Write to etcd:

docker exec 08_etcd2_1 /usr/local/bin/etcdctl put /grpc/local-server 192.168.50.5:9898

Start the client application

• Open DynamicServerAddressDemoApplication.java and click on the position in the red box in the figure below to start the client application:

• Pay attention to the log in the red box in the figure below, which proves that the client application successfully obtained server information from etcd:

• The browser accesses the http interface of the application <font color="blue">get-service-addr-from-etcd</font> and successfully receives the response, which proves that the gRPC call is successful:

• Go to the console of the local-server, the red box in the figure below proves that the remote call is indeed executed:

Restart the server and modify the port when restarting

• In order to verify whether the dynamic access to server information is valid, let's first change the port of the local-server application, as shown in the red box in the figure below, to <font color="red">9899</font>:

• After changing and restarting local-server, the red box in the following figure shows that the gRPC port has been changed to 9899:

• At this time, visit the http interface of <font color="blue">get-service-addr-from-etcd</font>, because get-service-addr-from-etcd does not know that the listening port of local-server has changed , So I went to visit port 9898, and it failed without accident:

Modify the port information of the server in etcd

Now execute the following command to change the server information in etcd to the correct one:

docker exec 08_etcd2_1 /usr/local/bin/etcdctl put /grpc/local-server 192.168.50.5:9899

Call the interface to trigger the client to re-instantiate the Stub object

• Smart, you must know what to do next: let the StubWrapper bean re-register in the spring environment, that is, call the http interface refreshstub provided by RefreshStubInstanceController:

• Check the console of the <font color="blue">get-service-addr-from-etcd</font> application, as shown in the red box as shown below. StubWrapper has been re-registered and the latest server information has been obtained from etcd:

Verify that the client can normally call the server service with the modified port

• Visit the web interface of the <font color="blue">get-service-addr-from-etcd</font> application again, as shown in the figure below, the gRPC call is successful:

• At this point, without modifying the configuration and restarting the service, the client can also adapt to the changes of the server. Of course, this article only provides a basic operation reference. The actual microservice environment will be more complicated. For example, the refreshstub interface may be affected by others. Service calls, so that the server can be updated more timely if changes are made, and the client itself can also be a gRPC service provider, so you must register yourself to etcd, and use the watch function of etcd to monitor the specified Whether the server is always alive, and how to load balance multiple instances of the same gRPC service, etc., these must be customized according to your actual situation;
• There is too much content in this article. It can be seen that for these officially unsupported microservice environments, it is very time-consuming and labor-intensive for us to make registration and discovery adaptations. If the design and selection can make our own decisions, we are more inclined to use ready-made registration centers. In the next article, let's try to use eureka to provide registration discovery service for gRPC;

You are not lonely, Xinchen is with you all the way

1. Java series
2. Spring series
3. Docker series
4. kubernetes series
5. Database + Middleware Series
6. DevOps series

Welcome to pay attention to the public account: programmer Xin Chen

Search "Programmer Xin Chen" on WeChat, I am Xin Chen, and I look forward to traveling the Java world with you...
https://github.com/zq2599/blog_demos