Technical practice: teach you to build gRPC service with Python

Abstract : gRPC is a high-performance, general-purpose open source RPC framework, which is designed by Google for mobile application development and based on the HTTP/2 protocol standard, based on the ProtoBuf serialization protocol, and supports many development languages.

This article is shared from the HUAWEI cloud community " build gRPC service with python", the original author: Jinggangshan_yangchun.

gRPC is a high-performance, general-purpose open source RPC framework, designed by Google for mobile application development and based on the HTTP/2 protocol standard, based on the ProtoBuf serialization protocol, and supports many development languages. The general structure diagram of a gRPC service is:

Figure 1 shows that grpc's services are cross-language, but they need to follow the same protocol (proto). Compared with REST services, an obvious advantage of gPRC is that it uses binary encoding, so it is faster than JSON/HTTP, has clear interface specifications and supports streaming, but its implementation is more important than rest services. A little more complicated, the following briefly introduces the steps to build a gRPC service.

1. Install the libraries needed by python

pip install grpcio
pip install grpcio-tools  
pip install protobuf

2. Define the interface of gRPC

The first step to create a gRPC service is to define the interface in the .proto file. proto is a protocol file. The communication interface between the client and the server is agreed through the proto file, and the corresponding language code files can be generated according to different languages. This protocol file mainly defines the service interface, as well as the data structure of the request parameters and corresponding results. For the specific proto syntax, please refer to the following link ( https://www.jianshu.com/p/da7ed5914088), about two-dimensional Arrays, dictionaries and other commonly used data types in python, see the link for the expression of proto syntax (https://blog.csdn.net/xiaoxiaojie521/article/details/106938519), the following is a simple example.

syntax = "proto3";
​
option cc_generic_services = true;
​
//定义服务接口
service GrpcService {
    rpc hello (HelloRequest) returns (HelloResponse) {}  //一个服务中可以定义多个接口，也就是多个函数功能
}
​
//请求的参数
message HelloRequest {
    string data = 1;   //数字1,2是参数的位置顺序，并不是对参数赋值
    Skill skill = 2;  //支持自定义的数据格式，非常灵活
};
​
//返回的对象
message HelloResponse {
    string result = 1;
    map<string, int32> map_result = 2; //支持map数据格式，类似dict
};
​
message Skill {
    string name = 1;
};

3. Use protoc and the corresponding plug-in to compile and generate the code of the corresponding language

python -m grpc_tools.protoc -I ./ --python_out=./ --grpc_python_out=. ./hello.proto

Use the compilation tool to convert the proto file into a py file, and run the above code directly in the current file directory.

1. -I specifies the directory where proto is located
2. -m specifies to generate py files through protoc
3. --python_out specifies the output path of the generated py file
4. proto file input by hello.proto

After executing the above command, two files, hello_pb2.py and hello_pb2_grpc.py, are generated.

4. Write grpc server code

#! /usr/bin/env python
# coding=utf8
​
import time
from concurrent import futures
​
import grpc
​
from gRPC_example import hello_pb2_grpc, hello_pb2
​
_ONE_DAY_IN_SECONDS = 60 * 60 * 24
​
​
class TestService(hello_pb2_grpc.GrpcServiceServicer):
    '''
    继承GrpcServiceServicer,实现hello方法
    '''
    def __init__(self):
        pass
​
    def hello(self, request, context):
        '''
        具体实现hello的方法，并按照pb的返回对象构造HelloResponse返回
        :param request:
        :param context:
        :return:
        '''
        result = request.data + request.skill.name + " this is gprc test service"
        list_result = {"12": 1232}
        return hello_pb2.HelloResponse(result=str(result),
                                       map_result=list_result)
​
def run():
    '''
    模拟服务启动
    :return:
    '''
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    hello_pb2_grpc.add_GrpcServiceServicer_to_server(TestService(),server)
    server.add_insecure_port('[::]:50052')
    server.start()
    print("start service...")
    try:
        while True:
            time.sleep(_ONE_DAY_IN_SECONDS)
    except KeyboardInterrupt:
        server.stop(0)
​
​
if __name__ == '__main__':
    run()

On the server side, the hello method needs to be implemented to meet the interface requirements of GrpcService in the proto file. The incoming parameter of the hello method is the HelloRequest defined in the proto file. The context is a reserved field. Don’t worry about it. The return parameter is in the proto file. The HelloResponse defined in the service startup code is standard, and the ip address and port number that provide the service can be modified according to needs.

5. Write the code for the gRPC client

#! /usr/bin/env python
# coding=utf8
​
import grpc
​
from gRPC_example import #! /usr/bin/env python
# coding=utf8
​
import grpc
​
from gRPC_example import hello_pb2_grpc, hello_pb2
​
​
def run():
    '''
    模拟请求服务方法信息
    :return:
    '''
    conn=grpc.insecure_channel('localhost:50052')
    client = hello_pb2_grpc.GrpcServiceStub(channel=conn)
    skill = hello_pb2.Skill(name="engineer")
    request = hello_pb2.HelloRequest(data="xiao gang", skill=skill)
    respnse = client.hello(request)
    print("received:",respnse.result)
​
​
if __name__ == '__main__':
    run()
​
​
def run():
    '''
    模拟请求服务方法信息
    :return:
    '''
    conn=grpc.insecure_channel('localhost:50052')
    client = hello_pb2_grpc.GrpcServiceStub(channel=conn)
    skill = hello_pb2.Skill(name="engineer")
    request = hello_pb2.HelloRequest(data="xiao gang", skill=skill)
    response = client.hello(request)
    print("received:",response.result)
​
​
if __name__ == '__main__':
    run()

The implementation of the client-side code is relatively simple, first define the access ip and port number, then define the HelloRequest data structure, and call hello remotely. It should be emphasized that the client and server must import the hello_pb2_grpc and hello_pb2 modules generated by compiling the same proto file. Even if the language used by the server and the client are different, this is also a manifestation of the consistency of grpc interface specifications.

6. Invoke the test

Start running the code of the server first, and then start the code of the client.

7. Summary of the use of gRPC

1. Define the interface document
2. Tool to generate server/client code
3. Service-side supplementary service code
4. After the client establishes a gRPC connection, use the automatically generated code to call the function
5. Compile and run

Click to follow and learn about Huawei Cloud's fresh technology for the first time~