Dewu Community

background

2021 is a year for our company to focus on the stability of production. In the early days of Dewu Community Service, a single application built by the PHP language supported millions of daily users. With the rapid development, performance and business are gradually unable to meet the future. In the first stage, the community and architecture team students provided technical selection solutions for php + yaf, Java + spring cloud, Go + grpc + K8S. Considering the service performance and migration cost, they finally chose Go + As the first choice for this project, grpc + K8S has established a milestone for the construction of community microservices.

With the development of business, the requirements for stability are getting higher and higher. In order to enhance the autonomy of business services, improve the stability and controllability of the cluster, and consider the lowest cost access method, while considering the community and transaction system (Dubbo Technology stack) is inextricably linked, and ultimately hopes to complete the easy integration of the two cluster systems, so the application layer framework Dubbo-go is selected to implement the registration and discovery of Golang services.

Golang microservice architecture, you may be more familiar with Go Micro and Go Kit (and Gizmo). Indeed, Go Micro’s community activity, Go Kit’s GitHub Star count is also over 18k, but there is no choice here, mainly Go Micro provides many functions, out of the box, but with limited flexibility; although go Kit is sought after, we are not restarting the Golang service. The application layer framework is too restrictive and the cost of code migration will be very high. Considering the above difficulties, I finally chose a Dubbo-go that is still in its growth stage.

Dubbo-go introduction

Dubbo-go is currently one of the hottest projects in the Dubbo multilingual ecology. It has joined the Apache Foundation along with the Dubbo ecology. So far, it has been used by many first- and second-tier Internet companies (Dingding, Ctrip, Graffiti, Open Classes, etc.). The community is highly active, responds quickly to the needs of developers, and has a faster version iteration speed that meets the needs of the development.

Figure 1*

The main Dubbo-go project is mainly based on Dubbo's hierarchical code design. The above picture shows the code layering of Dubbo-go, which is basically consistent with the existing layering of the Java version of Dubbo, so Dubbo-go also inherits some of Dubbo's Excellent features, such as clean code architecture, easy extension, and perfect service governance functions.

At present, Dubbo-go has implemented the common functions of Dubbo (such as responsible for balancing, clustering strategy, service multi-version multi-implementation, service multi-registration center multi-protocol publishing, generalized invocation, service downgrade circuit breaker, etc.), among which service registration discovery already supports zookeeper /etcd/consul/nacos mainstream registry. The detailed introduction will not be expanded here.

Scheme realization

According to the background of the existing project using gRPC as the service call, taking into account the degree of intrusion into the business code, and to be compatible with the original plan for normal use, the two sets of gRPC can be switched freely, so that the production environment can switch the Rpc governance framework Real-time and controllable, reduce production environment risks, so combined with Dubbo-go's own support for gRPC protocol to meet the above requirements. The registration center is selected as Nacos, which is consistent with the current existing middleware and meets the requirements of configuring some configuration items.

Figure II*

Here we have to think about two issues first. One is how the integration of Dubbo-go is compatible with the original gRPC solution, keeping the two solutions online to support production at the same time, and the second issue is how to achieve real-time switching between the two sets of gRPC.

compatibility

Before realizing this requirement, let's talk about the characteristics of gRPC. gRPC is an open source RPC framework of Google, which is designed for mobile and HTTP/2. The content exchange format uses ProtoBuf (Google Protocol Buffers). It has been open source for a long time and provides A flexible, efficient, and automatic serialization mechanism for structured data. Its function is similar to that of XML and Json, but it uses binary, (de)serialization is fast, and compression efficiency is high. The transmission protocol uses http2, and the performance is much better than http1.1.

According to the related features of gRPC introduced, it can be seen that gRPC has solved the two-layer problem of codec and transport. In conjunction with the figure, from the cluster layer upwards, there is no gRPC related part, as can be seen from Figure 1. To do gRPC-related adaptation, the protocol layer is the most suitable. We can extend a gRPCProtocol just like DubboProtocol. This gRPC protocol is roughly equivalent to an Adapter, which combines the implementation of the underlying gRPC with our own Dubbo- go together.

Figure 3*

Based on the above, Dubbo-go helped us solve the related integration of gRPC, which is equivalent to packaging the Dubbo-go governance layer on the basis of gRPC, and we start with the ProtoBuf modification of gRPC as the entry point. Dubbo-go is officially based on the Google protobuf extension The plug-in defines the protobuf custom logic code used by Dubbo-go gRPC, just complete the compatibility problem.

// HelloWorldServiceClientImpl is the client API for HelloWorldService service.
type HelloWorldServiceClientImpl struct {
    SayHello func(ctx context.Context, in *SayHelloReq, out *SayHelloResp) error
    //...
}

// service Reference
func (c *HelloWorldServiceClientImpl) Reference() string {
    return "helloWorldServiceImpl"
}

// GetDubboStub
func (c *HelloWorldServiceClientImpl) GetDubboStub(cc *grpc.ClientConn) HelloWorldServiceClient {
    return NewHelloWorldServiceClient(cc)
}

// Server interface
type HelloWorldServiceProviderBase struct {
    proxyImpl protocol.Invoker
}

// set invoker proxy
func (s *HelloWorldServiceProviderBase) SetProxyImpl(impl protocol.Invoker) {
    s.proxyImpl = impl
}

// get invoker proxy
func (s *HelloWorldServiceProviderBase) GetProxyImpl() protocol.Invoker {
    return s.proxyImpl
}

Real-time switch

Real-time switching, at first, to facilitate the two sets of different implementations of gRPC solutions in the stress test environment can be switched in real time for stress test data collection, and later, with a respect for production, the Rpc framework was switched when the production environment was first connected to Dubbo-go. Support free switching of services and methods, starting from stability, selectively switching to observe the service stability status.

The access to this requirement also starts with the modification of gRPC ProtoBuf as the entry point, and is implemented based on the Nacos configuration center. We encapsulate the original gRPC client call and Dubbo-go client call into a unified instantiation entry (Here referred to as ClinetDubbo), all methods of the client side add a specific implementation of inheriting ClinetDubbo (generated by the unified script of the protobuf extension plug-in), and the implementation content is roughly to obtain two sets of gRPC clients in ClinetDubbo. At this time, obtain the configuration center through Nacos configuration Which set of clients is opened depends on the judgment to realize the specific gRPC link.

Figure 4*

The gRPC conventional link on the left, and the Dubbo-go governance model link on the right

// ClientDubbo
type HelloWorldServiceClientDubbo struct {
    GrpcClient  HelloWorldServiceClient
    DubboClient *HelloWorldServiceClientImpl
    Open        bool
    Caller      string
}

// 具体的方法实现
func (c *HelloWorldServiceClientDubbo) SayHello(ctx context.Context, req *SayHelloReq, opts ...grpc.CallOption) (*SayHelloResp, error) {
    serverName := c.DubboClient.Reference()
    //获取 nacos配置源数据
    serverCfg := nacosCfg.GetServerCfg()
    if !c.Open {
        c.Open = serverCfg.AllOpen
    }
    cfg := serverCfg.ServiceCfg

    // 判断调用链路
    if !c.Open &&
        !cfg[serverName].Open &&
        (cfg[serverName].Consumers == nil || !cfg[serverName].Consumers[c.Caller]) &&
        !cfg[serverName].Method["SayHello"].Open &&
        (cfg[serverName].Method["SayHello"].Consumer == nil || !cfg[serverName].Method["SayHello"].Consumer[c.Caller]) {
        
        // 原gRPC链路
        return c.GrpcClient.SayHello(ctx, req, opts...)
    }

    // Dubbo-go治理链路
    out := new(SayHelloResp)
    err := c.DubboClient.SayHello(ctx, req, out)
    return out, err
}

Project integration

The following is an example of integrating the Dubbo-go framework based on the existing project structure:

provider

type HelloWorldService struct {
    *pb.UnimplementedHelloWorldServiceServer
    *pb.HelloWorldServiceClientImpl
    *pb.HelloWorldServiceProviderBase
}

func NewHelloWorldService() *HelloWorldService {
    return &HelloWorldService{
        HelloWorldServiceProviderBase: &pb.HelloWorldServiceProviderBase{},
    }
}

Add Dubbo-go extension part based on the original service provision to provide service registration.

consumer

//原有Grpc
var HelloWorldCli HelloWorldServiceClient  
//Dubbo-go
var HelloWorldProvider = &HelloWorldServiceClientImpl{} 

func GetHelloWorldCli() HelloWorldServiceClient {
    if HelloWorldCli == nil {
        HelloWorldCli = NewHelloWorldClient(grpc_client.GetGrpcClient(...))
    }
    return &HelloWorldServiceClientDubbo{
        GrpcClient:  HelloWorldCli,
        DubboClient: HelloWorldProvider,
        Caller:      dubboCfg.Caller,
        Open:        false,
    }
}

GetHelloWorldCli() simply encapsulates the client call. This method finally returns the method of the HelloWorldServiceClientDubbo structure. The client initiates the call and enters the specific method implemented by HelloWorldServiceClientDubbo, and executes the specific gRPC call link according to the configuration items.

Main

func main() {
    //provider
    config.SetProviderService(rpc.NewHelloWorldService(), ...)
    
    // 设置服务消费者
    config.SetConsumerService(api..., ....)
    
    // 加载dubbo
    config.Load()
}

The above is the overall idea and plan of community service integration Dubbo-go. We will find that the amount of code that needs to be changed in the existing project is very small, and there is no intrusion to the business side code.

summary

Dubbo-go integration enhances the governance capabilities in the gRPC call process of business services, increases the fault tolerance of service clusters based on cluster, and realizes the configurability of fault tolerance between application services; improves and unifies the original new and old architecture services of community services Full link monitoring and service indicator monitoring alarms; enhance the transparency and controllability of business partners to services in the cluster, and have more reference information on the overall link combing after encountering problems.

Based on the overall Dubbo ecosystem, it can easily support the intercommunication between Golang and Java Rpc, and achieve cross-language Rpc calls.

Figure 5*

At last

As a micro-service framework, Dubbo-go contains its own governance capabilities. How does this part of capabilities integrate with K8S?

K8S provides three-tier resources of pod/endpoint/service. You can monitor the events of the three-tier resources of pod/endpoint/service and make reasonable processing to achieve the purpose of service governance. There is no need to introduce additional components. By monitoring the k8s For the event of the most fine-grained resource pod, the pod list is obtained through k8s apiserver, but the service registration and service notification capabilities of etcd are used through apiserver. Others continue to use the service management capabilities of Dubbo-go. The model is simple and no additional modules are required, almost No need to make changes to Dubbo.

Text | Xiao Ke

Pay attention to Dewu Technology, and work hand in hand to the cloud of technology