Go language: How to implement cross-platform services through RPC!

What is RPC service

RPC (Remote Procedure Call) is a method of calling between different nodes in a distributed system. It can be understood as calling the functions/methods provided by the application on the B server on the A server, and the RPC is provided by the client The server initiates, calls the method of the server to communicate, and then the server returns the result to the client.

There are two core points of RPC: communication protocol and serialization . Serialization and deserialization is a way to encode and decode transmitted data. Common encoding and decoding methods include JSON, Protobuf, etc.

RPC call flow:
(Picture from Baidu Encyclopedia)

• The client calls the client handle and passes the parameters to the client handle
• The client handle packs and encodes the parameters
• The client's local system sends information to the server
• The server system sends the information to the server handle
• Server handle parsing information (decoding)
• The server handle calls the real server program
• After the server is processed, the result is returned to the client in the same way

Network communication is generally through Socket communication.

Getting started with RPC in Go language

In the Go SDK, the net/rpc package is built in to implement RPC. The net/rpc package provides the ability to access server-side object methods through the network.

We show the call of RPC through an addition operation, server-side example:

server/math_server.go

package server

type MathService struct {
}

type Args struct {
    A, B int
}

func (m *MathService) Add(args Args, reply *int) error {
    *reply = args.A + args.B
    return nil
}

In the above code:

• A MathService is defined, which represents a remote service object;
• Args structure represents parameters;
• The Add method implements the addition function, and the result is returned through the replay pointer variable.

With this service object, it can be registered in the exposed service list to provide other clients for use. To register the RPC service object, you can use the RegisterName method, the code is as follows:

server_main.go

package main

import (
    "log"
    "net"
    "net/rpc"
    "rpctest/server"
)

func main() {
    rpc.RegisterName("MathService", new(server.MathService))
    l, err := net.Listen("tcp", ":8088") //注意 “：” 不要忘了写
    if err != nil {
        log.Fatal("listen error", err)
    }
    rpc.Accept(l)
}

In the above code:

• A service object is registered through the RegisterName function, which has two parameters:

  • Service name, used by the client when calling (MathService)
  • The service object, which is the structure of MathService
• Establish a TCP link through the net.Listen function and listen on port 8088;
• Finally, the RPC service MathService is provided on the TCP link through the rpc.Accept function. In this way, the client can see the MathService service and its Add method.

In the RPC framework provided by net/rpc, if you want to register an object as an RPC service and allow clients to remotely access it, the method of the object (type) must meet the following conditions:

• The type of method is exportable (public);
• The method itself is also exportable;
• The method must have 2 parameters, and the parameter types are exportable or built-in;
• The method must return an error type.

The summary is:

func (t *T) MethodName(argType T1, replyType *T2) error

Here T1 and T2 can be serialized by encoding/gob.

• The first parameter argType is provided by the caller (client);
• The second parameter replyType is the result returned to the caller and must be a pointer type.

We have completed the RPC service, and then continue to complete the client call:

client_main.go

package main

import (
    "fmt"
    "log"
    "net/rpc"
    "rpctest/server"
)

func main() {
    client, err := rpc.Dial("tcp", "localhost:8088")
    if err != nil {
        log.Fatal("dialing")
    }
    args := server.Args{A: 1, B: 2}
    var reply int
    err = client.Call("MathService.Add", args, &reply)
    if err != nil {
        log.Fatal("MathService.Add error", err)
    }
    fmt.Printf("MathService.Add: %d+%d=%d", args.A, args.B, reply)
}

In the above code:

• Establish a TCP link through the rpc.Dial function. Note that the IP and port here must be consistent with those provided by the RPC service;
• Prepare the parameters required by the remote method, here are the args and reply in the example;
• Call the remote RPC service through the Call method;

• The Call method has 3 parameters and their functions:

  • The name of the called remote method, here is MathService.Add, the part before the dot is the name of the registered service, and the part after the dot is the method of the service;
  • The parameters provided by the client to call the remote method are args in the example;
  • In order to receive the result returned by the remote method, it must be a pointer, here is &replay in the example.

We have completed the server and client, the overall directory structure:

Then start running:

1. Run the server program first to provide RPC service:

go run server_main.go

2. Run the client program again and call RPC:

go run client_main.go

operation result:

MathService.Add: 1+2=3

See the above result, indicating that the RPC call was successful! ✿✿ヽ(°▽°)ノ✿

HTTP-based RPC

RPC can be called not only through the TCP protocol, but also through the HTTP protocol, or through the built-in net/rpc package. We changed the door-to-door code to the HTTP protocol call. The server code:

server_main.go

func main() {
   rpc.RegisterName("MathService", new(server.MathService))
   rpc.HandleHTTP()//新增的
   l, err := net.Listen("tcp", ":8088")
   if err != nil {
      log.Fatal("listen error:", err)
   }
   http.Serve(l, nil)//换成http的服务
}

Part of the client code modification:

client_main.go

func main()  {
   client, err := rpc.DialHTTP("tcp",  "localhost:8088")
   //此处省略其他没有修改的代码
}

After the modification is completed, we run the server and the client respectively, and the result is the same as when the tcp connection is made.

Debug URL

The RPC of the HTTP protocol provided by the net/rpc package also has a debugging URL. After running the server code, visit http://localhost:8088/debug/rpc Enter to see the server The registered RPC services and the methods of each service are shown in the figure:

As shown in the figure, you can see the registered RPC service, method signature, and the number of times it has been called.

JSON RPC cross-platform communication

The RPC service implemented above is based on gob coding. This coding is more difficult when making cross-language calls. However, the implementer and caller of the RPC service may often be in different programming languages, so the RPC service we implement must support Multi-language call.

JSON RPC over TCP

To implement cross-language RPC services, the core is to choose a common encoding, such as commonly used JSON. In the Go language, you only need to use the net/rpc/jsonrpc package to implement a JSON RPC service.

I transformed the above example into an RPC service that supports JSON. The server code is as follows:

server_main.go

func main() {
    rpc.RegisterName("MathService", new(server.MathService))
    l, err := net.Listen("tcp", ":8088")
    if err != nil {
        log.Fatal("listen error:", err)
    }
    for {
        conn, err := l.Accept()
        if err != nil {
            log.Println("jsonrpc.Serve: accept:", err.Error())
            return
        }
        //json rpc
        go jsonrpc.ServeConn(conn)
    }
}

In the above code, compared to the gob-encoded RPC service, the JSON RPC service passes the link to the jsonrpc.ServeConn , which achieves the purpose of making RPC calls based on JSON.

The modified part of the client code of JSON RPC is as follows:

client_main.go

func main()  {
   client, err := jsonrpc.Dial("tcp",  "localhost:8088")
   //省略了其他没有修改的代码
}

Just change the Dial method of establishing the link to the one in the jsonrpc package.

JSON RPC over HTTP

The built-in jsonrpc of the Go language does not implement HTTP-based transmission. Here, refer to the implementation of gob-encoded HTTP RPC to implement HTTP-based JSON RPC services.

server_main.go

func main() {
   rpc.RegisterName("MathService", new(server.MathService))
   //注册一个path，用于提供基于http的json rpc服务
   http.HandleFunc(rpc.DefaultRPCPath, func(rw http.ResponseWriter, r *http.Request) {
      conn, _, err := rw.(http.Hijacker).Hijack()
      if err != nil {
         log.Print("rpc hijacking ", r.RemoteAddr, ": ", err.Error())
         return
      }
      var connected = "200 Connected to JSON RPC"
      io.WriteString(conn, "HTTP/1.0 "+connected+"\n\n")
      jsonrpc.ServeConn(conn)
   })
   l, err := net.Listen("tcp", ":8088")
   if err != nil {
      log.Fatal("listen error:", err)
   }
   http.Serve(l, nil)//换成http的服务
}

The above code implements the core based on the HTTP protocol, uses http.HandleFunc to register a path, and provides external HTTP-based JSON RPC services. In the implementation of this HTTP service, the link is hijacked by the Hijack method, and then transferred to jsonrpc for processing, thus realizing the JSON RPC service based on the HTTP protocol.

Client calling code:

func main()  {
     client, err := DialHTTP("tcp",  "localhost:8088")
     if err != nil {
        log.Fatal("dialing:", err)
     }
     args := server.Args{A:1,B:2}
     var reply int
     err = client.Call("MathService.Add", args, &reply)
     if err != nil {
        log.Fatal("MathService.Add error:", err)
     }
     fmt.Printf("MathService.Add: %d+%d=%d", args.A, args.B, reply)
  }
  // DialHTTP connects to an HTTP RPC server at the specified network address
  // listening on the default HTTP RPC path.
  func DialHTTP(network, address string) (*rpc.Client, error) {
     return DialHTTPPath(network, address, rpc.DefaultRPCPath)
  }
  // DialHTTPPath connects to an HTTP RPC server
  // at the specified network address and path.
  func DialHTTPPath(network, address, path string) (*rpc.Client, error) {
     var err error
     conn, err := net.Dial(network, address)
     if err != nil {
        return nil, err
     }
     io.WriteString(conn, "GET "+path+" HTTP/1.0\n\n")
     // Require successful HTTP response
     // before switching to RPC protocol.
     resp, err := http.ReadResponse(bufio.NewReader(conn), &http.Request{Method: "GET"})
     connected := "200 Connected to JSON RPC"
     if err == nil && resp.Status == connected {
        return jsonrpc.NewClient(conn), nil
     }
     if err == nil {
        err = errors.New("unexpected HTTP response: " + resp.Status)
     }
     conn.Close()
     return nil, &net.OpError{
        Op:   "dial-http",
        Net:  network + " " + address,
        Addr: nil,
        Err:  err,
     }
  }

The core of the above code is to call the remote HTTP JSON RPC service by sending an HTTP request through the established TCP link. The HTTP GET method is used here.
Run the server and client separately, you can see the correct HTTP JSON RPC call result.

Above we use the RPC framework that comes with the Go language, but in actual development, it is not used much. The more commonly used is Google's gRPC framework, which is serialized through Protobuf and is based on the binary of the HTTP/2 protocol. Transmission, and supports many programming languages, and the efficiency is relatively high. We learn through RPC, and then learn gRPC is very easy to get started.