gRPC learning six: gRPC-Gateway integrated swagger

Welcome to my GitHub

https://github.com/zq2599/blog_demos

Content: Classification and summary of all original articles and supporting source code, involving Java, Docker, Kubernetes, DevOPS, etc.;

Links to gRPC learning series articles

1. deploy and set up GO
2. GO's gRPC development environment preparation
3. first trial GO version gRPC development
4. actual combat four types of service methods
5. gRPC-Gateway actual combat
6. gRPC-Gateway integrated swagger

Overview of this article

• This article is the sixth chapter of the "gRPC Learning" series. In the previous article, we actually battled gRPC-Gateway and exposed the gRPC service in RESTful form. At that time, due to space limitations, the swagger integration was not completed. This article will complete this work: blue">Develop gRPC service, provide gRPC-Gateway for it, and provide online swagger service</font>;
• This article is composed of the following chapters, which is also the normal process of gRPC-Gateway integrating swagger:
• Preview key knowledge points in advance;
• New project folder;
• Install the necessary go packages;
• Write the proto file to make swagger support http (the default is https);
• Generate the go source code required by gRPC and gRPC-Gateway;
• Generate the json file required by swagger;
• Download the source code of swagger-ui to generate go source code;
• Write gRPC server code;
• Write the code of the gRPC-Gateway server;
• verification;
• Note that all the operations in this article do not use the <font color="blue">root</font> account, but the <font color="red">golang</font> account created by before

Source download

• The source code in this actual combat can be downloaded on GitHub. The address and link information are shown in the following table ( https://github.com/zq2599/blog_demos):

name	link	Remarks
Project homepage	https://github.com/zq2599/blog_demos	The project's homepage on GitHub
git warehouse address (https)	https://github.com/zq2599/blog_demos.git	The warehouse address of the source code of the project, https protocol
git warehouse address (ssh)	[e-mail]:zq2599/blog_demos.git	The warehouse address of the source code of the project, ssh protocol

• There are multiple folders in this git project. The application of this chapter is under the <font color="blue">go-source</font> folder, as shown in the red box below:

• There are multiple subfolders in <font color="blue">go-source</font>. The source code of this article is in <font color="red">swaggerdemo</font>, as shown in the red box as shown below:

Preview key knowledge points in advance

The process of integrating swagger services in gRPC-Gateway is not simple. Let’s take a look at the key points in advance and know the following:

1. In order to simplify the actual combat process, the service exposed by gRPC-Gateway does not use <font color="blue">https</font>, but <font color="red">http</font>, but the services provided by swagger-ui The calling service is https, so you need to specify swagger in the proto file to call the service via http. When specifying it, the file <font color="red">protoc-gen-swagger/options/annotations.proto</font> , So you need to find the package corresponding to this file and put it in a suitable location;
2. swaggerdemo.swagger.json: This is the json file used by swagger-ui. Based on this file, swagger can correctly display the service and parameter definitions exposed by gRPC-Gateway. You can initiate a request on the page. This file is provided by the plugin <font color="blue">protoc-gen-swagger</font> generated, the plug-in was installed "gRPC-Gateway Actual Combat"
3. Integrate the swagger-ui code in the gRPC-Gateway code: The swagger-ui code consists of multiple png, html, and js files, which need to be converted into using the tool <font color="blue">go-bindata</font> Go source code and put it in a suitable location, the process is as follows:

4. To expose the swaggerdemo.swagger.json file through the web, the tool go-bindata-assetfs is required;
5. The way of using swagger: After opening the swagger-ui page, enter swaggerdemo.swagger.json into the swagger-ui page, and after it is parsed, the corresponding online interface service is generated;

Prerequisites

• This article is the "gRPC-Gateway Actual Combat" , please refer to the previous article to prepare the gRPC-Gateway environment;

Show file structure in advance

• This actual combat involves multiple files. First, the final file content will be displayed for your reference during the development process. All the content is in <font color="blue">$GOPATH/src/swaggerdemo</ font> directory:

[golang@centos7 src]$ tree swaggerdemo/
swaggerdemo/
├── gateway
│   └── gateway.go
├── pkg
│   └── ui
│       └── data
│           └── swagger
│               └── datafile.go
├── server
│   └── server.go
├── swaggerdemo.pb.go
├── swaggerdemo.pb.gw.go
├── swaggerdemo.proto
├── swaggerdemo.swagger.json
└── third_party
    └── swagger-ui
        ├── favicon-16x16.png
        ├── favicon-32x32.png
        ├── index.html
        ├── oauth2-redirect.html
        ├── swagger-ui-bundle.js
        ├── swagger-ui-bundle.js.map
        ├── swagger-ui.css
        ├── swagger-ui.css.map
        ├── swagger-ui-es-bundle-core.js
        ├── swagger-ui-es-bundle-core.js.map
        ├── swagger-ui-es-bundle.js
        ├── swagger-ui-es-bundle.js.map
        ├── swagger-ui.js
        ├── swagger-ui.js.map
        ├── swagger-ui-standalone-preset.js
        └── swagger-ui-standalone-preset.js.map

8 directories, 23 files

New project folder

• This actual combat has nothing to do with the code of the previous articles, but a brand new project. Please create a new name under <font color="blue">$GOPATH/src</font> as <font color="red"> swaggerdemo</font> folder;

Install the necessary go packages

1. Install git, execute the command <font color="blue">sudo yum install -y git unzip</font>
2. Several packages will be used in the project, and then install them one by one;
3. <font color="blue">go-bindata</font> is used to convert the source code of swagger-ui into GO code:

go get -u github.com/jteeuwen/go-bindata/...

4. <font color="blue">go-bindata-assetfs</font> provides file services externally after the application is started, so that swagger json files can be accessed through the web:

go get -u github.com/elazarl/go-bindata-assetfs/...

5. <font color="blue">glog</font> is a commonly used logging tool:

go get -u github.com/golang/glog

Write proto file

• Enter the directory <font color="blue">$GOPATH/src/swaggerdemo</font>, create a new <font color="red">swaggerdemo.proto</font>, the content is as follows, there are a few places to pay attention to later Will explain:

// 协议类型
syntax = "proto3";

// 包名
package swaggerdemo;

import "google/api/annotations.proto";
import "protoc-gen-swagger/options/annotations.proto";

// 定义swagger内容
option (grpc.gateway.protoc_gen_swagger.options.openapiv2_swagger) = {
  info: {
        title: "grpc gateway helloworld sample";
        version: "1.0";    
  };
  schemes: HTTP;
};

// 定义的服务名
service Greeter {
  // 具体的远程服务方法
  rpc SayHello (HelloRequest) returns (HelloReply) {
    option (google.api.http) = {
      post: "/helloworld"
      body: "*"
    };
  }
}

// SayHello方法的入参，只有一个字符串字段
message HelloRequest {
  string name = 1;
}

// SayHello方法的返回值，只有一个字符串字段
message HelloReply {
  string message = 1;
}

• The file swaggerdemo.proto and "gRPC-Gateway Actual Combat" The proto file in the article is mostly the same. The difference is that the configuration of swagger has been added. The function of this configuration is to allow swagger to configure remote calls to http, if not For these configurations, the default remote call of swagger is https. The gRPC-Gateway in this article provides http services, so these configurations should be added in the content of the above <font color="blue">swaggerdemo.proto</font> , The specific configuration has the following two places:
• Use the <font color="blue">import</font> keyword to import <font color="red">protoc-gen-swagger/options/annotations.proto</font>
• The following paragraph is the configuration of swagger, the focus is on <font color="blue">schemes</font>, there is only <font color="red">HTTP</font>:

option (grpc.gateway.protoc_gen_swagger.options.openapiv2_swagger) = {
  info: {
        title: "grpc gateway helloworld sample";
        version: "1.0";    
  };
  schemes: HTTP;
};

• Also put the <font color="red">protoc-gen-swagger/options/annotations.proto</font> mentioned in <font color="blue">swaggerdemo.proto</font> in the appropriate In order to find the annotations.proto file when using swaggerdemo.proto, execute the following command:

cd $GOPATH/src
cp -r ./github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger ./

• The <font color="blue">protoc-gen-swagger</font> folder in the above command is downloaded in the previous operation;

Generate the go source code required by gRPC and gRPC-Gateway

• Generate the go source code required by gRPC and gRPC-Gateway. This operation has been done before, here use <font color="blue">swaggerdemo.proto</font> to do it again, first enter the directory <font color=" blue">$GOPATH/src/swaggerdemo</font>
• Execute the following command to generate the source code required by gRPC:

protoc -I. \
-I$GOPATH/src \
-I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis \
--go_out=plugins=grpc:. \
swaggerdemo.proto

• Execute the following command to generate the source code required by gRPC-Gateway:

  protoc -I. \
  -I$GOPATH/src \
  -I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis \
  --grpc-gateway_out=logtostderr=true:. \
  swaggerdemo.proto

  Generate the json file required by swagger

• Still in the directory <font color="blue">$GOPATH/src/swaggerdemo</font>, execute the following command to generate the json required by swagger:

protoc -I. \
-I$GOPATH/src \
-I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis \
--swagger_out=logtostderr=true:. \
swaggerdemo.proto

• At this time, the following three files are added under the <font color="blue">$GOPATH/src/swaggerdemo</font> directory:
• swaggerdemo.pb.go: Go files required by gRPC
• swaggerdemo.pb.gw.go: Go files required by gRPC-Gateway
• swaggerdemo.swagger.json: The json file used by swagger-ui. According to this file, the page displayed by swagger will have service and parameter definitions exposed by gRPC-Gateway, and requests can be initiated on the page

Generate swagger-ui go file

• To provide swagger web pages in the service, you need to convert the source code of swagger-ui into a go file. The steps are as follows:
• The next command will download the source code of swagger-ui from Github. This file should have been downloaded from the official swagger, but after trying many times here, I found that the downloaded zip package is very damaged and cannot be decompressed, so I put this file on my own Github, the following operations are also downloaded from my own Github, but in fact this file is no different from the official swagger;
• Enter the directory <font color="blue">$GOPATH/src/swaggerdemo</font>, execute the following command to download the swagger-ui source code, and put it in the specified location:

wget https://raw.githubusercontent.com/zq2599/blog_download_files/master/files/swagger-ui.zip -O swagger-ui.zip \
&& unzip swagger-ui.zip \
&& mkdir -p $GOPATH/src/swaggerdemo/third_party/ \
&& mv ./swagger-ui-3.38.0/dist $GOPATH/src/swaggerdemo/third_party/ \
&& mv $GOPATH/src/swaggerdemo/third_party/dist $GOPATH/src/swaggerdemo/third_party/swagger-ui \
&& rm -f ./swagger-ui.zip \
&& rm -rf ./swagger-ui-3.38.0

3. Execute the following command to create a new folder, which is used to store the go source code of swagger-ui generated later:

mkdir -p $GOPATH/src/swaggerdemo/pkg/ui/data/swagger

4. Execute the following command to convert the swagger-ui source code into a datafile.go file:

cd $GOPATH/src/swaggerdemo/
go-bindata --nocompress -pkg swagger -o pkg/ui/data/swagger/datafile.go third_party/swagger-ui/...

5. At this time, the file <font color="red">datafile.go</font> is generated in the <font color="blue">$GOPATH/src/swaggerdemo/pkg/ui/data/swagger</font> directory

• All documents and materials have been prepared and start coding;

• Write gRPC server code

• Follow the configuration of <font color="blue">swaggerdemo.proto</font> to create a new gRPC service, the steps are as follows:
• Create a new folder <font color="blue">$GOPATH/src/swaggerdemo/server</font>;
• In the newly created server folder, add a new file server.go with the following content, which is just a normal gRPC service:

package main

import (
    "context"
    "log"
    "net"

    "google.golang.org/grpc"
    pb "swaggerdemo"
)

const (
    port = ":50051"
)

// 定义结构体，在调用注册api的时候作为入参，
// 该结构体会带上SayHello方法，里面是业务代码
// 这样远程调用时就执行了业务代码了
type server struct {
    // pb.go中自动生成的，是个空结构体
    pb.UnimplementedGreeterServer
}

// 业务代码在此写，客户端远程调用SayHello时，
// 会执行这里的代码
func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
    // 打印请求参数
    log.Printf("Received: %v", in.GetName())
    // 实例化结构体HelloReply，作为返回值
    return &pb.HelloReply{Message: "Hello " + in.GetName()}, nil
}

func main() {
    // 要监听的协议和端口
    lis, err := net.Listen("tcp", port)
    if err != nil {
        log.Fatalf("failed to listen: %v", err)
    }

    // 实例化gRPC server结构体
    s := grpc.NewServer()

    // 服务注册
    pb.RegisterGreeterServer(s, &server{})

    log.Println("开始监听，等待远程调用...")

    if err := s.Serve(lis); err != nil {
        log.Fatalf("failed to serve: %v", err)
    }
}

• The above is the code of the gRPC service, which is similar to the previous articles, so I won’t repeat it;

  Write gRPC-Gateway server code

• Start writing gRPC-Gateway server code, which is the focus of this article. In addition to providing the same gRPC-Gateway service as the previous article, it also provides swagger's json file service and swagger's ui service;
• Create a new folder <font color="blue">$GOPATH/src/swaggerdemo/gateway</font>;
• A new file <font color="blue">gateway.go</font> is added under the newly created gateway folder. The content is as follows. Several points to note will be explained later:

package main

import (
    "github.com/elazarl/go-bindata-assetfs"
    "log"
    "net/http"
    "path"
    "strings"

    "github.com/golang/glog"
    "github.com/grpc-ecosystem/grpc-gateway/runtime"
    "golang.org/x/net/context"
    "google.golang.org/grpc"
    swagger "swaggerdemo/pkg/ui/data/swagger"
    gw "swaggerdemo"
)

func run() error {
    ctx := context.Background()
    ctx, cancel := context.WithCancel(ctx)
    defer cancel()

    gwmux, err := newGateway(ctx)
    if err != nil {
        panic(err)
    }

    mux := http.NewServeMux()
    mux.Handle("/", gwmux)
    mux.HandleFunc("/swagger/", serveSwaggerFile)
    serveSwaggerUI(mux)

    log.Println("grpc-gateway listen on localhost:9090")
    return http.ListenAndServe(":9090", mux)
}

func newGateway(ctx context.Context) (http.Handler, error) {
    opts := []grpc.DialOption{grpc.WithInsecure()}

    gwmux := runtime.NewServeMux()
    if err := gw.RegisterGreeterHandlerFromEndpoint(ctx, gwmux, ":50051", opts); err != nil {
        return nil, err
    }

    return gwmux, nil
}

func serveSwaggerFile(w http.ResponseWriter, r *http.Request) {
    log.Println("start serveSwaggerFile")        


    if !strings.HasSuffix(r.URL.Path, "swagger.json") {
        log.Printf("Not Found: %s", r.URL.Path)
        http.NotFound(w, r)
        return
    }

    p := strings.TrimPrefix(r.URL.Path, "/swagger/")
    p = path.Join("../", p)

    log.Printf("Serving swagger-file: %s", p)

    http.ServeFile(w, r, p)
}

func serveSwaggerUI(mux *http.ServeMux) {
    fileServer := http.FileServer(&assetfs.AssetFS{
        Asset:    swagger.Asset,
        AssetDir: swagger.AssetDir,
        Prefix:   "third_party/swagger-ui",
    })
    prefix := "/swagger-ui/"
    mux.Handle(prefix, http.StripPrefix(prefix, fileServer))
}

func main() {
    defer glog.Flush()

    if err := run(); err != nil {
        glog.Fatal(err)
    }
}

• For this <font color="blue">gateway.go</font> file, the following points need to be paid attention to:
• The function of forwarding external RESTful requests to server.go is encapsulated in the <font color="blue">newGateway</font> method;
• If the request URL contains <font color="blue">/swagger</font>, it will be handled by the <font color="blue">serveSwaggerFile</font> method. The logic here is to convert the file <font color= "blue">$GOPATH/src/swaggerdemo/swaggerdemo.swagger.json</font> is returned to the requester;
• Focus on the <font color="blue">serveSwaggerUI</font> method. After the method is processed, if the request URL contains <font color="blue">/swagger-ui</font>, it will be handed to The previously generated <font color="red">datafile.go</font> is processed, that is, the swagger-ui page is opened;
• At this point, the development work has been completed and verification can begin;

verification

1. Enter the directory <font color="blue">$GOPATH/src/swaggerdemo/server</font> and execute <font color="red">go run server.go</font> to start the gRPC service;
2. Enter the directory <font color="blue">$GOPATH/src/swaggerdemo/gateway</font> and execute <font color="red">go run gateway.go</font> to start the gRPC-Gateway service;
3. Ensure that the firewall of the machine where the service is located has been closed;
4. The IP address of my server is <font color="blue"> http://192.168.133.204/ </font>, so the browser visits: http://192.168.133.204:9090/swagger/swaggerdemo. swagger.json , you can see the content of swagger.json, as shown below:

5. Visit the swagger-ui page, the address is: http://192.168.133.204:9090/swagger-ui/ , as shown in the figure below, it can be seen that swagger-ui is functioning normally:

6. At this time, the swagger-ui page does not display the interface content of gRPC-Gateway, you need to add <font color="blue"> http://192.168.133.204:9090/swagger/swaggerdemo.swagger.json </font> Fill in the red box 1 in the figure below, and then click the button in the red box 2 to display normally. The red box 3 is the service exposed by gRPC-Gateway:

7. Click the <font color="blue">Try it out</font> button in the red box below to initiate a request to the background on the page:

8. As shown in the figure below, modify the request parameters in red box 1 and click the button in red box 2 to initiate a request:

9. As shown in the figure below, the red box 1 is the request address, and it can be seen that it is the <font color="blue">http</font> request, which proves that our previous settings in the proto file have taken effect, and the red box 2 is the received return content , Obviously this content comes from server.go:

• At this point, the operation of gRPC-Gateway integration swagger is completed. It can be seen that this is a series of tedious operations. I hope this article can provide you with some references to help you integrate swagger smoothly;

You are not alone, Xinchen and original are with you all the way

1. Java series
2. Spring series
3. Docker series
4. kubernetes series
5. database + middleware series
6. DevOps series

Welcome to pay attention to the public account: programmer Xin Chen

Search "Programmer Xin Chen" on WeChat, I am Xin Chen, and I look forward to traveling the Java world with you...
https://github.com/zq2599/blog_demos