Out-of-the-box microservice framework Go-zero (advanced article)

We briefly introduced Go-zero before, see "Go-zero: A Microservice Framework Out of the Box" for details. This time we start from the hands-on implementation of a user module of a Blog project, and describe the use of Go-zero in detail.

In particular, all the materials involved in this article have been uploaded to the Github warehouse "kougazhang/go-zero-demo", and interested students can download it by themselves.

Go-zero actual combat project: blog

This article takes the background of the blog website as an example, and focuses on how to use Go-zero to develop the user module of the blog.

The user module is a common module of the background management system, and its functions are also very familiar to everyone. Managing users involves front-end operations, and the persistence of user information is inseparable from the database. Therefore, the user module can be described as "Sparrow is small and has all internal organs". This article will introduce in detail how to use go-zero to complete the user module functions, such as: user login, add users, delete users, modify users, query users, etc. (please refer to the warehouse code for the complete Api documentation)

Blog Overall architecture

At the top is the api gateway layer. go-zero requires the api gateway layer to proxy the request, and forward the request to the corresponding rpc service for processing through gRPC. This piece of business logic that forwards the specific request to the corresponding rpc service needs to be written by hand.

Next is the rpc service layer. The user in the rpc service in the above figure is the module that will be shown to you next. Each rpc service can be deployed separately. After the service is started, the relevant information will be registered with ETCD, so that the api gateway layer can discover the address of the specific service through ECTD. The business logic of the rpc service to process specific requests needs to be written by hand.

Finally, the Model layer. The model layer encapsulates the relevant logic of database operations. If it is a query related operation, it will first query whether there is a corresponding cache in redis. Non-query operations will directly operate MySQL. goctl can generate ordinary CRDU codes from sql files. As mentioned above, currently this part of goctl only supports MySQL.

The following demonstrates how to use go-zero to develop a user module for the blog system.

api gateway layer

Write the blog.api file

• Generate blog.api file

Execute the command goctl api -o blog.api to create the blog.api file.

• api file function

For the detailed grammar of the api file, please refer to the document [ https://go-zero.dev/cn/api-grammar.html]. This article will talk about the role and basic grammar of the

The api file is used to generate the relevant code of the api gateway layer.

• api file syntax

The syntax of the api file is very similar to the Golang language. The type keyword is used to define the structure, and the service part is used to define the api service.

The structure defined by type is mainly used to declare the input parameters and return value of the request, namely request and response.

The api service defined by service declares routing, handler, request and response.

Please understand the specific content in conjunction with the default generated api file below.

// 声明版本，可忽略
syntax = "v1"

// 声明一些项目信息，可忽略
info(
   title: // TODO: add title
   desc: // TODO: add description
   author: "zhao.zhang"
   email: "[e-mail]"
)

// 重要配置
// request 是结构体的名称，可以使用 type 关键词定义新的结构体
type request {
   // TODO: add members here and delete this comment
   // 与 golang 语言一致，这里声明结构体的成员
}

// 语法同上，只是业务含义不同。response 一般用来声明返回值。
type response {
   // TODO: add members here and delete this comment
}

// 重要配置
// blog-api 是 service 的名称.
service blog-api {
   // GetUser 是处理请求的视图函数
   @handler GetUser // TODO: set handler name and delete this comment
   // get 声明了该请求使用 GET 方法
   // /users/id/:userId 是 url，:userId 表明是一个变量
   // request 就是上面 type 定义的那个 request, 是该请求的入参
   // response 就是上面 type 定义的那个 response, 是该请求的返回值。
   get /users/id/:userId(request) returns(response)

   @handler CreateUser // TODO: set handler name and delete this comment
   post /users/create(request)
}

• Write the blog.api file

In view of the length of the article, please refer to the repository on gitee for the complete blog.api file. The code generated below is generated according to the blog.api file on the warehouse.

api related code

• Generate relevant code

Execute the command goctl api go -api blog.api -dir. To generate api related code.

• Directory Introduction

├── blog.api # api 文件
├── blog.go # 程序入口文件
├── etc
│   └── blog-api.yaml # api 网关层配置文件
├── go.mod
├── go.sum
└── internal
    ├── config
    │   └── config.go # 配置文件
    ├── handler # 视图函数层, handler 文件与下面的 logic 文件一一对应
    │   ├── adduserhandler.go
    │   ├── deleteuserhandler.go
    │   ├── getusershandler.go
    │   ├── loginhandler.go
    │   ├── routes.go
    │   └── updateuserhandler.go
    ├── logic # 需要手动填充代码的地方
    │   ├── adduserlogic.go
    │   ├── deleteuserlogic.go
    │   ├── getuserslogic.go
    │   ├── loginlogic.go
    │   └── updateuserlogic.go
    ├── svc # 封装 rpc 对象的地方，后面会将
    │   └── servicecontext.go
    └── types # 把 blog.api 中定义的结构体映射为真正的 golang 结构体
        └── types.go

• The calling relationship between

Because the rpc service has not been involved at this time, the calling relationship of each module in the api is a very simple calling relationship between monolithic applications. routers.go is a route. According to the request Method and url, the request is distributed to the corresponding handler. The handler will call the corresponding logic. The logic file is where we inject the code logic.

summary

Api layer related commands:

• Execute the command goctl api -o blog.api to create the blog.api file.
• Execute the command goctl api go -api blog.api -dir. To generate api related code.
• Adding the parameter goctl can also generate api layer files in other languages, such as java, ts, etc. After trying it, I found it difficult to use, so I won't expand it.

rpc service

Write proto file

• Generate user.proto file

Use the command goctl rpc template -o user.proto to generate the user.proto file

• The role of the user.proto file

The role of user.proto is to generate relevant code for the rpc service.

The grammar of protobuf is beyond the scope of go-zero, so I won't expand it in detail here.

• Write user.proto file

In view of the length of the article, please refer to the repository on gitee for the complete user.proto file.

Generate rpc related code

• Generate user rpc service related code

Use the command goctl rpc proto -src user.proto -dir. To generate the user rpc service code.

summary

rpc service related commands:

• Use the command goctl rpc template -o user.proto to generate the user.proto file
• Use the command goctl rpc proto -src user.proto -dir. To generate the user rpc service code.

api service calls rpc service

A: Why is this section arranged after the rpc service?

Q: Because the main content of the logic part is to call the corresponding user rpc service, we must start the content of this part after the user rpc code has been generated.

A: Steps to call the rpc service at the api gateway layer

Q: If you don't know the structure of this part of the directory, you can refer to the previous "api gateway layer-api related code-directory introduction".

• Edit the configuration file etc/blog-api.yaml to configure the relevant information of the rpc service.

Name: blog-api
Host: 0.0.0.0
Port: 8888
# 新增 user rpc 服务.
User:
  Etcd:
#  Hosts 是 user.rpc 服务在 etcd 中的 value 值  
    Hosts:
      - localhost:2379
# Key 是 user.rpc 服务在 etcd 中的 key 值
    Key: user.rpc

• edit file config/config.go

type Config struct {
   rest.RestConf
   // 手动添加
   // RpcClientConf 是 rpc 客户端的配置, 用来解析在 blog-api.yaml 中的配置
   User zrpc.RpcClientConf
}

• edit the file internal/svc/servicecontext.go

type ServiceContext struct {
   Config config.Config
   // 手动添加
   // users.Users 是 user rpc 服务对外暴露的接口
   User   users.Users
}

func NewServiceContext(c config.Config) *ServiceContext {
   return &ServiceContext{
      Config: c,
      // 手动添加
      //  zrpc.MustNewClient(c.User) 创建了一个 grpc 客户端
      User:   users.NewUsers(zrpc.MustNewClient(c.User)),
   }
}

• edit each logic file, here take internal/logic/loginlogic.go as an example

func (l *LoginLogic) Login(req types.ReqUser) (*types.RespLogin, error) {
   // 调用 user rpc 的 login 方法
   resp, err := l.svcCtx.User.Login(l.ctx, &users.ReqUser{Username: req.Username, Password: req.Password})
   if err != nil {
      return nil, err
   }
   return &types.RespLogin{Token: resp.Token}, nil
}

model layer

Write sql file

Write the SQL file user.sql to create the table and execute it in the database.

CREATE TABLE `user`
(
  `id` int NOT NULL AUTO_INCREMENT COMMENT 'id',
  `username` varchar(255) NOT NULL UNIQUE COMMENT 'username',
  `password` varchar(255) NOT NULL COMMENT 'password',
  PRIMARY KEY(`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4

Generate model related code

Run the command goctl model mysql ddl -c -src user.sql -dir ., the CRDU code for operating the database will be generated.

The model directory at this time:

├── user.sql # 手写
├── usermodel.go # 自动生成
└── vars.go # 自动生成

model generated code attention points

• The model code uses splicing SQL statements, and there may be a risk of SQL injection.
• The code to generate CRUD is relatively basic, and we need to manually edit the usermodel.go file and stitch the SQL required by the business ourselves. See the FindByName method in usermdel.go.

rpc calls the code of the model layer

rpc directory structure

For the rpc service, we only need to pay attention to the annotated files or directories below.

├── etc
│   └── user.yaml # 配置文件，数据库的配置写在这
├── internal
│   ├── config
│   │   └── config.go # config.go 是 yaml 对应的结构体
│   ├── logic # 填充业务逻辑的地方
│   │   ├── createlogic.go
│   │   ├── deletelogic.go
│   │   ├── getalllogic.go
│   │   ├── getlogic.go
│   │   ├── loginlogic.go
│   │   └── updatelogic.go
│   ├── server
│   │   └── usersserver.go
│   └── svc
│       └── servicecontext.go # 封装各种依赖
├── user
│   └── user.pb.go
├── user.go
├── user.proto
└── users
    └── users.go

rpc Steps to call the model layer code

• Edit the etc/user.yaml file

Name: user.rpc
ListenOn: 127.0.0.1:8080
Etcd:
  Hosts:
  - 127.0.0.1:2379
  Key: user.rpc
# 以下为手动添加的配置
# mysql 配置
DataSource: root:1234@tcp(localhost:3306)/gozero
# 对应的表
Table: user
# redis 作为换存储
Cache:
  - Host: localhost:6379

• Edit the internal/config/config.go file

type Config struct {
// zrpc.RpcServerConf 表明继承了 rpc 服务端的配置
   zrpc.RpcServerConf
   DataSource string          // 手动代码
   Cache      cache.CacheConf // 手动代码
}

• edit internal/svc/servicecontext.go to encapsulate model and other dependencies.

type ServiceContext struct {
   Config config.Config
   Model  model.UserModel // 手动代码
}

func NewServiceContext(c config.Config) *ServiceContext {
   return &ServiceContext{
      Config: c,
      Model:  model.NewUserModel(sqlx.NewMysql(c.DataSource), c.Cache), // 手动代码
   }
}

• edit the corresponding logic file, here take internal/logic/loginlogic.go as an example:

func (l *LoginLogic) Login(in *user.ReqUser) (*user.RespLogin, error) {
   // todo: add your logic here and delete this line
   one, err := l.svcCtx.Model.FindByName(in.Username)
   if err != nil {
      return nil, errors.Wrapf(err, "FindUser %s", in.Username)
   }

   if one.Password != in.Password {
      return nil, fmt.Errorf("user or password is invalid")
   }

   token := GenTokenByHmac(one.Username, secretKey)
   return &user.RespLogin{Token: token}, nil
}

Microservice demo run

We are running the entire microservice in a stand-alone environment, and we need to start the following services:

• Redis
• Mysql
• Etcd
• go run blog.go -f etc/blog-api.yaml
• go run user.go -f etc/user.yaml

Among the above services, the rpc service should be started first, and then the gateway layer should be started.

In the warehouse, I encapsulated the start.sh and stop.sh scripts to run and stop microservices in a stand-alone environment, respectively.

Well, through the above six steps, the common functions of the blog user module are completed.

Finally, I will help you to emphasize the key points. In addition to the commonly used commands of goctl, which need to be mastered, the naming of go-zero files is also regular. The configuration file is a yaml file placed in the etc directory, and the corresponding structure of the yaml file is in interval/config/config.go. Dependency management is generally encapsulated in interval/svc/xxcontext.go. The place where we need to fill in the business logic is the files in the interval/logic directory.

Recommended reading

Practical Notes: The mental journey of configuring monitoring services for

go-zero: out-of-the-box