go-zero microservices combat series (four, CRUD warm-up)

In the previous article, we have completed the construction of the entire project, and the service has been able to run locally. Next, we should start writing business logic code, but simply writing business logic code is relatively boring. I will continue to add the code of business logic to the lerbon project, and I will also add comments to the key parts.

So in this article, I mainly want to share with you the basic configuration of the service and a few typical code examples.

log definition

The logx package of go-zero provides a log function, which can output logs to stdout without any configuration by default. When we request the /v1/order/list interface, the output log is as follows. The default is json format output, including timestamp, basic information of http request, interface time consumption, and span and trace information of link tracking.

 {"@timestamp":"2022-06-11T08:23:36.342+08:00","caller":"handler/loghandler.go:197","content":"[HTTP] 200 - GET /v1/order/list?uid=123 - 127.0.0.1:59998 - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36","duration":"21.2ms","level":"info","span":"23c4deaa3432fd03","trace":"091ffcb0eafe7818b294e4d8122cf8a1"}

After the program starts, the framework will output the statistical log with the level of stat by default, which is used to output the current resource usage, mainly cpu and memory, as follows:

 {"@timestamp":"2022-06-11T08:34:58.402+08:00","caller":"stat/usage.go:61","content":"CPU: 0m, MEMORY: Alloc=3.3Mi, TotalAlloc=7.0Mi, Sys=16.3Mi, NumGC=8","level":"stat"}

When we don't need this type of log, we can turn off the output of this type of log in the following ways:

 logx.DisableStat()

Sometimes we only need to record the error log, you can cancel the output of the level info level log by setting the log level:

 logx.SetLevel(logx.ErrorLevel)

The fields of the log output can be extended, and the uid field is added to record the uid of the requested user. The log print content is as follows:

 logx.Infow("order list", logx.Field("uid",req.UID))

 {"@timestamp":"2022-06-11T08:53:50.609+08:00","caller":"logic/orderlistlogic.go:31","content":"order list","level":"info","uid":123}

We can also extend other third-party log libraries and set them through logx.SetWriter

 writer := logrusx.NewLogrusWriter(func(logger *logrus.Logger) {
    logger.SetFormatter(&logrus.JSONFormatter{})
})
logx.SetWriter(writer)

At the same time, logx also provides a wealth of configuration, you can configure the log output mode, time format, output path, whether to compress, log storage time, etc.

 type LogConf struct {
    ServiceName         string `json:",optional"`
    Mode                string `json:",default=console,options=[console,file,volume]"`
    Encoding            string `json:",default=json,options=[json,plain]"`
    TimeFormat          string `json:",optional"`
    Path                string `json:",default=logs"`
    Level               string `json:",default=info,options=[info,error,severe]"`
    Compress            bool   `json:",optional"`
    KeepDays            int    `json:",optional"`
    StackCooldownMillis int    `json:",default=100"`
}

It can be seen that the log function provided by logx is still very rich, and it supports various customization methods. The log is a very important dependency for us to troubleshoot online problems, and we will also make various alarms based on the log, so here we will first introduce some log usage.

service dependency

The BFF service will depend on multiple RPC services. By default, if the dependent RPC service is not started, the BFF service will start abnormally. The core service of the entire mall system, BFF is strongly dependent on order.rpc. In the case of strong dependence, if the dependent service is abnormal, the dependent service cannot be started normally.

 {"@timestamp":"2022-06-11T10:21:56.711+08:00","caller":"internal/discovbuilder.go:34","content":"bad resolver state","level":"error"}
2022/06/11 10:21:59 rpc dial: discov://127.0.0.1:2379/order.rpc, error: context deadline exceeded, make sure rpc service "order.rpc" is already started
exit status 1

Look at the following scenario, BFF depends on reply.rpc, because reply.rpc is abnormal, BFF cannot start normally, because reply.rpc is not the core dependency of the mall system, even if reply.rpc hangs, it will not affect the core process of the mall, so For BFF, reply.rpc is a weak dependency, and in the case of weak dependency, it should not affect the startup of the relying party.

 {"@timestamp":"2022-06-11T11:26:51.711+08:00","caller":"internal/discovbuilder.go:34","content":"bad resolver state","level":"error"}
2022/06/11 11:26:54 rpc dial: discov://127.0.0.1:2379/reply.rpc, error: context deadline exceeded, make sure rpc service "reply.rpc" is already started
exit status 1

The configuration of weak dependencies is provided in go-zero. After configuration, BFF can be started normally. You can see that order.rpc and product.rpc are both strong dependencies, while reply.rpc is configured with NonBlock: true as weak dependencies.

 OrderRPC:
    Etcd:
        Hosts:
          - 127.0.0.1:2379
        Key: order.rpc
ProductRPC:
  Etcd:
    Hosts:
      - 127.0.0.1:2379
    Key: product.rpc
ReplyRPC:
  Etcd:
    Hosts:
      - 127.0.0.1:2379
    Key: reply.rpc
  NonBlock: true

parallel call

In a high-concurrency system, the time-consuming of the interface is a point we are very concerned about. The fast response of the interface can improve the user experience. Long-time waiting will make the user experience poor, and the user will slowly leave us. Here we introduce a simple but very practical method to improve the response time of the interface, that is, parallel dependency calls.

The following figure shows the difference between serial calls and parallel calls. If a dependency is called serially, the time-consuming is equal to the sum of the time-consuming of all dependencies. If a dependency is called in parallel, the time-consuming is equal to the time-consuming of the most time-consuming dependency among all dependencies.

In the interface for obtaining product details, the parameter ProductIds is multiple product IDs separated by commas. Here we use the mapreduce provided by go-zero to obtain product details in parallel according to product IDs. The code is as follows. For details, please refer to product-rpc service :

 func (l *ProductsLogic) Products(in *product.ProductRequest) (*product.ProductResponse, error) {
    products := make(map[int64]*product.ProductItem)
    pdis := strings.Split(in.ProductIds, ",")
    ps, err := mr.MapReduce(func(source chan<- interface{}) {
        for _, pid := range pdis {
            source <- pid
        }
    }, func(item interface{}, writer mr.Writer, cancel func(error)) {
        pid := item.(int64)
        p, err := l.svcCtx.ProductModel.FindOne(l.ctx, pid)
        if err != nil {
            cancel(err)
            return
        }
        writer.Write(p)
    }, func(pipe <-chan interface{}, writer mr.Writer, cancel func(error)) {
        var r []*model.Product
        for p := range pipe {
            r = append(r, p.(*model.Product))
        }
        writer.Write(r)
    })
    if err != nil {
        return nil, err
    }
    for _, p := range ps.([]*model.Product) {
        products[p.Id] = &product.ProductItem{
            ProductId: p.Id,
            Name:      p.Name,
        }
    }
    return &product.ProductResponse{Products: products}, nil
}

On the product details page, not only the details of the product are displayed, but also the first page of the product evaluation is displayed on the page, and then click the evaluation details to jump to the evaluation details page. In order to avoid the client requesting multiple interfaces at the same time, we display the product details in The page returns the content of the comment homepage together. Because the comment content is not the core content, we have also downgraded it here, that is, we will ignore the error when requesting the reply.rpc interface to report an error, so that the product details can be displayed normally. Because obtaining product details and product reviews have no dependencies, we use mr.Finish to make parallel requests to reduce the time-consuming of the interface.

 func (l *ProductDetailLogic) ProductDetail(req *types.ProductDetailRequest) (resp *types.ProductDetailResponse, err error) {
    var (
        p *product.ProductItem
        cs *reply.CommentsResponse
    )
    if err := mr.Finish(func() error {
        var err error
        if p, err = l.svcCtx.ProductRPC.Product(l.ctx, &product.ProductItemRequest{ProductId: req.ProductID}); err != nil {
            return err
        }
        return nil
    }, func() error {
        var err error
        if cs, err = l.svcCtx.ReplyRPC.Comments(l.ctx, &reply.CommentsRequest{TargetId: req.ProductID}); err != nil {
            logx.Errorf("get comments error: %v", err)
        }
        return nil
    }); err != nil {
        return nil, err
    }
    var comments []*types.Comment
    for _, c := range cs.Comments {
        comments = append(comments, &types.Comment{
            ID: c.Id,
            Content:   c.Content,
        })
    }
    return &types.ProductDetailResponse{
        Product: &types.Product{
            ID:        p.ProductId,
            Name:      p.Name,
        },
        Comments: comments,
    }, nil
}

upload picture

Image uploading is a very common function. We need to upload product images in product-admin. Here we upload the product images to Alibaba Cloud OSS. The api is defined as follows

 syntax = "v1"

type UploadImageResponse {
    Success bool `json:"success"`
}

service admin-api {
    @handler UploadImageHandler
    post /v1/upload/image() returns (UploadImageResponse)
}

Add the following configuration to admin-api.yaml

 Name: admin-api
Host: 0.0.0.0
Port: 8888
OSSEndpoint: https://oss-cn-hangzhou.aliyuncs.com
AccessKeyID: xxxxxxxxxxxxxxxxxxxxxxxx
AccessKeySecret: xxxxxxxxxxxxxxxxxxxxxxxx

Add OSS client

 type ServiceContext struct {
    Config config.Config
    OssClient *oss.Client
}

func NewServiceContext(c config.Config) *ServiceContext {
    oc, err := oss.New(c.OSSEndpoint, c.AccessKeyID, c.AccessKeySecret)
    if err != nil {
        panic(err)
    }
    return &ServiceContext{
        Config: c,
        OssClient: oc,
    }
}

The upload logic needs to obtain the bucket first. The bucket is a pre-defined bucket, which can be created through api calls or manually created on Alibaba Cloud Workbench.

 func (l *UploadImageLogic) UploadImage() (resp *types.UploadImageResponse, err error) {
    file, header, err := l.r.FormFile(imageFileName)
    if err != nil {
        return nil, err
    }
    defer file.Close()
    bucket, err := l.svcCtx.OssClient.Bucket(bucketName)
    if err != nil {
        return nil, err
    }
    if err = bucket.PutObject(header.Filename, file); err != nil {
        return nil, err
    }
    return &types.UploadImageResponse{Success: true}, nil
}

Use Postman to upload images, note that you need to create a bucket before uploading images

Log in to Alibaba Cloud Object Storage to view uploaded images

concluding remarks

This article introduces some common configurations in service construction through log definitions and service dependencies. This article does not list all configurations, but exemplifies scenarios that are often asked in the community. Later articles will continue to improve. Service related configuration. Then, the optimization methods and coding methods of common functions are shown through two cases of service-dependent parallel calls and image uploads.

Not all the functions are listed here, but I also think of them. You can down the project and improve the project yourself. It’s superficial on paper, and you know you have to do it. Of course, I will continue to improve the project code. Learn and progress with everyone.

Hope this article is helpful to you, thank you.

Updated every Monday and Thursday

Code repository: https://github.com/zhoushuguang/lebron

project address

https://github.com/zeromicro/go-zero

Welcome go-zero and star support us!

WeChat exchange group

Follow the official account of " Microservice Practice " and click on the exchange group to get the QR code of the community group.