go-zero microservices combat series (11, finale)

This article is the last article of the whole series. I originally planned to write about k8s deployment in the last one or two articles of the series. During the conception process, I felt that I had insufficient knowledge of k8s, and I did not understand and master it. Under the premise that I think it is difficult to write an article that I am satisfied with, everyone may feel that the content is not dry after reading it. I am also learning some best practices of k8s and reading the source code of k8s recently. When the time is right, I may consider writing a series of k8s actual combat articles.

Content review

Each article in the entire series is listed below. The main features of this series of articles are that they are close to real development scenarios and are optimized for high concurrent requests and common problems. The content of the article is also step-by-step. First, the background of the project is introduced, and then The service is split, and after the service is split, the API definition and table structure design are similar to our actual development process in the company. Next is to do some basic CRUD operations on the database, followed by three articles. To explain the cache, because the cache is the basis of high concurrency, there is no way to talk about the high concurrency system without the cache, the cache is mainly to deal with high concurrent reads, and then use two articles to optimize the high concurrent write, and finally pass Distributed transactions guarantee data consistency between services. If everyone can understand each article thoroughly, I think most of the scenarios in the work can be easily dealt with.

The sample code supporting the article has not been written very well. There are several reasons. First, the mall has so many function points that it is difficult to cover all the logic. Second, most of the business logic is repeated, as long as everyone masters it. With the core sample code, other business logic can be supplemented by downgrading the code, so I think it will improve. If there is something you don’t understand, you can ask me in the community group, and every community group can find me.

go-zero microservices combat series (1, the beginning)

go-zero microservice combat series (two, service split)

go-zero microservice combat series (3. API definition and table structure design)

go-zero microservices combat series (four, CRUD warm-up)

go-zero microservice combat series (5, how to write cache code)

go-zero microservice combat series (six, cache consistency guarantee)

go-zero microservice combat series (seven, how to optimize the request volume is so high)

go-zero microservice combat series (8. How to handle tens of thousands of order requests per second)

go-zero microservices combat series (9. Ultimate optimization of spike performance)

go-zero microservice combat series (10. How to implement distributed transactions)

unit test

Software testing begins with unit testing. More complex tests are done on top of unit tests. Hierarchical model tested as follows:

Unit tests are the smallest and simplest form of software testing. These tests are used to evaluate the correctness of an individual software unit, such as a class, or a function. These tests do not take into account the correct determination of the overall system containing the software unit. Unit testing is also a specification used to ensure that a function or module fully conforms to the system's behavior requirements for it. Unit testing is often used to introduce the concept of test-driven development.

go test tool

The test dependency of go language go test tool, which is a driver of test code according to certain conventions and organizations. In the package directory, all source code files suffixed with _test.go go test are part of the test of ---3a174bc2e745af8d95abd5d2ae405033--- and will not be compiled into the final executable by go build .

In the *_test.go file there are three types of functions, unit test functions, benchmark functions and example functions:

type	Format	effect
test singular	The function name is prefixed with Test	Test whether some logical behavior of the program is correct
benchmark function	The function name is prefixed with Benchmark	Test the performance of a function
Example function	The function name is prefixed with Example	provide examples

go test will traverse all *_test.go functions in the file that conform to the above naming rules, and then generate a temporary main package to call the corresponding test function.

Single test format

Each test function must import the testing package. The basic format of the test function is as follows:

 func TestName(t *testing.T) {
    // ......
}

The name of the test function must start with Test , and the optional suffix name must start with a capital letter, for example:

 func TestDo(t *testing.T) { //...... }
func TestWrite(t *testing.T) { // ...... }

testing.T Used to report test failures and additional log information. The main methods are as follows:

 Name() string
Fail()
Failed() bool
FailNow()
logDepth(s string, depth int)
Log(args ...any)
Logf(format string, args ...any)
Error(args ...any)
Errorf(format string, args ...any)
Fatal(args ...any)
Fatalf(format string, args ...any)
Skip(args ...any)
Skipf(format string, args ...any)
SkipNow()
Skipped() bool
Helper()
Cleanup(f func())
Setenv(key string, value string)

Simple example

Under this path lebron/apps/order/rpc/internal/logic/createorderlogic.go:44 there is a function to generate order id, the function is as follows:

 func genOrderID(t time.Time) string {
    s := t.Format("20060102150405")
    m := t.UnixNano()/1e6 - t.UnixNano()/1e9*1e3
    ms := sup(m, 3)
    p := os.Getpid() % 1000
    ps := sup(int64(p), 3)
    i := atomic.AddInt64(&num, 1)
    r := i % 10000
    rs := sup(r, 4)
    n := fmt.Sprintf("%s%s%s%s", s, ms, ps, rs)
    return n
}

We create the createorderlogic_test.go file and write the corresponding unit test function for this method. The length of the generated order id is 24. The unit test function is as follows:

 func TestGenOrderID(t *testing.T) {
    oid := genOrderID(time.Now())
    if len(oid) != 24 {
        t.Errorf("oid len expected 24, got: %d", len(oid))
    }
}

Execute the go test command in the current path, and you can see the output as follows:

 PASS
ok      github.com/zhoushuguang/lebron/apps/order/rpc/internal/logic    1.395s

You can also add -v to output a more complete result, go test -v The output is as follows:

 === RUN   TestGenOrderID
--- PASS: TestGenOrderID (0.00s)
PASS
ok      github.com/zhoushuguang/lebron/apps/order/rpc/internal/logic    1.305s

go test -run

When executing the go test command, you can add the -run parameter, which corresponds to a regular expression, and the test function that matches the function name will be used by the go test command To execute, for example we can use go test -run=TestGenOrderID to run the value of TestGenOrderID this single test.

form-driven testing

Table-driven testing is not a tool, it's just a way and perspective to write clearer tests. Writing good tests is not an easy task, but in many cases, table-driven tests can cover many things. Each entry in the table is a complete test case, which contains the input and expected results, and sometimes Additional information such as the test name is included to make the test output easier to read. Using table tests can easily maintain multiple test cases and avoid frequent copying and pasting when writing unit tests.

In lebron/apps/product/rpc/internal/logic/checkandupdatestocklogic.go:53 we can write the following table-driven test:

 func TestStockKey(t *testing.T) {
    tests := []struct {
        name   string
        input  int64
        output string
    }{
        {"test one", 1, "stock:1"},
        {"test two", 2, "stock:2"},
        {"test three", 3, "stock:3"},
    }

    for _, ts := range tests {
        t.Run(ts.name, func(t *testing.T) {
            ret := stockKey(ts.input)
            if ret != ts.output {
                t.Errorf("input: %d expectd: %s got: %s", ts.input, ts.output, ret)
            }
        })
    }
}

Execute the command go test -run=TestStockKey -v the output is as follows:

 === RUN   TestStockKey
=== RUN   TestStockKey/test_one
=== RUN   TestStockKey/test_two
=== RUN   TestStockKey/test_three
--- PASS: TestStockKey (0.00s)
    --- PASS: TestStockKey/test_one (0.00s)
    --- PASS: TestStockKey/test_two (0.00s)
    --- PASS: TestStockKey/test_three (0.00s)
PASS
ok      github.com/zhoushuguang/lebron/apps/product/rpc/internal/logic    1.353s

Parallel testing

Table-driven testing usually defines more test cases, and the go language naturally supports concurrency, so it is easy to use its own advantages to parallelize table-driven testing, which can be achieved by t.Parallel() :

 func TestStockKeyParallel(t *testing.T) {
  t.Parallel()
    tests := []struct {
        name   string
        input  int64
        output string
    }{
        {"test one", 1, "stock:1"},
        {"test two", 2, "stock:2"},
        {"test three", 3, "stock:3"},
    }

    for _, ts := range tests {
        ts := ts
        t.Run(ts.name, func(t *testing.T) {
            t.Parallel()
            ret := stockKey(ts.input)
            if ret != ts.output {
                t.Errorf("input: %d expectd: %s got: %s", ts.input, ts.output, ret)
            }
        })
    }
}

test coverage

Test coverage refers to the percentage of code that is covered by the test suite. Usually we use statement coverage, that is, the proportion of code that is run at least once in the test to the total code. Go provides built-in functionality to check code coverage, i.e. use go test -cover to view test coverage:

 PASS
coverage: 0.6% of statements
ok      github.com/zhoushuguang/lebron/apps/product/rpc/internal/logic    1.381s

It can be seen that our coverage rate is only 0.6%, haha, this is very unqualified, a big unqualified. Go also provides a -coverprofile parameter to output coverage-related records to a file go test -cover -coverprofile=cover.out :

 PASS
coverage: 0.6% of statements
ok      github.com/zhoushuguang/lebron/apps/product/rpc/internal/logic    1.459s

Then execute go tool cover -html=cover.out , use cover tool to process the generated record information, this command will open a local browser window to generate a test report

resolve dependencies

For the dependencies in the single test, we generally use the mock method to deal with it. Gomock is an official testing framework provided by Go, which can be easily used in the built-in testing package or other environments. We use it to mock those interface types in the code to facilitate writing unit tests. For the use of gomock, please refer to the gomock documentation

Mock depends on interface. For dependencies in non-interface scenarios, we can use piling to mock data. Monkey is a very commonly used piling tool in Go unit testing. It rewrites the executable file through assembly language at runtime, and converts the target function Or the implementation of the method jumps to the stub implementation, which works like a hot patch. The monkey library is very powerful, but you need to pay attention to the following when using it:

• monkey does not support inline functions, you need to pass the command line parameter -gcflags=-l to turn off the inline optimization of the Go language when testing.
• monkey is not thread safe, so don't use it in concurrent unit tests.

other

drawing equipment

People in the community often ask what tools are used for drawing. The illustration tools in this series of articles are mainly the following two

https://www.onemodel.app/

https://whimsical.com/

code specification

The code is not only to realize the function, it is very important that the code is written for others to see, so we must have certain requirements for the quality of the code, to follow the specifications, you can refer to the official code review suggestions of go

https://github.com/golang/go/wiki/CodeReviewComments

talk about feelings

Time flies so fast, and before I know it, eleven articles have been written in this series. It has been written for more than a month at the rate of two weekly updates. Writing articles is laborious and time-consuming, and I am afraid that there will be something wrong in the writing, which will mislead everyone, so it is necessary to repeatedly check and consult relevant materials. On average, it takes about a day to write an article. Usually, the working days are busy, and I usually write on Saturdays and Sundays. Therefore, I have basically not had a rest on Saturdays and Sundays in the past month.

But I think the harvest is also very great. In the process of writing an article, the knowledge points that I have mastered are a process of review, which can deepen my understanding of the knowledge points, and the knowledge points that I have not mastered is another study. The process of new knowledge allows me to master new knowledge, so I and the readers are also learning and progressing together. Everyone knows that it is not easy to speak or write the knowledge that you understand so that others can also understand it. Therefore, writing articles also greatly improves your soft power.

Therefore, I will continue to insist on writing articles, insist on output, and learn and grow with everyone. At the same time, I also welcome everyone to contribute to the "Microservice Practice" public account. Some people may feel that their level is not good enough, and they are worried that the content they write is not high-end or compelling. I don’t think it is necessary. As long as you can explain the knowledge points, it will be great. It can be basic knowledge or best practices, etc. Wait. kevin will carefully review every article submitted, and he will point out the wrong places. There are still opportunities for one-on-one exchanges and learning with kevin, so please act quickly.

concluding remarks

Thank you very much for your support over the past month. Seeing that each article has so many likes, I am very happy and more motivated. Therefore, I am also planning to write a series of articles. There are currently two topics to be selected, namely "go-zero" Source Code Series" and "gRPC Actual Source Code Series", friends are welcome to leave your comments in the comment area and say which series you are looking forward to more. If the number of likes for this article exceeds 66, we will continue to open it.

Code repository: https://github.com/zhoushuguang/lebron

project address

https://github.com/zeromicro/go-zero

Welcome go-zero and star support us!

WeChat exchange group

Follow the official account of " Microservice Practice " and click on the exchange group to get the QR code of the community group.