Single test from head to toe-on effective unit testing (part 1)

1. "Rectify" the unit test
I used to think that unit testing is for a function. Any test that goes out of a function is not a unit test.
In fact, the definition of "unit" is up to you. If you are using functional programming, a unit most likely refers to a function. Your unit test will call this function with different parameters and assert that it returns the expected result; in an object-oriented language, from a method to a class can be a unit (from a single method to a The entire class can be a unit). Intention is very important (the word "intention" is mentioned for the first time in this article, it is very important)
We have unit testing, incremental testing, integration testing, regression testing, smoke testing, etc., with many names. Google saw this phenomenon of "contending of a hundred schools of thought" and created its own naming method, which is only divided into small tests, medium tests and large tests.
·Small test, aiming at the test of a single function, paying attention to its internal logic and mocking all required services.
Small tests bring excellent code quality, good exception handling, and elegant error reporting
·Medium-sized test to verify the interaction between two or more specified module applications
· Large-scale testing, also known as "system testing" or "end-to-end testing". Large-scale tests run at a higher level, verifying how the system as a whole works.

Conclusion: In our unit test, we can either write a case for a function, or write a case in series according to the calling relationship of the function.
2. Pyramid model
Before the pyramid model, the ice cream model was popular. Contains a large number of manual tests, end-to-end automated tests and a small number of unit tests. The consequence is that as the product grows, manual regression testing takes longer and longer, and the quality is difficult to control; automated cases fail frequently, and each failure corresponds to a long function call. What went wrong? There are few unit tests and they are basically useless.

Mike Cohn put forward the concept of "test pyramid" in his book "Succeeding with Agile". This analogy is very vivid, it lets you know at a glance that the test needs to be layered. It also tells you how many tests you need to write for each layer.
The test pyramid itself is a good rule of thumb. We’d better remember the two things Cohn mentioned in the pyramid model:
·Write different granularity tests
·The higher the level, the fewer tests you should write

At the same time, our understanding of the pyramid must not stop here, we must further understand:

I understand the pyramid model as-the ice cream melts. It means that the top "manual testing" should theoretically be automated and melted downwards. Priority is given to melting into unit tests. Unit tests cannot be covered in the middle layer (layered testing), and those that cannot be covered will be put in the middle layer. Go to the UI layer. Therefore, the case of the UI layer, if you don't have it, don't have it, and the running is slow and unstable. According to the leader of Qiao, I do not distinguish between unit testing or layered testing. The unity is called automated testing. Then all automated cases should be regarded as a whole. The case should not be redundant, and the unit test should cover this case. Remove from layering or ui.
The lower the test, the less relevant content is involved, while the higher-level test involves a wider range. For example, unit testing, which focuses on only one unit and nothing else. Therefore, as long as a unit is written, the test can be passed; while integration testing requires several units to be assembled together to be tested. The prerequisite for passing the test is that all these units are written, and this cycle is obviously better than The unit test needs to be long; the system test needs to connect the various modules of the entire system together, and all kinds of data are ready before it can pass.
In addition, because there are too many modules involved, any adjustment of any module may destroy high-level tests. Therefore, high-level tests are usually relatively fragile. In actual work, some high-level tests involve external systems. As a result, the complexity continues to increase.
3. Why do single test
We can't avoid this problem. News is one of the main forces in this reform of the R&D model, so the top-down promotion makes this problem less difficult: do it and do it. There are so many reasons not to do it:
(Tucao real voice collected)
· Unit testing wastes too much time
· Unit testing just proves what the code does
· I am a great programmer, can I skip unit testing?
· Later integration tests will catch all bugs
· The cost efficiency of unit testing is not high. I wrote all the tests, so what do the testers do?
· The company invited me to write code, not test
· It is not my job to test the correctness of the code
The meaning of unit testing
·Unit testing is very important to our product quality.
·Unit testing is the lowest type of testing in all tests. It is the first and most important link. It is the only test that can guarantee 100% code coverage. It is the basis and foundation of the entire software testing process. The premise is that unit testing prevents the late development from getting out of control due to too many bugs, and unit testing is the best cost-effective.
·According to statistics, about 80% of errors are introduced in the software design phase, and the cost of correcting a software error will increase as the software lifecycle progresses. The later the error is discovered, the more expensive it will be to repair it, and it will increase exponentially. As a coder, he is also the main executor of unit testing. He is the only person who can produce a defect-free program. No one else can do this.
·Code standard, optimized, testable code
·Reconstruct with confidence
·Automatically execute three-thousand times
The following picture, from Microsoft's statistics: A bug was discovered during the unit test phase, and it took an average of 3.25 hours, and if it missed the system test phase, it would take 11.5 hours.

The following picture is intended to illustrate two problems: 85% of defects are generated in the code design phase, and the later the bug is found, the higher the cost, and the exponential increase. Therefore, bugs can be found in the early unit tests, saving time and effort, once and for all, why not do it.

Is unit testing particularly time-consuming?
It cannot be one size fits all, and cannot just focus on the time-consuming phase of a single test.
I interviewed the development of the news client and the backend. First of all, I am sure that the single test will increase the amount of development and increase the development time;

A case is mentioned in the book "The Art of Unit Testing": I found two teams with similar development capabilities and developed similar requirements at the same time. The length of the single test team in the coding phase has doubled from 7 days to 14 days. However, the performance of this team in the integration test phase is very smooth, with a small amount of bugs, and rapid positioning of bugs. The final result, the overall delivery time and the number of defects, are the least for the single test team.

Single test, existence is reasonable. On the one hand, it is necessary to put the single test in the entire iteration cycle to observe its effect; on the one hand, writing a single test is also a technical skill, students who write well, less time and high code quality (that is, not to say that a single test is written, just Able to write single test)
Who will write the single test?
·Develop students to write a single test
·Test students have the ability to write single tests. The focus is on developing scaffolding, layered testing/end-to-end testing
Increment or stock
·Single test case is for incremental code
·When the stock code undergoes large-scale refactoring and the quality of the latter exposes great risks, it is a good time to promote the completion of the single test
4. The stage of unit testing
1. In a broad sense of unit testing, we refer to the organic combination of these three parts:
·code review
·Static code scanning
·Unit test case writing
2. Combining the practice of journalism, I divided the single test growth process into 4 goals, namely:
·Able to write, all staff can write
·Writing well, while paying attention to measurability issues, and solving them on a pilot basis
·Identify testability issues, and use refactoring methods to refactor proficiently; identify code architecture design issues; write case and business code synchronously
· TDD. But this goal is an expectation and cannot be regarded as a goal that must be achieved.

As of the day of publication, the news is in the third stage, that is, each iteration can produce high-quality cases, and the number of people and demand coverage are high; the focus is on measurability, and always pay attention to refactoring.

V. Indicators of unit testing
It's quite embarrassing, there is not a direct indicator to measure the effect of a single test. We are often asked, "How do you prove the effect of your newsletters?"
·Bug indicators (indirect indicators): the trend of the total number of bugs in successive iterations, the trend of new bugs in the iteration, and the rate of thousands of lines of bugs
·Single test demand coverage (above 50%), participant coverage (above 80%)
·Single test case total number trend, code line incremental trend
·Incremental code line coverage (80% at the access layer, 30% at the client)
·Single-function cyclomatic complexity (less than 40), number of single-function code lines (less than 80), number of scan alarms
Under the premise of continuous high throughput requirements for iterations, take news iOS data as an example:




Six. Go unit testing framework selection
Basic selection: testify + gomonkey
Additional: httptest + sqlmock


premise
·The test file ends with _test.go and is placed in the same directory as the tested file
· Test function, the function name starts with Test, and the first character after it must be an uppercase letter or an underscore, such as: TestParseReq_CorrectNum_TableDriven
·Test function, the parameter is t testing.T; for the bench test, the parameter is b testing.B
·Run the command line, my article has an in-depth explanation: go test command line

Testify general usage
https://github.com/stretchr/testify
testify is written based on gotesting, so grammatically, the execution command line is fully compatible with go test
Support a large number of efficient APIs, such as:
assert.Equal: Regular comparison is to replace the two with []byte for strict comparison
assert.Nil: When judging the object as nil, sometimes it is also used when judging err as null
assert.Error: Determine the specific type and content of err
assert.JSONEq: This is more useful, when comparing map; or when comparing struct, it will also be converted to map first. When using this api for comparison, as in the following example, I encapsulated the suggested method to convert struct to string (json):

· Support suite, use case set management
· At runtime, you can specify the set of use cases for execution

· Comes with mock tools, but only supports mocks of interface methods, and the usage is relatively complicated
· table-driven

Gomonkey usage (blue font indicates commonly used)
https://github.com/agiledragon/gomonkey
https://studygolang.com/articles/15034

·Support for stubbing a function
·Support to build a pile for a member method
·Support to build a pile for a global variable
·Support to build a pile for a function variable
Support a specific sequence of stubs for a function
· Support a specific sequence of stubs for a member method
Support a specific sequence of stubs for a function variable
A series of stubs are defined in a table-driven way
Note that for inline function stubs, parameters must be added to the go test command line to take effect. See official documentation. Therefore, I add -gcflags=all=-l to my command line by default.

I set up some code templates for goland and put them in the attachment.
ApplyFunc is an external function Stub (non-class method)
/* Usage: gomonkey.ApplyFunc (name of stub function, signed by stub function) function return value
*example:
    patches := gomonkey.ApplyFunc(fake.Exec, func(_ string, _ ...string) (string, error) {
    return outputExpect, nil
                   })
 */

patches := gomonkey.ApplyFunc(lcache.GetCache, func(_ string) (interface{}, bool) {
return getCommentsResp()
})
defer patches.Reset()
ApplyMethod is a Stub for the class function. But note here that the way to be stubbed is a private method. Gomonkey cannot be found through reflection. There are two solutions: 1) use the enhanced version of gomonkey; 2) not stub it, but choose to enter this function, This topic will be discussed later in the topic of mock.
/* Usage: gomonkey.ApplyMethod (reflection class name, signed by stub function) function return value
*example:
    var s *fake.Slice
    patches := ApplyMethod(reflect.TypeOf(s), "Add", func(_ *fake.Slice, _ int) error {
                return nil
            })
 */
 
var ac *auth.AuthCheck
patches := gomonkey.ApplyMethod(reflect.TypeOf(ac), "PrepareWithHttp", func(_ auth.AuthCheck, http.Request, ...auth.AuthOption) error {
return fmt.Errorf("prepare with nil object")
})
defer patches.Reset()
ApplyMethodSeq returns different results for the same Stub function
/* Usage: gomonkey.ApplyMethodSeq (class reflection, "stub function name", return structure);
Params{info1}, inside the brackets is the return value list of the stub function;
Times is the effective times
*example:
    e := &fake.Etcd{}
    info1 := "hello cpp"
    info2 := "hello golang"
    info3 := "hello gomonkey"
    outputs := []OutputCell{
         {Values: Params{info1, nil}},
         {Values: Params{info2, nil}},
         {Values: Params{info3, nil}},
      }
      patches := ApplyMethodSeq(reflect.TypeOf(e), "Retrieve", outputs)
      defer patches.Reset()
 */
conn := &redis.RedisConn{}
patch1 := gomonkey.ApplyFunc(redis.NewRedisHTTP, func(serviceName string, _ string) *redis.RedisConn {
conn := &redis.RedisConn{
redis.RedisConfig{},
&redis.RedisHelper{},
}
return conn
})
defer patch1.Reset()
 
// mock redis data. Return empty and not empty
outputCell := []gomonkey.OutputCell{
{Values: gomonkey.Params{"12", nil}, Times: 1},
{Values: gomonkey.Params{"", nil}, Times: 1},
}

patchs := gomonkey.ApplyMethodSeq(reflect.TypeOf(conn.RedisHelper), "Get", outputCell)
defer patchs.Reset()
Let me give you a few examples. The details can be found in the article linked above.
Here is a supplement, to stub a class method, you must find the real class (structure) corresponding to the method, for example:
//There is the following paragraph in the function under test, in which we want to stub the Get method, just find the class corresponding to the Get method.
readCountStr, _ := conn.Get(redisKey)
if len(readCountStr) == 0 {
return 0, nil
}
Locate conn, which is a struct of type RedisConn
 
type RedisConn struct {
RedisConfig
*RedisHelper
}
So for the first time, when I used gomonkey.AppleyMethod, I wrote:
 
patches := gomonkey.ApplyMethod(reflect.TypeOf(RedisConn),"Get", func(_ redis.RedisHelper,_ string, _ []string) ([]string, error){
return info,err_notNil
})
defer patches.Reset()

WeTest editor reminds: The content of the first part is here, I believe you have not read enough~ In the next part, we will talk about more exciting content about mocks, and how to not abuse mocks, let's take a look together. Next~ "Single Test from Head to Toe-Talking about Effective Unit Testing (Part 2)"