If you want to be a test engineer, you must know these 7 things first

Abstract: write code belongs to the development of the siege lion, and the test belongs to the test of the siege lion. In most cases, the two sides are in a "red and blue confrontation" state.

1. "Developer test" means "Developer to test"

Developer testing is a very important part of modern software engineering. Advanced project management methods and processes such as agile development and backbone development are all based on complete developer testing. When a version is delivered every month or even every week, it is impossible to invest a large number of test engineers to carry out large-scale system-level testing. At this time, most of the tests in the entire test pyramid need to be completed by automation.

We are talking about developer testing today. What is "developer testing"? Our company has a clear distinction between development and testing. Writing code belongs to the development of the siege lion, and testing belongs to the test of the siege lion. In most cases, the two sides are in a "red and blue confrontation" state. This is very similar to the situation of my R&D team more than 10 years ago. In software engineering nowadays, there are very few full-time "test siege lions". Many companies develop and test ratios greater than 10:1, and even some departments do not test siege lions. The role of the test siege lion is no longer the "coolie" of manually running test cases, but to manage the product test system, plan product tests, analyze and summarize functional test mind maps, design test cases, and lead the R&D team for testing. Work more like a "test expert/test coach". To give a simple example, the product I made before was an online video conference collaboration product. Our daily online regular meeting is to use our own products, and we will use the new functional test site developed by ourselves to hold "stand-up meetings." In addition to spending a small amount of time doing dialy update, then testing experts lead the team (including PO, architects, SM, Dev) to conduct centralized (half an hour) testing according to the plan. Therefore, "developer testing" means "developer testing", and our traditional test engineers face three ways: growth, transformation, and elimination. "Testing experts" also have a high right to speak in the project. My previous company used backbone development and had a "one-in, one-out" review. This type of "testing expert" of the team had a veto power. There are even PE-level testing experts in the company (equivalent to our 20-21-level technical experts).

• : For whether a feature can enter the release branch, turn on the feature toggle in the release branch to test the release level.
• comes out: When in the engineer release, the quality of the function is qualified and the feature toggle is allowed to enter the production line.

2. No test cannot be "automated test"

Going back to the testing pyramid, looking at the four dimensions of testing "development cost", "execution cost", "test coverage", and "problem location", white box testing based on code level is extremely important.

• Development cost: The cost of realizing test cases.
• Execution cost: the cost of running a test case.
• Test coverage: what we usually call line coverage and branch coverage
• Problem location: problems occur in the test, and the efficiency of the problem location

Through the evaluation of the test pyramid and its four test dimensions, we can get:

• Do as many Low Level Tests as possible: because their execution speed is much faster and relatively stable compared to the upper-level test types, and they can be executed multiple times a day. Generally speaking, LLT gray is implemented in continuous integration construction tasks, and even executed in MR, to ensure the quality of the code entering the code warehouse.
• In the case of automated guarantees, IT, ST, and UI Tests of a certain scale are executed: because their execution speed is slower, the environment is more dependent, and the test is unstable first. It is usually executed once at night to periodically check the code quality and feedback code problems.
• Do as few large-scale manual tests as possible: Because their execution speed is less stable than LLT, the labor cost is higher, and they cannot be executed multiple times a day. Each execution has to wait a long time to get feedback results. However, they are closer to real user scenarios, so make sure to perform the following tests within a certain period or at key node times to ensure software quality.

Now many companies have iteratively released the cycle shorter and shorter, even two weeks. Manual testing obviously cannot adapt to this development model, and the only way to automate manual testing use cases through various technical solutions is. At the code level, from the underlying business code to the UI code, as long as the architecture design is reasonable, UT can be used. The top-level UI interaction test, test cases can also be automated (most UI frameworks can be used for UI automated testing through the accessibility interface). Just imagine that even our mobile phone hardware can automatically test the extreme test of "smashing the phone". The software has What can't you do? At least some products of some of the industry's technology giants, from the code submission Merge Request to the product line can be counted in days. The testing of this product cannot and cannot be done through manual testing.

3. Developers test "to be in the present" and "to win the future"

Many people think that the low-level developer testing takes a lot of time and writes a lot of code to ensure the correctness of the function, but every time the code function or structure changes, the test code must be modified. It is more efficient for me to manually debug and verify. It is true that debugging code through UT and API testing is not much different from manually running debugging by yourself, but debugging the code through developer testing to ensure the quality of the current project iteration; but its more important role is not this.

We have such nouns in the bug classification: Build Regression Bug, Release Regression Bug.

• Build Regression Bug: The same function in development has a bug in the new version, but there is no such problem in the previous version, we call it Build Regression Bug.
• Release Regression Bug: The same function on the production line has a bug in the new version, but there is no such problem in the previous version. We call it Release Regression Bug.

Every time we commit to the code in the product, no one can guarantee that it will be 100% free of problems. Under the rapid iteration of agile development, it is impossible to conduct full-featured manual testing, so developer testing, especially the bottom layer UT, API testing, and integration testing can easily identify and find such problems. So it is said that developers test "profit in the present" and "win the future".

Four, TDD does not have to write test code first

For TDD, everyone’s perception is that you write the test code first, and then write the implementation code. This statement is true or not. The concept is correct, but if you do this strictly, the efficiency may not be the highest, which is one of the reasons why TDD is difficult to promote. We divide the coding implementation into three parts: implementation code, test code, and debugging code. According to the concept of TDD, first write test code, then code, and finally debug. When we usually implement the code, we are unlikely to think very clearly at the beginning, and define the interface completely and accurately. If we strictly follow the test, coding, and debugging, the test code should be frequently modified along with the coding. Of course, this is not a big problem in itself. In the actual implementation process, many people are used to setting up a code framework and a testing framework first, and then coding and testing. Debug after the test is completed. So from the perspective of Huawei's gray management, as long as the unit test is before debugging, it can be called a TDD development mode. BTW, of course, BDD is becoming popular now. What I want to say here is that if you can’t even do what I said about TDD, don’t consider BDD.

(Behavior-Driven Development: BDD combines the general technology and principles of TDD with the idea of Domain Driven Design (DDD). BDD is a design activity in which you can gradually build functional blocks according to expected behavior. The focus of BDD is the software development process Language and interaction used. Behavior-driven developers use their native language combined with a domain-driven design language to describe the purpose and benefits of their code. Teams using BDD should be able to provide a large number of "functional documents in the form of user stories" "And add executable scenarios or examples. BDD usually helps domain experts understand the implementation rather than expose code-level testing. It is usually defined in GWT format: GIVEN WHEN & THEN.)

Fifth, 100% coverage of UT is really bad

For unit testing, we all pay attention to a metric "coverage". Regardless of module, function, line, or branch coverage, there must be a certain percentage of coverage. But if you achieve 100% in each item, you will be given a "bad review". It's not that you can't do it well (I won't talk about whether you used the right way), but the cost and price-performance issues. With the most difficult-to-reach branch coverage (branch coverage), if 100% coverage is to be achieved, some memory allocation or fault-tolerant protected branches must be tested, then your test case is considered to be doubled, but it does not Corresponding value. Even some conditional branches of code have not been executed during the life cycle of the program.

• Module coverage: The business module code passes UT, and the architecture module code passes IT; from the perspective of UT coverage, there is no need to test the architecture code.
• Function coverage: Don't write UT for some code without any logic. For example, some of our functions are get/set an attribute, and the internal implementation uses a variable to assign and save it. This kind of function is written in UT for the purpose of coverage and has no real meaning.
• Line coverage: Generally speaking, a line coverage rate of 80% or less is a reasonable indicator. Some can be 0%, and some require 100%. If all codes exceed 90%, the cost is higher and the efficiency is lower. This is not recommended.
• Branch coverage: The more complex the business logic, the more test cases need to be written to cover, and some memory allocation error logic judgments may not require testing.

Six, use testing to drive architecture and code quality

Talking about test-driven architecture and code quality here, the main thing is to make the code have perfect testability. What is the testability of the code? Simply put, it is the decoupling of the relationship between the class and the class, the module and the module, the class and the class, and the module and the module through the interface programming. Dependent interfaces are passed in through passive injection instead of active acquisition. When the program is running normally, the interface parameters passed in are real business objects, and when testing, you can pass in the simulation implementation of fake. Of course, not all dependent modules do this. Some Utility Library that has nothing to do with business, or some specific data object implementations, can be called directly.

Here we talked about fake and mock. About Test Doubles, the basic concepts are as follows. You can search on the Internet for the specific meaning of each type.

• Dummy
• Stub
• Spy
• Mock
• Fake

At present, everyone in our company is basically using Mock Object when doing developer testing (actually, in the process of using, many of them are using Stubs with parameter return value control). Putting aside the conceptual problem, although the code can also be tested by Mock, in fact, using Mock basically means that our code is more related, the module display is more dependent, and the module portability is poor, especially the C language. Programming is particularly problematic. As a result, many modules are unable to carry out unit testing at all, and more are doing integration testing.

Why does this happen? Our high-level architects are more concerned about the system-level architecture design, and the relationship between system modules and various applications is very clear. Normally, high-level architects can compare system modules or applications. The design of the relationship is more reasonable. However, when it comes to the design and implementation of the specific application business, it is handed over to low-level architects to complete. In fact, the amount of code inside these modules is not small, many of which are hundreds of thousands or even millions of lines of code. At this time, the level of the architect determines the Clean Code quality of the code. Many of our current code problems are not system architecture issues, but the lack of strict requirements and reasonable architecture design in specific business implementation. If there is a set of architecture schemes to standardize at the application level, then at least the interface of the module and the interaction between the module and the module can be as clear and reasonable as the system design. Then the uncertain part is the tens of thousands of lines of code within each submodule.

The reason why test-driven architecture and code quality are proposed is that when a very high standard is put forward for testing, everyone has to solve the test problem from the architecture. When the test problem is solved, Clean Code L3 will naturally be achieved.

7. From "I want to write test-dependent code" to "I want to write test-dependent code"

This sentence looks very strange, but it is actually a fundamental way to solve the unit test. There are dependencies between modules, whether by Mock or Fake, no matter how reasonable the architecture is, this dependency cannot be eliminated. What we do more is to design a reasonable decoupling of dependencies and modules. The first "I want to write test dependency code" refers to when I implement my module, I want to write test code to test. But what I want to test is how to write my test dependencies. And the second "I want to write test dependency code" refers to when I implement my code, what I have to consider is how to solve my dependency when the module that depends on me is being tested, "I want to write a test Dependent code" (that is, the fake object and implementation I call) to help the modules that rely on me to solve the test dependency problem.

• Thinking change, test-driven: When developing a module, don't think about how to test yourself first, first think about how I can make it easier for others to test if others rely on me. The provider of the module must provide not only the module code, but also a reusable Faked object (call verification; return value; parameter verification; parameter processing; function simulation, etc.).
• The writer of the module code implements his own Fake implementation. Basically, most of the code is completed by the module writer. At the same time, this is a reusable Fake implementation. The module relying party adds its own code according to its own special business requirements. Basically follow the 80/20 principle.
• Architecturally rely on decoupling, and perform interface programming by injecting dependencies. Developer testing uses Fake to implement dependencies.
• When writing test code, all dependent interfaces and implementations of dependencies are basically completed, and more attention is paid to test cases rather than test dependencies.

Click to follow, and learn about Huawei Cloud's fresh technology for the first time~