Automation will increase test coverage. What is test coverage?

What is test coverage?

Test coverage is a term for computer science and technology published in 2018. It is one of the metrics for test quality and tells us how much code has been tested. It defines certain entities of the system, and the purpose is to cover them with tests. This is a way to indicate when we have done enough testing and tell us what else needs to be tested (thus expanding the coverage).

If you compare the test to cleaning the house, the rooms that are not commonly used will always be missed, and we will always forget areas such as the second bedroom bathroom, so the cleaning coverage does not include that bathroom.

Imagine if our cleaning standard only included cleaning the bedroom. According to this standard, if all the bedrooms are cleaned, does it mean that the whole house is clean?

Not so, because we completely ignored the kitchen, dining room, bathroom and other parts of the house! Therefore, it is important to measure test coverage meticulously and accurately. A certain degree of coverage is an indicator of test quality, but it is never an indicator of system quality, nor can it guarantee that all content has been tested. Test coverage tells us the percentage of test code, but it does not mean that it has been tested in every situation.

What are the advantages of test coverage?

Still taking house cleaning as an example, test coverage can be:
● Measure the quality of cleaning;
● Indicate when to stop cleaning;
● Remind us that there are other areas to clean up.

Some standards may be more powerful than others. Understanding them allows us to know the depth of the test and when to apply one or other standards. For example, if any test case set TS that covers criterion A also covers criterion B, it can be said that criterion A includes another criterion B.

According to the above principles, cleaning the house can follow the following standards:

1. Clean every bedroom;
2. Clean every part of the house (including bedroom, kitchen, bathroom, etc.);
3. Clean every small place, even corners, because they are likely to accumulate dust.

Obviously, criterion 3 includes 2 and 2 includes 1, and the relationship is transitive, that is, 3 includes 1. If we design a test case for Criterion 3, it should also include the first two standards. For test software, standards usually include various paths, conditions, statements, functions, etc. in the program.

Another real example may be the equivalence class division. First divide several equivalence classes, and then select representative data from each part as test cases for reasonable classification, so that all classes are covered. If you consider white box testing, there are statement coverage, branch coverage, path coverage, etc., especially for state machines, we have standards that indicate coverage of all nodes, all transitions, etc.

How does automated testing apply?

Now you can imagine throwing away the broom and replacing it with a super high-speed sweeping robot. It can free your hands, clear the floor faster, reduce missing areas, and allow yourself to focus on more important things at work.

At this point, we can draw the conclusion: test automation = higher test coverage.