Unit testing-"simple" fun

The ability to endure simple

Knowing that the big V Li Songwei told a story about interacting with his daughter, which is very interesting:

I turned off the light and said to my daughter, "Close your eyes and stop moving."
The daughter immediately protested loudly: "But I can't sleep!"
I had to emphasize it again: "I just ask you to close your eyes and don't move."

Li Songwei "The Ability to Endure Simple"

He did not ask his daughter to "fall asleep as soon as possible", but made a simpler request; but the smart daughter immediately thought of "I can't sleep even if I close my eyes" and protested.

In this article, he said: "For smart people, the most unbearable situation is not how difficult a thing is, but pure simplicity." "A task without difficult challenges will make them feel powerless. ", "Repetitive practice is their deadly spot."

unit test

Unit testing seems to be a "simple and repetitive" process. Whether it looks or is written, it is composed of a lot of GIVEN-WHEN-THEN.

But under this "simple" appearance, there are two "simple" but very important questions hidden:

1. Why write a single test?
2. How to write a good single test?

According to the routine, you should first say "Why do you want to write a single test", but too routine is a bit boring, so let's talk about "how to write a good single test".

Noodle style code

The so-called spaghetti code means that a certain code is the same as pasta (not macaroni).

It's not a good thing anyway.

I recently saw such a piece of code, the function is to create a month's duty record:

def onduty(names):
  date = datetime.strptime("2021-07-01", "%Y-%m-%d")
  idx = 0
  while date < datetime.strptime("2021-07-31", "%Y-%m-%d"):
    post_data = {
      "date": date.strftime("%Y-%m-%d"),
      "name": names[idx],
      "backup": names[(idx+1)%len(names)],
    }
    requests.post(API_URL, json=post_data)
    idx = (idx + 1) % len(names)
    date += timedelta(days=1)

Note: The original code has 60 lines, which is slightly simplified here.

This is a typical "logic is complete, but not single testable" code:

• Need to request an external system (core reason)
• Hard-coded the time period (secondary issue)

So how should a single test be written for it?

Refactor

If a piece of code is not easy to write a single test, it means that there is a problem with its code structure.

Lu Xun "I Haven't Said This Word"

For code with structural problems, the first thing to do is obviously to refactor.

We first focus on the main problem of this code: calling "requests.post" requests an external system, which causes it to be coupled with the external system.

An easy idea to think of is to decouple through dependency injection:

def onduty(names, saver)
  ...
  saver(post_data)
  ...

After such a simple transformation, it becomes a "single testable" code: by mocking a saver, we can collect and verify its output, for example

class Saver(object):
  def __init__(self):
    self.output = []
  def mocker(self):
    def f(post_data):
      self.output.append(post_data)
    return f
    
def test():
  f = Saver()
  onduty(['a', 'b', 'c'], f.mocker())
  check(f.output)

But the code written in this way is very obscure. A more reasonable method is to split this logic into "Generate Duty List" and "Report to Duty System":

def onduty(names):
  arrangement = arrange(names)
  register(arrangement)

Then we can implement "Generate Watch List" as a pure function:

def arrange(names):
  arrangement = []
  date = datetime.strptime("2021-07-01", "%Y-%m-%d")
  idx = 0
  while date < datetime.strptime("2021-07-31", "%Y-%m-%d"):
    arrangement.append({
      "date": date.strftime("%Y-%m-%d"),
      "name": names[idx],
      "backup": names[(idx+1)%len(names)],
    })
    idx = (idx + 1) % len(names)
    date += timedelta(days=1)
  return arrangement

Just like y = f(x) math class, it does not produce any side effects, so we can easily write a single test arrange

def TestArrage():
    // Given
    names = ['a', 'b', 'c']
    // When
    arrangement = arrange(names)
    // Then
    check(arrangment)

The implementation of "Reporting on Duty System" is like this:

def register(arrangement):
  for item in arrangement:
    requests.post(API_URL, item)

Because it involves external systems, it is really not suitable for writing single tests, and it is more suitable to use functional tests to ensure its correctness.

In addition, because arrange , the verification logic of a single test will be very cumbersome, so we can reconstruct it and input the date as a parameter:

def arrange(names, from_date, to_date):
  ...

This makes the responsibilities of the code more clear, not only can improve the reusability of this code, but also can check more special cases (such as large and small months, leap years, etc.).

To summarize:

• Improve the "single testability" of the code through refactoring
• Resolve external dependencies through dependency injection-"interface-oriented programming"
• By splitting the business logic of different links, the cohesion of the code is further improved
• Improve code reusability by parameterizing hard-coded values

Of course, the above is just a simple example, not a complete single test methodology. There are many other links that need to be considered in practice:

• Choose the appropriate single test framework (such as JUnit)
• How to use mock tools/libraries to improve coverage
• How to make trade-offs between statement coverage, branch coverage, and condition coverage
• How to combine CI tools and use single test coverage to evaluate code quality
• ……

Interested students can refer to Tencent Technology Engineering's " talk about unit testing those things ".

Benefits of single test

Through the above operation, we have seen the benefits of single test:

• In order to write a single test, poorly structured code must be refactored, thereby improving the quality .

And more important than refactoring existing code is:

• In order to write a single test, the newly added code must also ensure a reasonable structure, thereby improving the thinking of .

Of course, students who have just started to practice the single test may feel that this reduces the coding speed;

But after a period of repetition , this way of thinking will be internalized , naturally will be able to write high-quality code.

In practice, the single test actually greatly improves the efficiency .

It is often time-consuming to construct a complete test. It takes 1 minute to compile, 1 minute to start, and 1 second to send a test request. The "price ratio" is very low (this may be the reason why many students do not like testing).

The single test only needs to compile and run a small part of the code, so the code logic can be quickly verified.

Since a large number of code bugs have been discovered and fixed during the single test, the number of subsequent "modification-compile-start-test" links can be greatly reduced, which also greatly improves the overall test efficiency.

There is also a piece of data from Microsoft in "Talking about Unit Testing":

The average time it takes to find bugs in different test phases:

• The unit testing phase took an average of 3.25 hours
• The integration test phase took an average of 6.25 hours (+92%)
• The system test phase takes an average of 11.5 hours (+254%)

A case I encountered recently is also a good example: the project at hand has multiple versions in parallel. The functions I developed in version A need to be merged to version B. After the merge, after running a round of test cases, you can say with more confidence, merge There is no problem with the code after —

Similarly, when we need to add new features to a piece of code, if there is an existing unit test, I can modify it with confidence.

Concluding remarks

In "The Ability to Endure Simple", Li Songwei said:

Therefore, among the students I know, except for a few talented talents, the factor that really affects a person's achievement may not be IQ or hard work, but how "reliable" he is.

To write high-quality code, start from writing a single test steadily.

btw, Li Songwei's article is too classic, I can't help but quote another paragraph:

It's too slow to eat bite by bite. Can't wait to eat a hundred mouthfuls in one bite, who told me that there are still so many in the pot?
So the important things have to be said three times. But what is it that made you read it three times in the last paragraph, do you remember?

If you don’t remember, you can try the following sentence:

Join Data to help customers achieve data-driven.

Join Data to help customers achieve data-driven.

Join Data to help customers achieve data-driven.

Shence Data is a big data company dedicated to "helping 30 million enterprises restructure their data foundation and realize digital operations". The company is developing rapidly. There are R&D centers in Beijing, Shanghai, Wuhan, Chengdu, Xi'an, Hefei and other places. The back-end, front-end, client, QA and other positions are waiting to be vacant. Students who are interested in big data should not miss it— —

→ Click here to view all positions of

Welcome to follow

Reference link

1. The ability to endure simple
2. Talk about unit testing those things