Wanzi detailed essay: thoroughly understand the Jest unit testing framework

• What is Jest
• What does the test mean
• How do I know what to test

• Test block, assertion and matcher

  • How to implement the test block
  • How to implement the assertion and matcher
  • CLI and configuration

• Simulate

  • How to simulate a function

• Execution environment

  • Scope isolation
  • V8 virtual machine
  • Run single test callback
  • hook function
• Generate report
• jest-cli
• jest-config
• jest-haste-map
• jest-runner
• jest-environment-node
• jest-circus
• jest-runtime
• last & source code

Thoroughly understand the Jest unit testing framework

This article mainly gives you an in-depth understanding of the operating principle behind Jest, and simply implements a Jest unit test framework from scratch to facilitate understanding of how the unit test engine works. I believe we are already familiar with Jest writing single tests, but how is Jest? We may be very unfamiliar at work, so let us walk into Jest's heart and explore how the unit test engine works together.

First attach the code implementation of the Jest core engine to students in need, welcome to pay attention and exchanges: https://github.com/Wscats/jest-tutorial

What is Jest

Jest is a Javascript testing framework developed by Facebook. It is a JavaScript library for creating, running and writing tests.

Jest is released as an NPM package and can be installed and run in any JavaScript project. Jest is currently one of the most popular test libraries for the front-end.

What does testing mean

In technical terms, testing means checking whether our code meets certain expectations. For example: a sum ) should return the expected output given some operation results.

There are many types of tests, and you will soon be overwhelmed by the terminology, but the long story short tests fall into three categories:

• unit test
• Integration Testing
• E2E testing

How do i know what to test

In terms of testing, even the simplest code block may confuse beginners. The most common question is "how do I know what to test?".

If you are writing a web page, a good starting point is to test every page of the application and every user interaction. However, the web page also needs to be composed of code units such as functions and modules to be tested.

There are two situations most of the time:

• You inherit the legacy code, which has no built-in tests
• You must implement a new feature out of thin air

so what should I do now? In both cases, you can think of the test as: checking whether the function produces the expected result. The most typical test process is as follows:

• Import the function to be tested
• Give the function an input
• Define the desired output
• Check if the function produces the expected output

Generally, it's that simple. Master the following core ideas, writing tests will no longer be scary:

Input -> Expected output -> Assertion result.

Test blocks, assertions and matchers

We will create a simple Javascript function code for the addition of 2 numbers, and write a corresponding Jest-based test for it

const sum = (a, b) => a + b;

Now, for testing, create a test file in the same folder and name it test.spec.js . This special suffix is a convention of Jest and is used to find all test files. We will also import the function under test in order to execute the code under test. The Jest test follows the BDD style test, each test should have a main test test block, and there can be multiple test blocks, now it is sum to write test blocks for the 06125c04aa1aaf method, here we write a test to add 2 numbers and verify expected outcome. We will provide the numbers as 1 and 2, and expect to output 3.

test It requires two parameters: a string used to describe the test block, and a callback function used to wrap the actual test. expect wraps the objective function and combines it with the matcher toBe to check whether the function calculation result meets expectations.

This is the complete test:

test("sum test", () => {
  expect(sum(1, 2)).toBe(3);
});

We observe the above code and find two points:

• test block is a separate test block, which has the function of describing and dividing the scope, that is, it represents the general container for the test sum
• expect is an assertion that uses inputs 1 and 2 to call the sum method in the function under test, and expects an output of 3.
• toBe is a matcher, used to check the expected value, if it does not meet the expected result, it should throw an exception.

How to implement the test block

The test block is actually not complicated. The simplest implementation is as follows. We need to store the callback function of the actual test of the test package, so we encapsulate a dispatch method to receive the command type and the callback function:

const test = (name, fn) => {
  dispatch({ type: "ADD_TEST", fn, name });
};

We need to create a state save the test. The callback function of the test is stored in an array.

global["STATE_SYMBOL"] = {
  testBlock: [],
};

dispatch method only needs to identify the corresponding command at this time, and save the test callback function into the global state .

const dispatch = (event) => {
  const { fn, type, name } = event;
  switch (type) {
    case "ADD_TEST":
      const { testBlock } = global["STATE_SYMBOL"];
      testBlock.push({ fn, name });
      break;
  }
};

How to implement assertions and matchers

The assertion library is also very simple to implement. You only need to encapsulate a function to expose the matcher method to satisfy the following formula:

expect(A).toBe(B)

Here we implement toBe . When the result is not equal to the expectation, just throw an error:

const expect = (actual) => ({
    toBe(expected) {
        if (actual !== expected) {
            throw new Error(`${actual} is not equal to ${expected}`);
        }
    }
};

try/catch will be used to capture errors in the test block and print stack information to locate the problem.

In simple cases, we can also use the assert module that comes with Node to make an assertion. Of course, there are many more complex assertion methods, and the principles are similar in essence.

CLI and configuration

After writing the test, we need to enter the command in the command line to run the single test. Normally, the command is similar to the following:

node jest xxx.spec.js

The essence here is to parse the parameters of the command line.

const testPath = process.argv.slice(2)[0];
const code = fs.readFileSync(path.join(process.cwd(), testPath)).toString();

In complex situations, you may also need to read the parameters of the local Jest configuration file to change the execution environment, etc. Jest uses third-party libraries yargs execa and chalk to parse, execute and print commands here.

simulation

In complex test scenarios, we must not mock a Jest term: simulation (06125c04aa1e9e)

In the Jest documentation, we can find that Jest has the following description of simulation: "The simulation function erases the actual implementation of the function, captures the call to the function, and the parameters passed in these calls, so that the link between the test codes becomes easy"

In short, a simulation can be created by assigning the following code snippets to functions or dependencies:

jest.mock("fs", {
  readFile: jest.fn(() => "wscats"),
});

This is a simple simulation example that simulates the return value of the readFile function of the fs module in testing specific business logic.

How to simulate a function

Next, we will study how to implement it. The first is jest.mock . Its first parameter accepts the module name or module path, and the second parameter is the specific implementation of the module’s external exposure method.

const jest = {
  mock(mockPath, mockExports = {}) {
    const path = require.resolve(mockPath, { paths: ["."] });
    require.cache[path] = {
      id: path,
      filename: path,
      loaded: true,
      exports: mockExports,
    };
  },
};

Our solution is actually test test block above. You only need to find a place to save the specific implementation method, and replace it when you actually use the modified module later, so we save it in require.cache , of course We can also store it in the global state .

jest.fn is not difficult. Here we use a closure mockFn to store the replaced functions and parameters, which is convenient for subsequent test inspections and statistical call data.

const jest = {
  fn(impl = () => {}) {
    const mockFn = (...args) => {
      mockFn.mock.calls.push(args);
      return impl(...args);
    };
    mockFn.originImpl = impl;
    mockFn.mock = { calls: [] };
    return mockFn;
  },
};

Execution environment

Some students may have noticed that in the test framework, we do not need to manually introduce the test , expect and jest . Each test file can be used directly, so we need to create a running environment in which these methods are injected.

Scope isolation

Because the single test file needs scope isolation when running. Therefore, the test engine is designed to run in the global scope of node, and the code of the test file runs in the local scope of the vm virtual machine in the node environment.

• Global scope global
• Local scope context

The two scopes dispatch method.

dispatch collects the test block, life cycle and test report information under the vm local scope to the node global scope STATE_SYMBOL , so dispatch mainly involves the following communication types:

• Test block

  • ADD_TEST

• The life cycle

  • BEFORE_EACH
  • BEFORE_ALL
  • AFTER_EACH
  • AFTER_ALL

• testing report

  • COLLECT_REPORT

V8 virtual machine

Since everything is ready, we only need to inject the methods needed for the test into the V8 virtual machine, that is, inject the local scope of the test.

const context = {
  console: console.Console({ stdout: process.stdout, stderr: process.stderr }),
  jest,
  expect,
  require,
  test: (name, fn) => dispatch({ type: "ADD_TEST", fn, name }),
};

After injecting the scope, we can make the code of the test file run in the V8 virtual machine. The code I passed here is the code that has been processed into a string. Jest will do some code processing, security processing and SourceMap here. For sewing and other operations, our example does not need to be so complicated.

vm.runInContext(code, context);

The time difference can be used to calculate the running time of a single test before and after the code is executed. Jest will also pre-evaluate the size and number of single test files here, and decide whether to enable Worker to optimize the execution speed.

const start = new Date();
const end = new Date();
log("\x1b[32m%s\x1b[0m", `Time: ${end - start} ms`);

Run single test callback

After the execution of the V8 virtual machine is completed, the global state will collect all the packaged test callback functions in the test block. Finally, we only need to traverse all these callback functions and execute them.

testBlock.forEach(async (item) => {
  const { fn, name } = item;
  await fn.apply(this);
});

Hook function

We can also add a life cycle in the single test execution process, such as beforeEach , afterEach , afterAll and beforeAll hook functions.

Adding the hook function to the above infrastructure is actually injecting the corresponding callback function in each process of executing the test. For example, beforeEach is placed before the testBlock traversal execution test function, and afterEach is placed after the testBlock traversal execution test function. Simple, just need to put the right position to expose the hook function of any period.

testBlock.forEach(async (item) => {
  const { fn, name } = item;
  beforeEachBlock.forEach(async (beforeEach) => await beforeEach());
  await fn.apply(this);
  afterEachBlock.forEach(async (afterEach) => await afterEach());
});

beforeAll and afterAll can be placed before and after all the tests of testBlock

beforeAllBlock.forEach(async (beforeAll) => await beforeAll());
testBlock.forEach(async (item) => {})
afterAllBlock.forEach(async (afterAll) => await afterAll());

Generate report

After the single test is executed, the information set of success and capture errors can be collected,

try {
    dispatch({ type: "COLLECT_REPORT", name, pass: 1 });
    log("\x1b[32m%s\x1b[0m", `√ ${name} passed`);
} catch (error) {
    dispatch({ type: "COLLECT_REPORT", name, pass: 0 });
    log("\x1b[32m%s\x1b[0m", `× ${name} error`);
}

Then hijack log , let the detailed results be printed on the terminal, or generate reports locally with the IO module.

const { reports } = global["STATE_SYMBOL"];
const pass = reports.reduce((pre, next) => pre.pass + next.pass);
log("\x1b[32m%s\x1b[0m", `All Tests: ${pass}/${reports.length} passed`);

So far, we have implemented the core part of a simple Jest test framework. The above parts basically implement test blocks, assertions, matchers, CLI configuration, function simulation, use of virtual machines and scope and life cycle hook functions. We can On this basis, enrich the assertion method, matcher and support parameter configuration. Of course, the actual implementation of Jest will be more complicated. I only refined the more critical parts, so I attached my personal notes of reading the Jest source code for your reference.

jest-cli

Download Jest source code and execute it in the root directory

yarn
npm run build

It essentially runs two files build.js and buildTs.js in the script folder:

"scripts": {
    "build": "yarn build:js && yarn build:ts",
    "build:js": "node ./scripts/build.js",
    "build:ts": "node ./scripts/buildTs.js",
}

build.js essentially uses the babel library. Create a new build folder in the package/xxx package, and then use transformFileSync to generate the file into the build folder:

const transformed = babel.transformFileSync(file, options).code;

And buildTs.js essentially uses the tsc command to compile the ts file into the build folder, and use the execa library to execute the command:

const args = ["tsc", "-b", ...packagesWithTs, ...process.argv.slice(2)];
await execa("yarn", args, { stdio: "inherit" });

The successful execution will display as follows, it will help you compile all files js files and ts files under the packages folder to the build folder of the directory where you are:

Next we can start the jest command:

npm run jest
# 等价于
# node ./packages/jest-cli/bin/jest.js

Here you can do analysis processing according to the different parameters passed in, such as:

npm run jest -h
node ./packages/jest-cli/bin/jest.js /path/test.spec.js

It will execute the jest.js file, and then enter build/cli file. The run method will parse various parameters in the command. The specific principle is that the yargs library cooperates with process.argv to achieve

const importLocal = require("import-local");

if (!importLocal(__filename)) {
  if (process.env.NODE_ENV == null) {
    process.env.NODE_ENV = "test";
  }

  require("../build/cli").run();
}

jest-config

After obtaining a variety of command parameters will be executed runCLI method core, which is @jest/core -> packages/jest-core/src/cli/index.ts core method library.

import { runCLI } from "@jest/core";
const outputStream = argv.json || argv.useStderr ? process.stderr : process.stdout;
const { results, globalConfig } = await runCLI(argv, projects);

runCLI method will use the incoming parameter argv parsed in the command just now to read the configuration file information readConfigs readConfigs comes from packages/jest-config/src/index.ts . There will be normalize to fill in and initialize some default configured parameters. Its default parameters are in packages/jest-config/src/Defaults.ts is recorded in the 06125c04aa2760 file. For example, if you only run js single test, require.resolve('jest-runner') will be set as the runner for running single test by default, and the outputStream output content will be generated to the console in cooperation with the chalk library.

By the way, here is the principle of introducing jest into the module. Here, we will require.resolve(moduleName) find the path of the module by 06125c04aa2790, and save the path in the configuration, and then use the requireOrImportModule packages/jest-util/src/requireOrImportModule.ts to call the encapsulated native import/reqiure method to match the path in the configuration file Take out the module.

• globalConfig configuration from argv
• configs from the configuration of jest.config.js

const { globalConfig, configs, hasDeprecationWarnings } = await readConfigs(
  argv,
  projects
);

if (argv.debug) {
  /*code*/
}
if (argv.showConfig) {
  /*code*/
}
if (argv.clearCache) {
  /*code*/
}
if (argv.selectProjects) {
  /*code*/
}

jest-haste-map

jest-haste-map is used to get all the files in the project and the dependencies between them. It import/require call, extracting them from each file and building a map that contains each file and its Dependencies. Haste here is the module system used by Facebook. It also has something called HasteContext because it has HasteFS (Haste File System). HastFS is just a list of files in the system and all dependencies associated with it. It is A map data structure in which the key is the path and the value is the metadata. The contexts generated here will be used until the onRunComplete stage.

const { contexts, hasteMapInstances } = await buildContextsAndHasteMaps(
  configs,
  globalConfig,
  outputStream
);

jest-runner

_run10000 method will obtain contexts according to the configuration information globalConfig and configs . contexts will store the configuration information and path of each local file, and then will bring the callback function onComplete , the global configuration globalConfig and the scope contexts into the runWithoutWatch method.

Next, you will enter the runJest packages/jest-core/src/runJest.ts file. Here, the passed contexts will be used to traverse all the unit tests and save them in an array.

let allTests: Array<Test> = [];
contexts.map(async (context, index) => {
  const searchSource = searchSources[index];
  const matches = await getTestPaths(
    globalConfig,
    searchSource,
    outputStream,
    changedFilesPromise && (await changedFilesPromise),
    jestHooks,
    filter
  );
  allTests = allTests.concat(matches.tests);
  return { context, matches };
});

And use the Sequencer method to sort the single tests

const Sequencer: typeof TestSequencer = await requireOrImportModule(
  globalConfig.testSequencer
);
const sequencer = new Sequencer();
allTests = await sequencer.sort(allTests);

runJest method will call a key method packages/jest-core/src/TestScheduler.ts of the scheduleTests method.

const results = await new TestScheduler(
  globalConfig,
  { startRun },
  testSchedulerContext
).scheduleTests(allTests, testWatcher);

scheduleTests method will do a lot of things, will allTests in contexts collected contexts in the duration collected timings array and subscribe to four life cycle before executing all single measurements:

• test-file-start
• test-file-success
• test-file-failure
• test-case-result

Then the contexts through and with a new empty object testRunners do some processing save up, which calls @jest/transform provided createScriptTransformer method introduced by the processing module.

import { createScriptTransformer } from "@jest/transform";

const transformer = await createScriptTransformer(config);
const Runner: typeof TestRunner = interopRequireDefault(
  transformer.requireAndTranspileModule(config.runner)
).default;
const runner = new Runner(this._globalConfig, {
  changedFiles: this._context?.changedFiles,
  sourcesRelatedToTestsInChangedFiles: this._context?.sourcesRelatedToTestsInChangedFiles,
});
testRunners[config.runner] = runner;

The scheduleTests method calls packages/jest-runner/src/index.ts of runTests method.

async runTests(tests, watcher, onStart, onResult, onFailure, options) {
  return await (options.serial
    ? this._createInBandTestRun(tests, watcher, onStart, onResult, onFailure)
    : this._createParallelTestRun(
        tests,
        watcher,
        onStart,
        onResult,
        onFailure
      ));
}

In the final _createParallelTestRun or _createInBandTestRun method:

• _createParallelTestRun

There will be a runTestInWorker , which, as the name suggests, is to perform a single test in the worker.

• _createInBandTestRun will execute packages/jest-runner/src/runTest.ts a core method runTest , and runJest will execute a method runTestInternal , which will prepare a lot of things before executing a single test, involving global method rewriting and hijacking of import and export methods.

await this.eventEmitter.emit("test-file-start", [test]);
return runTest(
  test.path,
  this._globalConfig,
  test.context.config,
  test.context.resolver,
  this._context,
  sendMessageToJest
);

In the runTestInternal fs module will be used to read the content of the file and put it into cacheFS , which will be cached for quick reading later. For example, if the content of the file is json later, it can be cacheFS Date.now will also be used to calculate the time-consuming time. .

const testSource = fs().readFileSync(path, "utf8");
const cacheFS = new Map([[path, testSource]]);

In the runTestInternal method will introduce packages/jest-runtime/src/index.ts , it will help you cache the module and read the module and trigger the execution.

const runtime = new Runtime(
  config,
  environment,
  resolver,
  transformer,
  cacheFS,
  {
    changedFiles: context?.changedFiles,
    collectCoverage: globalConfig.collectCoverage,
    collectCoverageFrom: globalConfig.collectCoverageFrom,
    collectCoverageOnlyFrom: globalConfig.collectCoverageOnlyFrom,
    coverageProvider: globalConfig.coverageProvider,
    sourcesRelatedToTestsInChangedFiles: context?.sourcesRelatedToTestsInChangedFiles,
  },
  path
);

jest-environment-node

@jest/console package is used here to rewrite the global console. In order for the console of the single test file code block to print the results on the node terminal smoothly, in conjunction with the jest-environment-node package, environment.global facilitate subsequent methods to obtain these scopes in the vm. In essence, it is the scope provided for the vm operating environment, global global methods involved in the rewriting are as follows:

• global.global
• global.clearInterval
• global.clearTimeout
• global.setInterval
• global.setTimeout
• global.Buffer
• global.setImmediate
• global.clearImmediate
• global.Uint8Array
• global.TextEncoder
• global.TextDecoder
• global.queueMicrotask
• global.AbortController

testConsole is essentially rewritten using node's console to facilitate subsequent coverage of the console method in the vm scope

testConsole = new BufferedConsole();
const environment = new TestEnvironment(config, {
  console: testConsole,
  docblockPragmas,
  testPath: path,
});
// 真正改写 console 地方的方法
setGlobal(environment.global, "console", testConsole);

runtime mainly uses these two methods to load the module, first judge whether it is an ESM module, if it is, use runtime.unstable_importModule load the module and run the module, if not, use runtime.requireModule load the module and run the module.

const esm = runtime.unstable_shouldLoadAsEsm(path);

if (esm) {
  await runtime.unstable_importModule(path);
} else {
  runtime.requireModule(path);
}

jest-circus

Immediately after the runTestInternal in testFramework will accept the incoming runtime to call the single test file to run, the testFramework method comes from a library with a more interesting name packages/jest-circus/src/legacy-code-todo-rewrite/jestAdapter.ts , where legacy-code-todo-rewrite means legacy code to-do items rewrite , jest-circus mainly global Some methods of rewriting, involving these few:

• afterAll
• afterEach
• beforeAll
• beforeEach
• describe
• it
• test

Here call will be before a single test jestAdapter function, that is, the above-mentioned runtime.requireModule load xxx.spec.js files have been used before to perform here initialize preset the execution environment globals and snapshotState , and rewrite beforeEach , if configured resetModules , clearMocks , resetMocks , restoreMocks and setupFilesAfterEnv will execute the following methods respectively:

• runtime.resetModules
• runtime.clearAllMocks
• runtime.resetAllMocks
• runtime.restoreAllMocks
• runtime.requireModule or runtime.unstable_importModule

When the initialize method is initialized, because initialize rewritten the global describe and test , these methods are /packages/jest-circus/src/index.ts Here, test note that there is a dispatchSync method in the 06125c04ab63c6 method, which is a key method for global maintenance. A copy of state , dispatchSync is to test the functions and other information in state , dispatchSync uses name eventHandler method to modify state , this idea is very similar to the data flow in redux.

const test: Global.It = () => {
  return (test = (testName, fn, timeout) => (testName, mode, fn, testFn, timeout) => {
    return dispatchSync({
      asyncError,
      fn,
      mode,
      name: "add_test",
      testName,
      timeout,
    });
  });
};

The single test xxx.spec.js , that is, the testPath file will initialize . Note that the introduction here will execute this single test. Since the single test xxx.spec.js file is written according to the specification, there will be test and describe , so all at this time The test and describe will be stored in the global state .

const esm = runtime.unstable_shouldLoadAsEsm(testPath);
if (esm) {
  await runtime.unstable_importModule(testPath);
} else {
  runtime.requireModule(testPath);
}

jest-runtime

Here, we will first judge whether it is an esm module, if it is, use unstable_importModule to import it, otherwise use requireModule to import it, specifically will it enter the following function.

this._loadModule(localModule, from, moduleName, modulePath, options, moduleRegistry);

The logic of \_loadModule has only three main parts

• Judge whether it is a json suffix file, execute readFile to read the text, and use transformJson and JSON.parse to transform the output content.
• Judge whether it is a node suffix file, and execute the require native method to import the module.
• For files that do not meet the above two conditions, execute the \_execModule execution module.

\ _execModule babel will be used to transform fs to read the source code, the transformFile is packages/jest-runtime/src/index.ts the transform method.

const transformedCode = this.transformFile(filename, options);

\_execModule will use the createScriptFromCode method to call node's native vm module to actually execute js. The vm module accepts safe source code, and uses the V8 virtual machine with the incoming context to execute the code immediately or delay the execution of the code. Here you can accept different Scope to execute the same code to calculate different results, which is very suitable for the use of test frameworks. The injected vmContext here is the above global rewrite scope including afterAll, afterEach, beforeAll, beforeEach, describe, it, test, so we The single test code will get these methods with injection scope when it runs.

const vm = require("vm");
const script = new vm().Script(scriptSourceCode, option);
const filename = module.filename;
const vmContext = this._environment.getVmContext();
script.runInContext(vmContext, {
  filename,
});

When rewriting the above global methods and preservation state later, will enter into the real implementation describe logic callback function inside, in packages/jest-circus/src/run.ts the run inside method used here getState method to describe code blocks taken out, then _runTestsForDescribeBlock perform this function, and then Enter the _runTest method, and then use _callCircusHook execute the before and after hook functions, and use _callCircusTest execute.

const run = async (): Promise<Circus.RunResult> => {
  const { rootDescribeBlock } = getState();
  await dispatch({ name: "run_start" });
  await _runTestsForDescribeBlock(rootDescribeBlock);
  await dispatch({ name: "run_finish" });
  return makeRunResult(getState().rootDescribeBlock, getState().unhandledErrors);
};

const _runTest = async (test, parentSkipped) => {
  // beforeEach
  // test 函数块，testContext 作用域
  await _callCircusTest(test, testContext);
  // afterEach
};

This is the core position of the hook function implementation and also the core element of the Jest function.

finally

I hope this article can help you understand the core implementation and principles of the Jest testing framework. Thank you for reading patiently. If the articles and notes can bring you a hint of help or inspiration, please don’t be stingy with your Star and Fork. The articles are continuously updated synchronously, your affirmation Is my biggest motivation to move forward😁

• https://github.com/Wscats/jest-tutorial