Webpack principle series 9: Tree-Shaking realization principle

1. What is Tree Shaking

Tree-Shaking is a Dead Code Elimination technology based on the ES Module specification. It will statically analyze the import and export between modules during operation, determine which exported values in the ESM module have not been used by other modules, and delete them. Realize the optimization of packaged products.

Tree Shaking was first implemented by Rich Harris in Rollup earlier. Webpack has been connected since version 2.0 and has become a widely used performance optimization method.

1.1 Start Tree Shaking in Webpack

In Webpack, three conditions must be met at the same time to start the Tree Shaking function:

• Use ESM specification to write module code
• Configure optimization.usedExports to true , start the marking function

• Start the code optimization function, which can be achieved in the following ways:

  • Configuration mode = production
  • Configuration optimization.minimize = true
  • Provide optimization.minimizer array

E.g:

// webpack.config.js
module.exports = {
  entry: "./src/index",
  mode: "production",
  devtool: false,
  optimization: {
    usedExports: true,
  },
};

1.2 Theoretical basis

In the old version of JavaScript modularization schemes such as CommonJs, AMD, CMD, the import and export behavior is highly dynamic and unpredictable, for example:

if(process.env.NODE_ENV === 'development'){
  require('./bar');
  exports.foo = 'foo';
}

The ESM scheme circumvents this behavior from the specification level. It requires that all import and export statements can only appear at the top level of the module, and the imported and exported module names must be string constants, which means that the following codes are illegal under the ESM scheme of:

if(process.env.NODE_ENV === 'development'){
  import bar from 'bar';
  export const foo = 'foo';
}

Therefore, the dependencies between modules under ESM are highly deterministic and have nothing to do with the running state. The compilation tool only needs to perform static analysis on the ESM module, and it can infer from the code literal which module values have not been used by other modules. It is a necessary condition to realize the Tree Shaking technology.

1.3 Example

For the following code:

// index.js
import {bar} from './bar';
console.log(bar);

// bar.js
export const bar = 'bar';
export const foo = 'foo';

In the example, the bar.js module exports bar and foo , but only the bar export value is used by other modules. After the Tree Shaking process, the foo variable will be deleted as useless code.

Second, the realization principle

Webpack in, Tree-shaking realization First First mark a module export value which is not used, the second is to use Terser delete these export statements have not been used. The marking process can be roughly divided into three steps:

• In the Make phase, collect the module export variables and record them in the ModuleGraph variable of the module dependency graph
• In the Seal phase, traverse the ModuleGraph to mark whether the exported variables of the module are used
• When the product is generated, if the variable is not used by other modules, delete the corresponding export statement

The marking function needs to be configured with optimization.usedExports = true turned on

In other words, the effect of marking is to delete export statements that are not used by other modules, such as:

In the example, the bar.js module (second from left) exports two variables: bar and foo , of which foo not used by other modules, so after marking, the export statement corresponding to the foo . In contrast, if the marking function is not enabled ( optimization.usedExports = false ), the export statement will be retained regardless of whether the variable is used or not, as shown in the product code on the second right of the figure above.

Note that at this time, the code const foo='foo' foo variable is still intact. This is because the marking function will only affect the export statement of the module. The Terser plug-in is the one Shaking For example, in the above example, the foo variable has become a Dead Code after being marked-code that cannot be executed. At this time, you only need to use the DCE function provided by Terser to delete this definition statement to achieve a complete Tree Shaking effect.

Next, I will expand the source code of the marking process and explain in detail the implementation process of Tree Shaking in Webpack 5. Students who are not interested in the source code can skip to the next chapter.

2.1 Collect module export

First of all, Webpack needs to figure out what export values each module has. This process occurs in the make phase. The general process is:

For more instructions on the Make stage, please refer to the previous article [Summary of Ten Thousand Words] to understand the core principle of .

1. Convert all ESM export statements of the module to Dependency objects, and record them to the dependencies module object, the conversion rules:

• Named exports are converted to HarmonyExportSpecifierDependency objects
• default export is converted to HarmonyExportExpressionDependency object

For example, for the following module:

export const bar = 'bar';
export const foo = 'foo';

export default 'foo-bar'

The corresponding dependencies value is:

2. After all modules are compiled, the compilation.hooks.finishModules hook is triggered, and the FlagDependencyExportsPlugin plug-in callback is executed.
3. FlagDependencyExportsPlugin plug-in reads the module information stored in module objects
4. Traverse the dependencies array of the module object, find all dependent objects of type HarmonyExportXXXDependency ExportInfo objects and record them in the ModuleGraph system

After processing by the FlagDependencyExportsPlugin plug-in, all ESM-style export statements will be recorded in the ModuleGraph system, and subsequent operations can directly read the module's export value from ModuleGraph.

Reference materials:

1. [Summary of 4D Characters] One article thoroughly understands the core principles of
2. bit difficult webpack knowledge point: Dependency Graph in-depth analysis

2.2 Export of markup modules

After the module export information is collected, Webpack needs to mark out which export values are used by other modules and which are not in the export list of each module. This process occurs in the Seal phase. The main process:

1. Trigger the compilation.hooks.optimizeDependencies hook and start executing the FlagDependencyUsagePlugin plug-in logic
2. In the FlagDependencyUsagePlugin plug-in, gradually traverse all module objects stored in ModuleGraph from entry
3. Traversing module corresponding object exportInfo array
4. Execute the compilation.getDependencyReferencedExports method for each exportInfo object to determine whether the corresponding dependency object is used by other modules
5. Export values used by any module, call the exportInfo.setUsedConditionally method to mark them as used.
6. exportInfo.setUsedConditionally internally modify the exportInfo._usedInRuntime record how the export is used
7. Finish

The above is an extremely simplified version. There are still a lot of branch logic and complex collection operations in the middle. Let's grasp the key point: the operation of marking module export is concentrated in the FlagDependencyUsagePlugin plug-in, and the execution result will eventually be recorded in the corresponding module export statement. exportInfo._usedInRuntime dictionary.

2.3 Generate code

After the previous collection and marking steps, Webpack has clearly recorded in the ModuleGraph system which values are exported by each module, and each exported value is not used by that module. Next, Webpack will generate different codes based on the usage of the exported values, for example:

Focus on the bar.js file, which is also the export value. bar is used by the index.js module, so the corresponding __webpack_require__.d call "bar": ()=>(/* binding */ bar) generated. As a comparison, foo only retains the definition statement and does not generate the corresponding export in the chunk.

For the content of Webpack products and __webpack_require__.d method, please refer to the 1616e2db7989d6 Webpack Principle Series 6: Thorough Understanding of Webpack Runtime .

This section of generation logic is implemented by the HarmonyExportXXXDependency class corresponding to the export statement. The general process is:

1. In the packaging phase, call the HarmonyExportXXXDependency.Template.apply method to generate code
2. In the apply method, read the exportsInfo determine which derived values are used and which are not used
3. HarmonyExportInitFragment objects for the exported values that have been used and those that have not been used, and save them to the initFragments array
4. Traverse the initFragments array to generate the final result

Basically, the logic of this step is to generate export statements using the exported values of the exportsInfo

2.4 Delete Dead Code

After the previous steps, the unused values in the module export list will not be defined in the __webpack_exports__ object, forming a Dead Code effect that cannot be executed, such as the foo variable in the above example:

After that, DCE tools such as Terser and UglifyJS will "shake" this invalid code to form a complete Tree Shaking operation.

2.5 Summary

In summary, the implementation of Tree Shaking in Webpack is divided into the following steps:

• In FlagDependencyExportsPlugin plug-in module according to dependencies list collection module derived value, and recorded ModuleGraph system exportsInfo in
• Collect the usage of the exported value of the module in the FlagDependencyUsagePlugin exportInfo._usedInRuntime collection
• In the HarmonyExportXXXDependency.Template.apply method, different export statements are generated according to the usage of the export value
• Use the DCE tool to delete Dead Code to achieve a complete tree shake effect

The above implementation principles require high background knowledge. It is recommended that readers synchronize with the following documents to consume:

1. [Summary of 4D Characters] One article thoroughly understands the core principles of
2. bit difficult webpack knowledge point: Dependency Graph in-depth analysis
3. Webpack principle series 6: Thorough understanding of Webpack runtime

Three, best practices

Although Webpack has natively supported the Tree Shaking function since 2.x, it is limited by the dynamic characteristics of JS and the complexity of the module. Until the latest version 5.0, it still has not solved the problems caused by many code side effects, making the optimization effect not as good as Tree. Shaking was originally supposed to be so perfect, so users need to consciously optimize the code structure, or use some patching techniques to help Webpack more accurately detect invalid code and complete the Tree Shaking operation.

3.1 Avoid meaningless assignments

When using Webpack, you need to consciously avoid some unnecessary assignment operations. Observe the following sample code:

Example, index.js module references bar.js module foo and assigned to f variables, but the follow-up did not continue to use foo or f variables, this scenario bar.js module exports foo values not actually being used, should be deleted, But Webpack's Tree Shaking operation did not take effect, and the foo export is still retained in the product:

The shallow reason for this result is that Webpack's Tree Shaking logic stays at the level of static code analysis, just a simple judgment:

• Whether module export variables are referenced by other modules
• Does this variable appear in the main code of the reference module

Without going further, it is semantically analyzed whether the value derived from the module is actually used effectively.

The deeper reason is that the assignment statement of JavaScript is not pure . Depending on the specific scenario, it may have unexpected side effects, such as:

import { bar, foo } from "./bar";

let count = 0;

const mock = {}

Object.defineProperty(mock, 'f', {
    set(v) {
        mock._f = v;
        count += 1;
    }
})

mock.f = foo;

console.log(count);

In the example, the Object.defineProperty mock object causes the mock.f = foo assignment statement to count variable. In this scenario, even using complex dynamic semantic analysis, it is difficult to perfectly Shaking out all useless while ensuring the correct side effects. Code branches and leaves.

Therefore, when using Webpack, developers need to consciously avoid these meaningless repeated assignment operations.

3.3 Use #pure label pure function calls

Similar to assignment statements, function call statements in JavaScript may also have side effects, so by default Webpack does not perform Tree Shaking operations on function calls. However, developers can add /*#__PURE__*/ remarks before the call statement to clearly tell Webpack that this function call will not have side effects on the context, for example:

In the example, the foo('be retained') call does not carry the /*#__PURE__*/ note, and the code is retained; for comparison, the foo('be removed') is deleted by Tree Shaking after the Pure statement is brought.

3.3 Forbid Babel to translate module import and export statements

Babel is a very popular JavaScript code converter, which can equivalently translate high version JS code into lower version code with better compatibility, enabling front-end developers to use the latest language features to develop compatibility with older browsers Code.

However, some of the features provided by Babel will cause the Tree Shaking function to fail. For example, Babel can import/export style ESM statements into CommonJS style modular statements, but this feature causes Webpack to be unable to import and export the translated modules. Static analysis, example:

The example uses babel-loader process the *.js file, and sets the Babel configuration item modules = 'commonjs' to translate the modular scheme from ESM to CommonJS, resulting in the translation code (the upper one on the right) not correctly marking the unused derived value foo . For comparison, the figure 2 on the right modules = false foo . At this time, the 0616e2db798e64 variable is correctly marked as Dead Code.

Therefore, when using babel-loader , it is recommended to babel-preset-env the moduels configuration item of false to 0616e2db798e82, and turn off the translation of module import and export statements.

3.4 Optimize the granularity of exported values

The Tree Shaking logic acts on the export statement of ESM, so for the following export scenarios:

export default {
    bar: 'bar',
    foo: 'foo'
}

Even if only one of the attributes of the default default object will still be kept intact. Therefore, in actual development, you should try to maintain the granularity and atomicity of the exported value. The optimized version of the above example code:

const bar = 'bar'
const foo = 'foo'

export {
    bar,
    foo
}

3.5 Use packages that support Tree Shaking

If possible, try to use npm packages that support Tree Shaking, for example:

• Use lodash-es instead of lodash , or use babel-plugin-lodash achieve similar effects

However, not all npm packages have Tree Shaking space. Frameworks such as React and Vue2 have already optimized the production version to the extreme. At this time, the business code needs the complete functions provided by the entire code package, basically not Too much need for Tree Shaking.