The fifth part of the Webpack series: A thorough understanding of the Webpack runtime

The full text is 5000 words, in-depth analysis of the content, structure and generation principle of Webpack runtime, welcome to like and pay attention. Writing is not easy, and reprinting in any form is prohibited without the author's consent! ! !

background

In the previous article bit difficult webpack knowledge point: Chunk subcontracting rules in detail , we explained in detail the default subcontracting rules of Webpack, as well as part of the execution logic of the seal phase. Now we will follow the execution process of Webpack and continue. Analyze the implementation principle in depth, and the specific content includes:

• What is included in the build product of Webpack? How does the product support features such as modularity, asynchronous loading, and HMR?
• What is runtime? How to collect runtime dependencies during the Webpack build process? How to merge the runtime and business code to bundle ?

In fact, this article and the previous articles of principle nature may not be able to immediately solve the practical problems you may be facing in your business, but in a longer time dimension, the knowledge, thinking, and speculative process presented in these articles may Can give you in the long run:

• Ability to analyze and understand complex open source code
• Understand the Webpack architecture and implementation details, and quickly locate the root cause based on the appearance next time you encounter a problem
• Understand the context provided by Webpack for hooks and loaders, be able to understand other open source components more smoothly, and even be able to implement your own components freely

Therefore, I hope that interested students can persist, and I will output a lot of articles about the principles of Webpack implementation in the future! If you happen to also want to improve your knowledge reserve in Webpack, follow me and we will learn together!

Compilation product analysis

In order to run business projects normally and correctly, Webpack needs to package the business code written by the developer and the runtime supports and deploys these business codes into the product (bundle). If the building is used as an analogy, the business code is equivalent to a brick Tile cement is a logic that can be seen, touched and directly sensed; it is equivalent to a reinforced foundation buried under bricks and tiles during operation. It usually does not pay attention to but determines the function and quality of the entire building.

Most Webpack features require a specific steel foundation to run, such as:

• Asynchronous on-demand loading
• HMR
• WASM
• Module Federation

Let's start with the simplest example, and gradually expand to understand the Webpack runtime code under each feature.

basic structure

Let’s start with the simplest example, for the following code structure:

// a.js
export default 'a module';

// index.js
import name from './a'
console.log(name)

Use the following configuration:

module.exports = {
  entry: "./src/index",
  mode: "development",
  devtool: false,
  output: {
    filename: "[name].js",
    path: path.join(__dirname, "./dist"),
  },
};

The content of the configuration is relatively simple, so I won't go into it, and just look at the result of the compilation:

Although it looks very non-mainstream, careful analysis can still disassemble the code context. The bundle as a whole is wrapped by an IIFE, and the contents are as follows from top to bottom:

• __webpack_modules__ object contains all modules except the entrance. In the example, it is the a.js module
• __webpack_module_cache__ object, used to store the referenced module
• __webpack_require__ function to implement module reference (require) logic
• __webpack_require__.d , a tool function, which implements the module object that appends the exported content of the module
• __webpack_require__.o , a tool function, used to judge object attributes
• __webpack_require__.r , utility function, declare ESM module ID in ESM mode
• The last IIFE, corresponding to the entry module, index.js above example, is used to start the entire application

These __webpack_ beginning weird Webpack functions collectively referred to as run-time code, such as the previously mentioned effect of the entire business project is put skeleton terms of the above simple examples set out several functions, objects, are Collaborate to build a simple modular system to realize the modular features declared by the ES Module specification.

The final function in the above example is __webpack_require__ , which implements the inter-module reference function, the core code:

function __webpack_require__(moduleId) {
    /******/ // 如果模块被引用过
    /******/ var cachedModule = __webpack_module_cache__[moduleId];
    /******/ if (cachedModule !== undefined) {
      /******/ return cachedModule.exports;
      /******/
    }
    /******/ // Create a new module (and put it into the cache)
    /******/ var module = (__webpack_module_cache__[moduleId] = {
      /******/ // no module.id needed
      /******/ // no module.loaded needed
      /******/ exports: {},
      /******/
    });
    /******/
    /******/ // Execute the module function
    /******/ __webpack_modules__[moduleId](
      module,
      module.exports,
      __webpack_require__
    );
    /******/
    /******/ // Return the exports of the module
    /******/ return module.exports;
    /******/
  }

It can be inferred from the code that its function:

• Find the corresponding module code according to the moduleId parameter, execute it and return the result
• If moduleId has been referenced, the exported content __webpack_module_cache__ cache object will be directly returned to avoid repeated execution

Among them, the business module code is stored in the __webpack_modules__ at the beginning of the bundle, and the content is as follows:

var __webpack_modules__ = {
    "./src/a.js": (
        __unused_webpack_module,
        __webpack_exports__,
        __webpack_require__
      ) => {
        // ...​
      },
  };

Combining the __webpack_require__ function and the __webpack_modules__ variable can correctly reference the code module, such as the IIFE at the end of the generated code in the above example:

(() => {
    /*!**********************!*\
  !*** ./src/index.js ***!
  \**********************/
    /* harmony import */ var _a__WEBPACK_IMPORTED_MODULE_0__ =
      __webpack_require__(/*! ./a */ "./src/a.js");

    console.log(_a__WEBPACK_IMPORTED_MODULE_0__.name);
  })();

These functions and objects constitute the most basic ability of Webpack runtime-modularity. We will put them in the second section of the article "Implementation Principles" for their generation rules and principles. Let's continue to look at asynchronous module loading and modules. The corresponding runtime content in the hot update scenario.

Asynchronous module loading

Let's look at a simple asynchronous module loading example:

// ./src/a.js
export default "module-a"

// ./src/index.js
import('./a').then(console.log)

The Webpack configuration is similar to the previous example:

module.exports = {
  entry: "./src/index",
  mode: "development",
  devtool: false,
  output: {
    filename: "[name].js",
    path: path.join(__dirname, "./dist"),
  },
};

The generated code is too long and will not be posted. Compared with the modular function shown in the initial basic structure example, when using the asynchronous module loading feature, the following runtime will be added:

• __webpack_require__.e : logically wraps a layer of middleware pattern and promise.all , used to load multiple modules asynchronously
• __webpack_require__.f : __webpack_require__.e . For example, when using the Module Federation feature, you need to register the middleware here to modify the execution logic of the e function
• __webpack_require__.u : A function for splicing asynchronous module names
• __webpack_require__.l : Asynchronous module loading function based on JSONP implementation
• __webpack_require__.p : The full URL of the current file, which can be used to calculate the actual URL of the asynchronous module

It is recommended that readers run the examples to compare the actual generated code and feel their specific functions. These runtime modules build up the asynchronous loading capabilities of Webpack, the core of which is the __webpack_require__.e function. Its code is very simple:

__webpack_require__.f = {};
/******/    // This file contains only the entry chunk.
/******/    // The chunk loading function for additional chunks
/******/    __webpack_require__.e = (chunkId) => {
/******/      return Promise.all(Object.keys(__webpack_require__.f).reduce((promises, key) => {
/******/        __webpack_require__.f[key](chunkId, promises);
/******/        return promises;
/******/      }, []));
/******/    };

From the code point of view, only a set of middleware mode __webpack_require__.f Promise.all . The actual loading work is __webpack_require__.f.j and __webpack_require__.l . Look at the two functions separately:

/******/  __webpack_require__.f.j = (chunkId, promises) => {
/******/        // JSONP chunk loading for javascript
/******/        var installedChunkData = __webpack_require__.o(installedChunks, chunkId) ? installedChunks[chunkId] : undefined;
/******/        if(installedChunkData !== 0) { // 0 means "already installed".
/******/    
/******/          // a Promise means "currently loading".
/******/          if(installedChunkData) {
/******/            promises.push(installedChunkData[2]);
/******/          } else {
/******/            if(true) { // all chunks have JS
/******/              // ...
/******/              // start chunk loading
/******/              var url = __webpack_require__.p + __webpack_require__.u(chunkId);
/******/              // create error before stack unwound to get useful stacktrace later
/******/              var error = new Error();
/******/              var loadingEnded = ...;
/******/              __webpack_require__.l(url, loadingEnded, "chunk-" + chunkId, chunkId);
/******/            } else installedChunks[chunkId] = 0;
/******/          }
/******/        }
/******/    };

__webpack_require__.f.j implements the chunk of the splicing, caching, and exception handling __webpack_require__.l function:

/******/    var inProgress = {};
/******/    // data-webpack is not used as build has no uniqueName
/******/    // loadScript function to load a script via script tag
/******/    __webpack_require__.l = (url, done, key, chunkId) => {
/******/      if(inProgress[url]) { inProgress[url].push(done); return; }
/******/      var script, needAttach;
/******/      if(key !== undefined) {
/******/        var scripts = document.getElementsByTagName("script");
/******/        /​/ ...
/******/      }
/******/      //​ ...
/******/      inProgress[url] = [done];
/******/      var onScriptComplete = (prev, event) => {
/******/        //​ ...
/******/      }
/******/      ;
/******/      var timeout = setTimeout(onScriptComplete.bind(null, undefined, { type: 'timeout', target: script }), 120000);
/******/      script.onerror = onScriptComplete.bind(null, script.onerror);
/******/      script.onload = onScriptComplete.bind(null, script.onload);
/******/      needAttach && document.head.appendChild(script);
/******/    };

__webpack_require__.l implements the loading and execution of asynchronous chunk content through script.

e + l + f.j three runtime functions of 060b6214b9caf1 support the ability of Webpack asynchronous module to run. In actual usage, only need to call the e function to complete the asynchronous module loading and running. For example, the above example corresponds to the generated entry content:

/*!**********************!*\
  !*** ./src/index.js ***!
  \**********************/
__webpack_require__.e(/*! import() */ "src_a_js").then(__webpack_require__.bind(__webpack_require__, /*! ./a */ "./src/a.js"))

Module hot update

Module hot update-HMR is an ability that can significantly improve development efficiency. When the module code changes, it can compile the module separately and transmit the latest compilation result to the browser, and the browser can replace it with the new module code Remove the old code, so as to achieve the module-level code hot replacement capability. Falling to the final experience, after the developer starts Webpack, there is no need to manually refresh the browser page in the process of writing and modifying the code, and all changes can be synchronously presented to the page in real time.

In terms of implementation, the implementation link of HMR is very long and interesting. We will open a separate article to discuss in the follow-up. This article mainly focuses on the runtime code brought into the HMR feature. Some special configuration items are needed to start the HMR capability:

module.exports = {
  entry: "./src/index",
  mode: "development",
  devtool: false,
  output: {
    filename: "[name].js",
    path: path.join(__dirname, "./dist"),
  },
  // 简单起见，这里使用 HtmlWebpackPlugin 插件自动生成作为 host 的 html 文件
  plugins: [
    new HtmlWebpackPlugin({
      title: "Hot Module Replacement",
    }),
  ],
  // 配置 devServer 属性，启动 HMR
  devServer: {
    contentBase: "./dist",
    hot: true,
    writeToDisk: true,
  },

According to the above configuration, use the command webpack serve --hot-only start Webpack, you can find the product in the dist folder:

Compared with the previous two examples, the runtime code generated by HMR reaches 1.5w+ lines, which can be described as a burst. The main runtime contents are:

• webpack-dev-server , webpack/hot/xxx , querystring and other frameworks required by HMR. This part accounts for most of the code
• __webpack_require__.l : Same as asynchronous module loading, asynchronous module loading function based on JSONP implementation
• __webpack_require__.e : Same as asynchronous module loading
• __webpack_require__.f : Same as asynchronous module loading
• __webpack_require__.hmrF : Function for splicing hot update module url
• webpack/runtime/hot : This is not a single object or function, but contains a bunch of methods to implement module replacement

It can be seen that the HMR runtime is a superset of the above asynchronous module loading runtime, and the asynchronous module loading runtime is a superset of the first basic example runtime, superimposed on layers. Included in HMR:

• Modularity
• Asynchronous module loading capability-realize asynchronous loading of changed modules
• Hot replacement capability-replace the old module with the pulled new module, and trigger a hot update event

There is too much content, so we will open an article next time to talk about HMR.

Realization principle

Reading the above three examples carefully, I believe the reader should have vaguely captured some important rules:

• In addition to the business code, the bundle must also contain the runtime code to run normally
• runtime is determined by the business code, specifically the characteristics used by the business code . For example, when asynchronous loading is used, the __webpack_require__.e function needs to be packaged, so there must be a runtime dependent collection process.
• The business code written by the developer will be wrapped into the appropriate runtime function to achieve overall coordination

Falling to the implementation of the Webpack source code, the runtime generation logic can be divided into two steps:

1. dependency collection : Traverse the business code module to collect the characteristic dependencies of the module, so as to determine the dependency list of the entire project on the Webpack runtime
2. generates : merge the dependency list of runtime and pack it into the final output bundle

Both steps occur in the packaging stage, that is, in the compilation.seal function of the Webpack(v5) source code:

The above picture is a part of the Webpack knowledge graph I summarized. You can follow the public account [Tecvan] Reply [1] Get the online address

Pay attention to the figure above. When entering the runtime processing link, Webpack has already parsed the ModuleDependencyGraph and ChunkGraph , which means that the following can be calculated at this time:

• Need to output those chunk
• Each chunk contains those module and the content of module
• The father-son dependency between chunk and chunk

For students who are not clear about the relationship between bundle, module, and chunk, it is recommended to read more:

• [Summary of 4D characters] One article thoroughly understands the core principles of
• bit difficult webpack knowledge point: Dependency Graph in-depth analysis
• bit difficult webpack knowledge point: Chunk subcontracting rules in detail

Based on this information, the next step is to collect runtime dependencies.

Dependent collection

The dependency of Webpack runtime is conceptually similar to the dependency of Vue. They are used to express the dependency of modules on other modules, but the implementation method is based on dynamic and collected during operation, while Webpack collects dependencies based on static code analysis. . The implementation logic is roughly as follows:

The calculation logic that the runtime relies on is concentrated in the compilation.processRuntimeRequirements function, and the code contains three loops:

• Loop through all module first time, and collect all runtime dependencies of module
• The second loop traverses all chunk , and integrates all the module of chunk chunk
• Loop through all runtime chunks for the third time, collect chunk , then traverse all dependencies and publish the runtimeRequirementInTree hook, (mainly) the RuntimePlugin plugin subscribes to the hook and creates the corresponding RuntimeModule subclass instance according to the dependency type

Let's talk about the details below.

The first cycle: collecting module dependencies

In the packing (Seal) stage, complete ChunkGraph after construction, it will immediately call Webpack codeGeneration function iterates module array, call their module.codeGeneration function performs translation module, the translation module results:

Among them, the sources attribute is the translated result of the module; while runtimeRequirements is calculated based on AST, which is the runtime required to run the module. The calculation process has nothing to do with the subject of this article. We will dig a hole next time. Keep talking.

After completion of all translation modules, start calling compilation.processRuntimeRequirements enters the first heavy cycle, the result of the above-described translation runtimeRequirements recorded ChunkGraph object.

The second cycle: integrating chunk dependencies

The first loop module , and the second loop traverses the chunk array to collect all module , for example:

In the example figure, module a contains two runtime dependencies; module b contains one runtime dependency. After the second round of integration, the corresponding chunk will contain three runtime dependencies corresponding to two modules.

The third cycle: Dependent identification to RuntimeModule object

In the source code, the third loop has the least code but the most complicated logic, and roughly performs three operations:

• Traverse all runtime chunks and collect the runtime dependencies of chunk
• runtimeRequirementInTree hooks for all dependencies under this runtime chunk
• RuntimePlugin RuntimeModule subclass object according to the identification information that the runtime depends on, and adds the object to the ModuleDepedencyGraph and ChunkGraph systems for management

So far, the runtime dependency has completed the whole process from module content analysis, collection, to the creation of the dependency corresponding Module subclass, and then add Module to the ModuleDepedencyGraph / ChunkGraph / 060b6214b9d2a8 system. The module dependency diagram of the business code and runtime code is completely corresponding. , You can prepare to enter the next stage-to produce the final product.

But before continuing to explain the product logic, we need to solve two problems:

• What is a runtime chunk? What is the relationship with ordinary chunk
• What is RuntimeModule ? What is the difference with ordinary Module

Summary: Chunk and Runtime Chunk

In the previous article bit difficult webpack knowledge point: Chunk subcontracting rules in detail I tried to fully explain the default subcontracting rules of Webpack, review in three specific cases, Webpack will create a new chunk :

• Each entry item will generate a chunk object, called initial chunk
• Each asynchronous module will generate a corresponding chunk object, called async chunk
• After Webpack 5, if the entry configuration contains the runtime value, add a chunk object that specifically accommodates the runtime in addition to the entry, which can be called a runtime chunk at this time

By default, initial chunk usually contains all the runtime code needed to run the entry, but the third rule that appeared after webpack 5 breaks this limitation, allowing developers to separate the runtime from initial chunk and make it independent for multiple entries to share. The runtime chunk .

Similarly, most of the runtime code corresponding to the asynchronous module is included in the corresponding referrer, for example:

// a.js
export default 'a-module'

// index.js
// 异步引入 a 模块
import('./a').then(console.log)

In this example, the asynchronous index introducing a module, then, according to a default allocation rule produces two chunk : file entry corresponding to index initial chunk , a corresponding asynchronous module async chunk . From this point ChunkGraph perspective chunk[index] is chunk[a] parent, into runtime code is chunk[index] , standing angle browser running chunk[a] must run before chunk[index] , both form a clear parent-child relationship.

Summary: RuntimeModule system

When I first read the Webpack source code, I found it strange. Module is the basic unit of Webpack resource management, but Module 54 subclasses derived from Module => RuntimeModule => xxxRuntimeModule , most of which are inherited from 060b6214b9d5d9:

In bit difficult webpack knowledge point: Dependency Graph in-depth analysis article we talked about the generation process and function of the module dependency graph, but the content of this article is developed around the business code, mostly used NormalModule . When the seal function collects runtime, RuntimePlugin RuntimeModule subclasses for runtime dependencies one by one, for example:

• Rely on __webpack_require__.r in the modular implementation, then create the MakeNamespaceObjectRuntimeModule object correspondingly
• ESM relies on __webpack_require__.o , then the corresponding HasOwnPropertyRuntimeModule object is created
• Asynchronous module loading depends on __webpack_require__.e EnsureChunkRuntimeModule object is created correspondingly
• and many more

Therefore, it can be deduced that all RuntimeModule end of 060b6214b9d702 correspond one-to-one with specific runtime functions. The result of collecting dependencies is to create a bunch of supporting RuntimeModule outside the business code. These subclass objects are then added to ModuleDependencyGraph , and Into the entire module dependency system.

Resource merge generation

After the above runtime dependency collection process, all the content required by the bundle is ready, and then you can prepare to write it out to the file, that is, the emit phase in the core process in the following figure:

My other article, [Summary of 4D Characters] An article thoroughly understands the core principles of has a more detailed explanation of this one. Here, I will briefly talk about the code flow from the perspective of runtime:

• Call compilation.createChunkAssets , traverse chunks module corresponding to the chunk, including business modules and runtime modules, into one resource ( Source subclass) object
• Call compilation.emitAsset to mount the resource object to the compilation.assets attribute
• Call compiler.emitAssets to write all assets to FileSystem
• Post compiler.hooks.done hook
• End of run

Dig a hole

Webpack is really complicated. Every time I write a topic with full confidence, I will find more new pits, such as the concerns that can be derived from this article:

• In addition to the NormalModule and RuntimeModule systems, what are the roles of the other Module subclasses?
• What is the content translation process of a single Module? How is the runtime dependency calculated in this process?
• In addition to recording runtime requirements of modules and chunks, what role does ChunkGraph play?

Dig the pit slowly, fill the pit slowly. If you find the article useful, please like it, follow it and forward it.

Previous articles

• [Summary of 4D characters] One article thoroughly understands the core principles of
• Ten minutes to refine Webpack: detailed explanation of
• bit difficult webpack knowledge point: Dependency Graph in-depth analysis
• bit difficult knowledge point: Detailed explanation of
• [Recommended collection] Webpack 4+ collection of excellent learning materials