The third part of the Webpack series: Dependency Graph in-depth analysis

The full text is 2500 words, and the reading time is about 30 minutes. If you find the article useful, you are welcome to like it, but it is not easy to write, and it is forbidden to reprint it in any form without the author's consent! ! !

background

The concept of Dependency Graph comes from the official website Dependency Graph | webpack . The original explanation is as follows:

Any time one file depends on another, webpack treats this as a dependency_. This allows webpack to take non-code assets, such as images or web fonts, and also provide them as _dependencies for your application.

When webpack processes your application, it starts from a list of modules defined on the command line or in its configuration file. Starting from these entry points_, webpack recursively builds a _dependency graph that includes every module your application needs, then bundles all of those modules into a small number of bundles - often, just one - to be loaded by the browser.

The core meaning of the translation is: when webpack processes the application code, it will recursively build dependency graph _ from the entry provided by the developer, and then package these modules into bundles.

However, the facts are far more than simple as described on the official website. Dependency Graph runs through the entire running cycle of webpack, from module analysis in the make phase, to chunk generation in the seal phase, and tree-shaking functions, which are highly dependent on the Dependency Graph. It is a construction of webpack resources. Very core data structure.

This article will focus on the implementation of the Dependency Graph of webpack\@v5.x, and discuss three aspects:

• What data structure is presented in the webpack implementation of Dependency Graph
• How to collect dependencies between modules during Webpack running, and then build a Dependency Graph
• After the Dependency Graph is built, how is it consumed?

After studying this article, you will further understand the processing details of webpack module parsing. Combined with the previous article [Summary], you can thoroughly understand the core principle of , and you can have a more thorough understanding of the core mechanism of webpack.

Follow the public account [Tecvan], reply [1], and get the mind map of the Webpack knowledge system

Dependency Graph

This section will go into the webpack source code and interpret the internal data structure of Dependency Graph and the process of dependency collection. Before the official launch, it is necessary to review several important concepts of webpack:

• Module : The mapping object of the resource inside webpack, including the path, context, dependency, content and other information of the resource
• Dependency : Refer to other modules in the module, such as the import "a.js" statement, webpack will first express the reference relationship as a Dependency subclass and associate the module object. After the current module content is parsed, start the next cycle and start converting the Dependency object into an appropriate Module subclass.
• Chunk : An object used to organize the output structure. After webpack analyzes the content of all module resources and constructs a complete Dependency Graph, it will construct one or more chunk instances according to the user configuration and the content of the Dependency Graph, each chunk and the final output The files of are roughly one-to-one correspondence.

data structure

The implementation of Dependency Graph of Webpack 4.x is relatively simple, and it is mainly referenced and referenced by the series of built-in attribute records of Dependence/Module.

After Webpack 5.0, a relatively complex class structure was implemented to record the dependencies between modules, and the logic related to module dependencies was decoupled from Dependence/Module into a set of independent type structures. The main types are:

• ModuleGraph : A container for recording Dependency Graph information. On the one hand, it saves all module and dependency objects involved in the construction process, as well as the references between these objects; on the other hand, it provides various tools and methods for users to quickly read module or dependency additional information
• ModuleGraphConnection : A data structure that records the reference relationship between modules. The parent module in the reference relationship is recorded originModule module is recorded through the 060aa43cd5570f attribute. In addition, a series of function tools are provided for judging the validity of the corresponding reference relationship
• ModuleGraphModule : Module object under the Dependency Graph system, including the incomingConnections module object-the ModuleGraphConnection collection pointing to the module itself, that is, who refers to the module itself; outgoingConnections -the external dependency of the module, that is, the module refers to others Module.

The relationship between classes is roughly as follows:

The above class diagram requires additional attention:

• ModuleGraph object records Dependency object and the ModuleGraphConnection connection object through the _dependencyMap attribute. In the subsequent processing, the reference and Dependency corresponding to the 060aa43cd557b1 instance can be quickly found based on this layer of mapping.
• ModuleGraph objects through _moduleMap the module additional basis ModuleGraphModule information, ModuleGraphModule greatest effect is to record the reference relationship with the reference module, the subsequent processing may be found based on the attribute module instance with all the dependencies are dependencies

Dependent collection process

ModuleGraph , ModuleGraphConnection , and ModuleGraphModule cooperate to gradually collect the dependencies between modules during the webpack construction process (make phase). Review the previous article [Summary] This article has a thorough understanding of the construction flow chart mentioned in the

The construction process itself is very complicated. It is recommended that readers compare and read [Summary of Ten Thousand Words] to understand the core principles of and deepen their understanding. The dependency collection process mainly occurs in two nodes:

• addDependency : After webpack resolves the reference relationship from the module content, it creates an appropriate Dependency subclass and calls this method to record to the module instance
• handleModuleCreation : After the module is parsed, webpack traverses the dependency collection of the parent module, calls this method to create the submodule object corresponding to Dependency compilation.moduleGraph.setResolvedModule method to record the parent-child reference information on the moduleGraph object

setResolvedModule method is roughly as follows:

class ModuleGraph {
    constructor() {
        /** @type {Map<Dependency, ModuleGraphConnection>} */
        this._dependencyMap = new Map();
        /** @type {Map<Module, ModuleGraphModule>} */
        this._moduleMap = new Map();
    }

    /**
     * @param {Module} originModule the referencing module
     * @param {Dependency} dependency the referencing dependency
     * @param {Module} module the referenced module
     * @returns {void}
     */
    setResolvedModule(originModule, dependency, module) {
        const connection = new ModuleGraphConnection(
            originModule,
            dependency,
            module,
            undefined,
            dependency.weak,
            dependency.getCondition(this)
        );
        this._dependencyMap.set(dependency, connection);
        const connections = this._getModuleGraphModule(module).incomingConnections;
        connections.add(connection);
        const mgm = this._getModuleGraphModule(originModule);
        if (mgm.outgoingConnections === undefined) {
            mgm.outgoingConnections = new Set();
        }
        mgm.outgoingConnections.add(connection);
    }
}

The above code mainly changes the in and out connections _dependencyMap moduleGraphModule to collect the upstream and downstream dependencies of the current module.

Example analysis

Look at a simple example, for the following dependencies:

compilation.handleModuleCreation function is called recursively during the construction phase to gradually complete the Dependency Graph structure, and the following data results may eventually be generated:

ModuleGraph: {
    _dependencyMap: Map(3){
        { 
            EntryDependency{request: "./src/index.js"} => ModuleGraphConnection{
                module: NormalModule{request: "./src/index.js"}, 
                // 入口模块没有引用者，故设置为 null
                originModule: null
            } 
        },
        { 
            HarmonyImportSideEffectDependency{request: "./src/a.js"} => ModuleGraphConnection{
                module: NormalModule{request: "./src/a.js"}, 
                originModule: NormalModule{request: "./src/index.js"}
            } 
        },
        { 
            HarmonyImportSideEffectDependency{request: "./src/a.js"} => ModuleGraphConnection{
                module: NormalModule{request: "./src/b.js"}, 
                originModule: NormalModule{request: "./src/index.js"}
            } 
        }
    },

    _moduleMap: Map(3){
        NormalModule{request: "./src/index.js"} => ModuleGraphModule{
            incomingConnections: Set(1) [
                // entry 模块，对应 originModule 为null
                ModuleGraphConnection{ module: NormalModule{request: "./src/index.js"}, originModule:null }
            ],
            outgoingConnections: Set(2) [
                // 从 index 指向 a 模块
                ModuleGraphConnection{ module: NormalModule{request: "./src/a.js"}, originModule: NormalModule{request: "./src/index.js"} },
                // 从 index 指向 b 模块
                ModuleGraphConnection{ module: NormalModule{request: "./src/b.js"}, originModule: NormalModule{request: "./src/index.js"} }
            ]
        },
        NormalModule{request: "./src/a.js"} => ModuleGraphModule{
            incomingConnections: Set(1) [
                ModuleGraphConnection{ module: NormalModule{request: "./src/a.js"}, originModule: NormalModule{request: "./src/index.js"} }
            ],
            // a 模块没有其他依赖，故 outgoingConnections 属性值为 undefined
            outgoingConnections: undefined
        },
        NormalModule{request: "./src/b.js"} => ModuleGraphModule{
            incomingConnections: Set(1) [
                ModuleGraphConnection{ module: NormalModule{request: "./src/b.js"}, originModule: NormalModule{request: "./src/index.js"} }
            ],
            // b 模块没有其他依赖，故 outgoingConnections 属性值为 undefined
            outgoingConnections: undefined
        }
    }
}

As can be seen from the above Dependency Graph, essentially ModuleGraph._moduleMap has formed a directed acyclic graph structure, wherein the dictionary _moduleMap the key for the nodes of the graph, the corresponding value ModuleGraphModule structure outgoingConnections embodiment of FIG edge attributes, the upper Starting from the starting point index.js and outgoingConnections , all the vertices of the graph can be traversed.

effect

Taking webpack\@v5.16.0 as an example, the keyword moduleGraph appeared 1277 times, almost covering webpack/lib folder, and its effect can be seen. Although the frequency of occurrence is very high, in general, it can be seen that there are two main functions: information index, conversion to ChunkGraph to determine the output structure.

Information Index

ModuleGraph type provides many tool functions for querying module / dependency information, for example:

• getModule(dep: Dependency) : Find the corresponding module instance according to dep
• getOutgoingConnections(module: Module) : Find all dependencies of the module
• getIssuer(module: Module) : Find module where 060aa43cd55ae3 is referenced (for more information about the issuer mechanism, please refer to another article of : 160aa43cd55ae6 Ten minutes to refine Webpack: module.issuer attribute detailed explanation )

and many more.

Many plug-ins, Dependency subclasses, and Module subclass implementations within Webpack\@v5.x need to use these tool functions to find specific modules and dependent information, such as:

• SplitChunksPlugin in chunks optimization process, it is necessary to use moduleGraph.getExportsInfo query each module exportsInfo (information collection module exports, strongly associated with tree-shaking, will single out the follow-up article to explain) information to determine how to separate chunk .
• In the compilation.seal function, you need to traverse the dep corresponding to the entry and call moduleGraph.getModule get the complete module definition
• ...

Then, when you write a plug-in, you can consider appropriately referring webpack/lib/ModuleGraph.js to confirm that you can obtain the information you need using those functions.

Build ChunkGraph

In the main process of Webpack, after the make construction phase is over, it will enter the seal phase, and begin to sort out how to organize the output content. In webpack\@v4.x, the seal stage mainly revolves around Chunk and ChunkGroup . After 5.0, a new set ChunkGraph is also introduced to implement resource generation algorithm similar to Dependency Graph.

In compilation.seal function, according to the first default rule - Each entry corresponds organized as a chunk, after the call webpack/lib/buildChunkGraph.js document definition buildChunkGraph method, traversing make phase generates moduleGraph the object module so that the dependencies into chunkGraph object.

The logic of this piece is also very complicated, so I won't expand it here. Next time, I will chunk/chunkGroup/chunkGraph separate article explaining the output rules of modules constructed by objects such as 060aa43cd55c96.

to sum up

The Dependency Graph concept discussed in this article is widely used inside webpack, so understanding this concept will help us understand the webpack source code, or learn how to write plug-ins and loaders. In the process of analysis, many new blind spots of knowledge have actually been excavated:

• What is the complete mechanism of Chunk?
• How is the complete system of Dependency realized and what is the role?
• How to collect the exportsInfo of Module? How is it used in the tree-shaking process?

If you are also interested in the above questions, you are welcome to like and pay attention. We will output more useful articles around webpack in the follow-up.

Past articles:

• [Summary of Ten Thousand Words] One article thoroughly understands the core principles of
• Ten minutes to refine Webpack: detailed explanation of
• [Source code interpretation] In-depth explanation of