Handwritten simple packaging tool webpack-demo

webpack as a packaging tool, before learning it, I felt very unfamiliar with it. Recently, I spent some time learning it.

The biggest gain of learning is to write a simple packing tool webpack-demo .

webpack-demo divided into three main parts:

• Generate abstract syntax tree
• Obtain the dependencies of each module
• Generate code that the browser can execute

Dependence preparation

There are three files in the src index.js , message.js , word.js . Their dependencies are: index.js is the entry file, of which index.js depends on message.js , and message.js depends on word.js .

index.js：

import message from "./message.js";

console.log(message);

message.js：

import { word } from "./word.js";

const message = `say ${word}`;

export default message;

word.js：

var word = "uccs";

export { word };

Now we need to write a bundle.js to package these three files into a file that the browser can run.

The packaged configuration items are written in webpack.config.js . The configuration is relatively simple, only entry and output .

const path = require("path");
module.exports = {
  entry: path.join(__dirname, "./src/index.js"),
  output: {
    path: path.join(__dirname, "dist"),
    filename: "main.js",
  },
};

Code analysis

Get the code of the entry file

Get the content of the entry file through node provided by fs.readFileSync

const fs = require("fs");
const content = fs.readFileSync("./src/index.html", "utf-8");

After getting the content of the entry file, you need to get its dependency ./message . Because it is type string Naturally, I think of using string interception to obtain, but this method is too troublesome. If there are many dependencies, this expression will be particularly complicated.

What better way to get its dependencies?

Generate abstract syntax tree

babel provides a code analysis tool @babel/parser , this tool has a method parse , receiving two parameters:

• code : source code
• options : The source code uses ESModule , you need to pass in sourceType: module

function getAST(entry) {
  const source = fs.readFileSync(entry, "utf-8");
  return parser.parse(source, {
    sourceType: "module",
  });
}

This ast is an object called an abstract syntax tree, which can represent the current piece of code.

ast.program.body stores our program. The declared statement can be found through the abstract syntax tree, and the placement of the statement statement is the related dependency.

From the figure below, you can see that the first import , and the second statement is an expression statement.

The next step is to get all the dependencies in this code.

One way is to write your own traversal, to traverse body in type: ImportDeclaration , this method is a bit troublesome.

Is there a better way to get it?

Get related dependencies

babel provides a tool @babel/traverse , you can quickly find ImportDeclaration .

traverse receives two parameters:

• ast : Abstract Syntax Tree
• options : Traverse, you need to find out what kind of elements, such as ImportDeclaration , as long as there is ImportDeclaration abstract syntax tree, it will enter this function.

function getDependencies(ast, filename) {
  const dependencies = {};
  traverse(ast, {
    ImportDeclaration({ node }) {
      const dirname = path.dirname(filename);
      const newFile = path.join(dirname, node.source.value);
      dependencies[node.source.value] = newFile;
    },
  });
  return dependencies;
}

ImportDeclaration : A node node will be received, and all ImportDeclaration will be analyzed.

From the figure above, we can see that node.source.value is dependent. dependencies save the dependency in the 06159a96108d85 object. The dependency path here is relative to bundle.js or an absolute path, otherwise the packaging will go wrong.

Code conversion

After the dependency analysis is complete, the source code needs to be converted, because the import syntax cannot be run directly in the browser.

babel provides a tool @babel/core , which is babel , and provides a transformFromAst method that can convert ast into code that can be run by the browser.

It receives three parameters:

• ast : Abstract Syntax Tree
• code : No need, you can pass in null
• options : during the transition needed in the presents: ["@babel/preset-env"]

function transform(ast) {
  const { code } = babel.transformFromAst(ast, null, {
    presets: ["@babel/preset-env"],
  });
  return code;
}

Get all dependencies

After the entry file is analyzed, its related dependencies are placed in dependencies . The next step will be to rely on the modules, layer by layer analysis and finally analyze the information of all modules, how to achieve this function?

First define a buildModule to obtain entryModule . entryModule includes filename , code , dependencies

function buildModule(filename) {
  let ast = getAST(filename);
  return {
    filename,
    code: transform(ast),
    dependencies: getDependencies(ast, filename),
  };
}

By traversing modules get all the information module, when first completed for after cycle, message.js module analysis pushed to modules , this time modules length became 2 , so it will continue for cycle to analyze message.js , found message.js dependent have word.js , will call buildModule analysis relies, and pushed to modules in. modules length becomes 3 , to analyze the word.js dependency, the dependency is not found, the end of the cycle.

Through continuous circulation, the entry file, its dependencies, and the dependencies it depends on can modules be pushed to 06159a961090f3.

const entryModule = this.buildModule(this.entry);
this.modules.push(entryModule);
for (let i = 0; i < this.modules.length; i++) {
  const { dependencies } = this.modules[i];
  if (dependencies) {
    for (let j in dependencies) {
      // 有依赖调用 buildmodule 再次分析，保存到 modules
      this.modules.push(this.buildModule(dependencies[j]));
    }
  }
}

modules is an array, it is a bit difficult to finally generate browser executable code, so here is a conversion

const graphArray = {};
this.modules.forEach((module) => {
  graphArray[module.filename] = {
    code: module.code,
    dependencies: module.dependencies,
  };
});

Generate browser executable code

After all the dependencies are calculated, you need to generate code that the browser can execute.

This code is a self-executing function, passing in graph .

graph passed in, it needs to be JSON.stringify , because directly passing in the object in the string will become [object Object] .

In the packaged code, there is a require , which is not supported by the browser, so we need to define this method.

require needs to do a path conversion when importing the path, otherwise, no dependencies will be found, so localRequire is defined.

require is also a self-executing function inside, receiving three parameters: localRequire , exports , code .

const graph = JSON.stringify(graphArray);
const outputPath = path.join(this.output.path, this.output.filename);
const bundle = `
  (function(graph){
    function require(module){
      function localRequire(relativePath){
        return require(graph[module].dependencies[relativePath])
      }
      var exports = {};
      (function(require, exports, code){
        eval(code)
      })(localRequire, exports, graph[module].code)
      return exports;
    }
    require("${this.entry}")
  })(${graph})
`;
fs.writeFileSync(outputPath, bundle, "utf-8");

Summarize

After writing a simple packaging tool by hand, I have a deeper understanding of the internal dependency analysis and code conversion webpack

Reference materials: From basic to actual combat, take you to master the new version of Webpack4.0