Front-end construction for the past decade

Write in front

The modular front-end / build tools from the very beginning of a browser-based running loaded RequireJs/Sea.js to assemble all the resources will depend on packaged webpack / rollup / parcel of bundle class modular building tools, to the current bundleless browser-based native ES Modular snowpack / vite , the front-end modularization/build tool has been developed for almost 10 years now.

This article mainly reviews the development history and implementation principles of front-end modularization/build tools in the past 10 years.

After reading this article, you can learn the following knowledge:

• Modular specification scheme
• The evolution of front-end construction tools, and a systematic understanding of front-end construction
• The birth process of each tool and the problems it solves
• Webpack/parcel/vite construction process and principle analysis

(Because it involves some history and trends, the views in this article only represent personal subjective views)

Browser-based modularity

CommonJS

Everything starts with the CommonJS specification.

CommonJS originally called ServerJs , the goal would have been to outside the browser of javascript normative codes, at that time NodeJs not yet born, there are some scattered applied to the service side of the JavaScript code, but did not complete ecosystem.

After that, NodeJs learned from CommonJS community to create its own module system.

RequireJs and AMD

CommonJs is a set of synchronous module import specifications, but it is not possible to achieve synchronous loading on the browser. This set of specifications obviously does not work on the browser, so the browser-based asynchronous module AMD ( Asynchronous Module Definition ) specification was born .

define(id?, dependencies?, factory);

define("alpha", ["require", "exports", "beta"], function (
  require,
  exports,
  beta
) {
  exports.verb = function () {
    return beta.verb();
    //Or:
    return require("beta").verb();
  };
});

AMD specification uses rely on the pre- , first write the dependencies that need to be used in the dependencies factory callback after all the dependencies are downloaded. The module is obtained by passing parameters, and the module is also supported by require("beta") . , But in fact, this require is just syntactic sugar. The module is not require , but factory callback as mentioned earlier. There was a lot of controversy about relying on pre-processing and execution timing. , Is not tolerated CommonJs

Browser application on the time CommonJs there is another genre module/2.0 , which BravoJS Modules / 2.0-Draft specification and FlyScript of Modules / Wrappings specification.

The code implementation is roughly as follows:

module.declare(function (require, exports, module) {
  var a = require("a");
  exports.foo = a.name;
});

However, RequireJs and cannot compete.

For the content of this see Yubo's 16114ee600f412 front-end modular development history .

Sea.js and CMD

Continues to RequireJs make suggestions, but do not continue to be adopted later, Yu Bo combined RequireJs and module/2.0 specification wrote based on CMD ( the Common Module Definition ) specification Sea.js .

define(factory);

define(function (require, exports, module) {
  var add = require("math").add;
  exports.increment = function (val) {
    return add(val, 1);
  };
});

In the CMD specification, a module is a file. The module will only be executed when it is require
Compared to AMD specifications, CMD more concise, but also easier to compatible CommonJS and Node.js of Modules specification.

Summarize

RequireJs and Sea.js use the dynamic creation script to load the js module asynchronously.

When the author was still a front-end Xiaobai using these two libraries, he was curious how it obtained the dependencies before the function call. Later, after reading the source code, he realized that it was a simple function toString method.

Get the function code string by factory back toString to 06114ee600f59f, and then get the string dependency in the require

This is why neither of the two allow require change the name or assign the value of the variable, nor to use the variable depending on the string, and only the string literal can be used.

Specifications of the dispute at the time still quite chaotic, first CommonJs community, then with AMD / CMD norms and NodeJs of module norms, but when those CommonJs implementation library gradually decline, and as NodeJs more and more fire, we lips CommonJs seems that 06114ee600f654 is only NodeJs represented by modules .

Build tool for bundle class

Grunt

With NodeJs increasingly popular, based NodeJs automated build tools Grunt birth

Grunt can help us automate tasks that need to be repeated, such as compression (minification), compilation, unit testing, linting, etc., as well as a powerful plugin ecology.

Grunt adopts the idea of configuration:

module.exports = function (grunt) {
  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON("package.json"),
    uglify: {
      options: {
        banner:
          '/*! <%= pkg.name %> <%= grunt.template.today("yyyy-mm-dd") %> */\n',
      },
      build: {
        src: "src/<%= pkg.name %>.js",
        dest: "build/<%= pkg.name %>.min.js",
      },
    },
  });

  // 加载包含 "uglify" 任务的插件。
  grunt.loadNpmTasks("grunt-contrib-uglify");

  // 默认被执行的任务列表。
  grunt.registerTask("default", ["uglify"]);
};

nodejs of a series of automation tools based on 06114ee600f7b8 also marks the front end has entered a new era.

browserify

browserify CommonJs on the browser side. It uses the NodeJs , and then compiles all dependent files into a bundle file, which is used in the browser through the <script> tag, and supports npm libraries.

var foo = require("./foo.js");
var gamma = require("gamma");

var elem = document.getElementById("result");
var x = foo(100);
elem.textContent = gamma(x);

$ browserify main.js > bundle.js

At that time, RequireJs(r.js) also had an api on the node side, it could compile the AMD output it to a single file, but the mainstream was still using RequireJs browser side.

AMD / RequireJS：

require(["./thing1", "./thing2", "./thing3"], function (
  thing1,
  thing2,
  thing3
) {
  // 告诉模块返回/导出什么
  return function () {
    console.log(thing1, thing2, thing3);
  };
});

CommonJS：

var thing1 = require("./thing1");
var thing2 = require("./thing2");
var thing3 = require("./thing3");

// 告诉模块返回/导出什么
module.exports = function () {
  console.log(thing1, thing2, thing3);
};

Compared with the compromise made by the AMD specification for the browser, CommonJs is more friendly in terms of server-side precompilation.

The commonly used collocation is browserify + Grunt , use Grunt plug-in of browserify to build modular code, and compress and transform the code.

UMD

Now, with RequireJs , it has also been browserify but both are using different modular specification, so with UMD - Universal module specifications, UMD specification is to be compatible AMD and CommonJS specification. It's the following stuff:

(function (global, factory) {
  typeof exports === "object" && typeof module !== "undefined"
    ? (module.exports = factory())
    : typeof define === "function" && define.amd
    ? define(factory)
    : (global.libName = factory());
})(this, function () {
  "use strict";
});

Gulp

As mentioned above, Grunt is based on configuration. The configuration is difficult to get started. You need to understand the parameters of each configuration. When the configuration complexity increases, the code looks more confusing.
gulp based on code configuration and Node.js stream makes the construction easier and more intuitive. More complex tasks can be configured.

var browserify = require("browserify");
var source = require("vinyl-source-stream");
var buffer = require("vinyl-buffer");
var uglify = require("gulp-uglify");
var size = require("gulp-size");
var gulp = require("gulp");

gulp.task("build", function () {
  var bundler = browserify("./index.js");

  return bundler
    .bundle()
    .pipe(source("index.js"))
    .pipe(buffer())
    .pipe(uglify())
    .pipe(size())
    .pipe(gulp.dest("dist/"));
});

The above is an browserify , which can be seen very concise and intuitive.

webpack

Before talking about webpack , let’s talk about teacher Ruan Yifeng’s complaints.

webpack1 supports CommonJs and AMD modular systems, optimizes dependencies, supports subcontracting, supports multiple types of script, image, file, css/less/sass/stylus, mocha/eslint/jshint packaging, and a rich plug-in system.

The above three libraries Grunt/Gulp/browserify are biased towards tools, and webpack integrates all the above functions, compared to tools, it has a larger and complete function.

webpack is more engineering-oriented, but it did not immediately become popular at the time, because the front-end development at the time was not too complicated. There were some mvc frameworks but they were short-lived. The front-end technology stack was in requireJs/sea.js, grunt/ Choose between gulp, browserify, and webpack.

webpack real fire up in 2015/2016 , along with ES2015 ( ES6 ) release, not only brought new syntax, it also brings front-end module belonging to the norms ES module , vue/react/Angular three frame house on fire, webpack2 release: Support ES module , babel , typescript , jsx, Angular 2 components and vue components, webpack with react/vue/Angular the best choice, so far front-end development is inseparable from webpack , webpack truly becomes the core of front-end engineering.

webpack will not be repeated here.

principle

webpack main three modules of 06114ee600fbda are, the latter two are also often configured by us:

• Core process
• loader
• plugins

webpack relies on Tapable for event distribution. There are a large number of hooks hooks inside. In the Compiler and compilation , events are distributed through hooks. The hooks plugins . The actual code is all plugins , while loader is responsible for conversion. After the code receives a source code processing, it returns the processing result content string -> result string .

Because there are too many hooks, the webpack looks very convoluted. Let's briefly talk about the general process:

1. Get parameters through the command line and webpack.config.js
2. Create compiler object, initialize plugins
3. Start the compilation phase, add entry resources for addEntry
4. addModule creation module
5. runLoaders execute loader
6. Dependency collection, js resolves to AST acorn , then finds dependencies, and repeats 4 steps
7. After constructing the dependency tree , enter the generation phase and call compilation.seal
8. After a series of optimize optimization dependencies, chunks generated and written to the file

webpack , now talk about 2 disadvantages:

• Complex configuration
• Slow construction of large projects

The configuration complexity has always been webpack has been complained about. It is mainly due to the overweight plug-in system, complex plug-in configuration, the plug-in documentation is not clear, the plug-in update too fast, or the documentation fails to keep up.

For example, now webpack has reached 5. The online search is all webpack3 . It is often a new feature. After configuring according to the document, if there is an error, check it online, copy a section here, copy a section there, and come a few more An error was reported, and it was finally able to run after another meal.

Later, in response to this problem, the front-end scaffolding was derived, react out of create-react-app , vue out of vue-cli , the scaffolding built- webpack development, reached 0 configuration, developers do not need to care about the complex configuration of webpack

rollup

In 2015, after the front-end ES module released, rollup out.

rollup compiles the ES6 module, and proposes Tree-shaking , according to the ES module , deletes the code that is not actually used, supports exporting a variety of standardized syntax, and the exported code is very concise. If you read the vue dist , you will know the exported vue The code does not affect reading at all.

rollup plug-in system support: babel , CommonJs , terser , typescript other functions.

Compared with browserify of CommonJs , rollup focuses on ES module .
Compared to webpack front-end engineering of large and, rollup focus on pure javascript , mostly used as the packed tool tool or library library.

Libraries such as react and vue all use rollup package the project, and the vite mentioned below also relies on rollup as the production environment to package js.

Tree-shaking

export const a = 1;
export const b = 2;

import { a } from "./num";

console.log(a);

The declaration of b will be deleted after the above code is finally packaged.

This relies on ES module , which can be determined in the compilation stage which variables are imported and exported by the module.

CommonJs is based on runtime module import, what it exports is a whole, and the require(variable) can be a variable, so it cannot be recognized as a dependency ast

webpack4 also began to support tree-shaking , but due to historical reasons, there are too many CommonJS , requiring additional configuration.

parcel

The two shortcomings of webpack mentioned above parcel is to solve these two shortcomings. speed zero configuration .

Packaging tool	time
browserify	22.98s
webpack	20.71s
parcel	9.98s
parcel - with cache	2.64s

The above is parcel official data of 06114ee601036c, based on a reasonable size application, including 1726 modules, 6.5M uncompressed size. Built on a 2016 MacBook Pro with 4 physical core CPUs.

parcel uses the worker process to enable multi-core compilation and uses file caching.

parcel supports 0 configuration, built-in html、babel、typescript、less、sass、vue such as 06114ee60103c2, no need to configure, and different from webpack only js files can be used as the entrance, in parcel everything is a resource, so html files and css files can be packaged as entrances.

So there is no need for the complicated configuration of webpack parcel index.html command can directly start a server hot update to develop the vue/react project.

Parcel also has its disadvantages:

• The cost of 0 configuration, 0 configuration is good, but if you want to configure some complex configuration, it is very troublesome.
• Ecological, compared to webpack relatively small, it is more troublesome to find solutions if you encounter errors.

principle

1. commander Get command
2. Start the server service, start the watch monitoring file, start the WebSocket service for hmr, start multithreading
3. If it is the first time to start, start compiling for the entry file
4. asset based on the extension, such as jsAsset , cssAsset , vueAsset , if parcel recognizes the less file, if there is no less library in the project, it will be installed automatically
5. Read the cache, if there is a cache, skip to step 7
6. Multi-threaded compile the file, call the method parse -> ast -> collect dependencies -> transform (transform code) -> generate (generate code) asset , the dependencies are collected in this process, and the compiled results are written into the cache
7. Compile dependent files, repeat step 4 to start
8. createBundleTree Create dependency tree, replace hash etc., package package to generate final code
9. When the watch changes, repeat step 4, and send the result 7 WebSocket for hot update.

A complete modular packaging tool has the above functions and processes.

Build tool based on browser ES module

browserify , webpack , rollup , parcel are all recursive circular dependencies, and then assembled into a dependency tree, and then generate code after optimizing the dependency tree.
But the disadvantage of this is that it is slow and needs to traverse all dependencies. Even if parcel utilizes multiple cores, webpack also supports multithreading. When packaging large projects, it is still slow and may take a few minutes, and there is a performance bottleneck.

So the runtime packaging tool based on the browser's native ESM




Only the resources used in the screen are packaged instead of the entire project. Compared with bundle such as 06114ee60108b4, the development experience can only be described as speed.
(The actual production environment packaging will still build dependent packaging)

snowpack and vite

Because snowpack and vite are similar, they are both bundleless so let's take it together. The difference can be seen in the difference between vite and snowpack , so I won't go into details here.

bundleless runtime packaging tool is milliseconds, because there is no need to package any content, only two server are needed, one for page loading and the other for HMR of WebSocket . When the browser sends a native ES module request , server only needs to compile the current file after receiving the request and return it to the browser without any dependency.

bundleless tools production environment packed when still bundle constructed so dependent view of the way, vite using rollup packed js the production environment.

Take the principle of vite an example:

vite After starting the server, all html inlet preliminarily using esbuild recompiled, all the node_modules dependent compiled and cached in, e.g. vue cache as a single file.

When opening and entering the link in the browser to render the index.html file, use the ES module that comes with the browser to request the file.

<script type="module" src="/src/main.js"></script>

vite receive a src/main.js of http file requests, using esbuild start compiling main.js , here not main.js compile dependencies inside.

import { createApp } from "vue";
import App from "./App.vue";

createApp(App).mount("#app");

After the browser obtains and compiles main.js , it sends two requests again, one is for vue , because vue is pre-cached as mentioned earlier, and the cache is returned directly to the browser, and the other is the App.vue file, which requires @vitejs/plugin-vue to compile After the compilation is complete, the result is returned to the browser ( @vitejs/plugin-vue will be automatically configured when the scaffolding template is created).

Because it is based on the browser-based ES module CommonJs and UMD need to be converted to ESM during the compilation process.

Vite also uses the HTTP header to speed up the reloading of the entire page (again, let the browser do more for us): requests for source code modules will 304 Not Modified , and requests for dependent modules will be strongly cached Cache-Control: max-age=31536000,immutable If they are cached, they will not need to be requested again. Even if the cache fails, as long as the service is not killed, the compilation result is still stored in the program memory and will be returned quickly.

The above mentioned several times esbuild , esbuild use go language, so i/o the operation and running faster than an interpreted language NodeJs faster, esbuild known as speed is node 10 ~ 100 times that of other tools to write.

ES module relies on the concept of runtime compilation + esbuild + cache so that vite is far behind other build tools.

Summarize

Simple summary:

• Front-end runtime modularity

  • RequireJs AMD specification
  • sea.js CMD specification

• Automation tool

  • Grunt based on configuration
  • Gulp based on code and file flow

• Modular

  • browserify only responsible for modularization based on the CommonJs
  • rollup based on ES module , tree shaking optimized code, supports a variety of standard export, can integrate compression, compilation, commonjs syntax and other functions through plug-ins

• Engineering

  • webpack and complete modular construction tool
  • parcel Extreme speed 0 configuration modular construction tool
  • snowpack/vite ESM runtime modular build tool

nodejs , the front-end construction tools have derived a series of tools. In addition to some other tools listed in the article, or based on the secondary packaging of these tools, the front-end was not without construction tools before nodejs Although it is rare, it can only be said that nodejs allows more people to participate, especially the front end can use familiar languages to participate in the development tools. There are 170,000 packages on npm and 30 billion downloads per week. .

In this process, there are also some problems left over from the modularization history. We are still using the UMD specification library to be compatible with this AMD specification. Most of the npm packages are based on CommonJs and have to be compatible with ESM and CommonJs .

webpack has been dominating the front-end for 5 years. When people mention development projects, they only think of webpack . Who will replace it in the next 5 years? snowpack/vite , when the packaging speed reaches 0 seconds, is it possible that a new generation of build tools will appear in the future? What will happen to the front end in the next 10 years?