The inspiration from Vue Conf's sharing of Vite

Vite last year, it has come out, but I really get to know it in recent Vue Conf Li Kui on the Vite: Share next generation web tools. Some of the points he mentioned attracted me. At the beginning of sharing, he briefly explained the key points of this sharing:


Among them, ESM and esbuild will be described in detail below

Next he mentioned Bundle-Based Dev Server . That is, the webpack processing method we have been using:


Here is a quote from the official website:

As we began to build larger and larger applications, the JavaScript code that needed to be processed also increased exponentially. It is not uncommon for large projects to contain thousands of modules. We are beginning to encounter performance bottlenecks- JavaScript usually take a long time (or even a few minutes!) to start the development server. Even with HMR , the effect of file modification takes a few seconds to reflect in the browser. come out. In this cycle, slow feedback will greatly affect the developer's development efficiency and happiness.

A brief summary is that if the application is more complex, Webpack is relatively not so silky:

• Webpack Dev Server cold start time will be longer
• Webpack HMR The response speed of hot update is slow

This is Vite appeared, you can simply understand it as: No-Bundler construction scheme. In fact, it is the ability to ESM

But the first tool to use the browser's native ESM capabilities is not Vite , but a tool Snowpack Of course, this article will not expand to compare Vite and it. What do you want to know about the difference between Vite and X?

At this point, I can't help but start thinking about a question: Why Vite the tool 060cb395281cf0 appear, and what prerequisites is it based on?

With this question, combined with sharing and Vite and some articles in the community, I found the following modules that can be inseparable Vite

• ES Modules
• HTTP2
• ESBuild

I have actually heard of these, but the specific details have not been thoroughly understood. Today, it's time to analyze it in depth.

ES Modules

In modern front-end engineering systems, we have actually been using ES Modules :

import a from 'xxx'
import b from 'xxx'
import c from 'xxx'

It may be too ordinary, everyone has long been used to it. But if you don't have a deep understanding of ES Modules , there may be some obstacles for us to understand some of the existing wheels (such as Vite

ES Modules is a module system natively supported by the browser. Previously, CommonJS and other module systems based on AMD RequireJS commonly used.

Take a look at the current browser support for it:


Mainstream browsers (except IE11) have been supported, and its biggest feature is to use export , import on the browser side to import and export modules, set type="module" script tag, and then use the module content.

It says so much, after all, only ES Modules of introduce myself. For a long time, he is like the black box, we do not know the internal execution mechanism. Let us take a closer look.

Let's take a look at the role of the module system: script tag is likely to cause global scope pollution, and it is necessary to maintain a series script . The project is large and it is becoming more and more difficult to maintain. The module system makes the dependencies between various modules obvious by declaratively exposing and referencing modules.

When you are developing with modules, you are actually building a dependency graph. The connections between different modules represent the import statements in the code.

It is these import statements that tell the browser or Node which code to load.

What we need to do is to specify an entry file for the dependency graph. Starting from this entry file, the browser or Node will follow the import statement to find other dependent code files.

For the ES module, there are three main steps:

• structure. Find, download and parse all files into the module record.
• instantiation. Look for an area in memory to store all the exported variables (but no value has been filled yet). Then let export and import point to these memory blocks. This process is called linking.
• evaluated. Run the code and fill in the actual value of the variable in the memory block.

Construction

In the construction phase, each module will go through three things:

• Find : Find out where to download the file containing the module (also called module analysis)
• Download : Get files (download from URL or load from file system)
• Parse : Parse the file into module records

Find

Usually we will have a main file main.js as the beginning of everything:

<script src="main.js" type="module"></script>

Then use the import statement to introduce the content exported by other modules:


import sentence is called Module Specifier . It tells Loader where to find the imported modules.

One thing to note about module identifiers: they sometimes need to be processed differently between the Node Each host has its own way of interpreting the module identifier string.

At present, you can only use URL as Module Specifier in the browser, that is, use URL to load the module.

Download

And there is a problem that follows. The browser does not know which modules the file depends on before parsing the file. Of course, it can't parse the file before getting the file.


This will cause the entire process of resolving dependencies to be blocked.

Blocking the main thread like this will make applications that use modules too slow to be usable. This is ES module specification divides the algorithm into multiple stages. Separate the construction process, so that the browser can download files by itself and establish its own understanding of the module diagram before performing the synchronization initialization process.

For the ES module, before any evaluation, you need to construct the entire module diagram in advance. This means that you cannot have variables in your module identifier, because these variables do not yet have values.


But sometimes it is really useful to use variables in the module path. For example, you may need to switch to load a certain module according to the code's running situation or operating environment.

In order for the ES module to support this, there is a proposal dynamic import. With it, you can use import statements import(${path} /foo.js

Its principle is that any import() will be used as an entry to an independent dependency graph. The dynamically imported module opens a new dependency graph and handles it separately.


Parse analysis

In fact, parsing the file is to help the browser understand the composition of the module, and we call the module composition table that it parses as Module Record module record.

The module record contains the AST the current module, the variables of which modules are referenced, and some specific properties and methods in the past.

Once the module record is created, it will be recorded in the module map Module Ma . After they are recorded, if we have the same URL request, Loader directly employed Module Map in URL corresponding Module Record .

There is a detail in the analysis that may seem trivial, but it actually has a big impact. All modules are interpreted as using "use strict" at the top. There are some other nuances. For example, the keyword await retained at the top of the code module, this values is undefined .

This different parsing method is called parsing target. If you use different targets to parse the same file, you will get different results. So before you start parsing, you need to know the type of file being parsed: is it a module?

This is easy in the browser. You only need to set type="module" script tag. This tells the browser that this file should be parsed as a module.

But in Node , there is no HTML tag, so other ways are needed to identify it. The current mainstream solution in the community is to modify the file suffix to .mjs to tell Node that this will be a module. However, it has not yet been standardized, and there are still many compatibility issues.

At this point, at the end of the loading process, the normal main entry file has become a pile of module records Module Record .

The next step is to instantiate this module and link all the instances together.

Instantiation phase

In order to instantiate Module Record , the engine will be used Depth First Post-order Traversal (after a depth priority order) traversal, JS engine will for every kind Module Record create a Module Environment Record module environment record, it will manage Module Record corresponding to the variable, and all export allocate memory space.


ES Modules This type of connection is called Live Bindings (dynamic binding).

The reason why ES Modules uses Live Bindings is because it will help to do static analysis and avoid some problems, such as circular dependencies.

The CommonJS export is copy after export objects, which means that if the export module to change the value later, you import the module does not see the change.

This is the usual conclusion: CommonJS module exports are copies of values, and ES Modules are references to values.

Evaluation phase (evaluate)

The last step is to fill in the memory. Remember that we have connected all export and import through memory, but the memory has not yet had a value.

JS engine adds value to these memory areas by executing top-level code (code outside of functions).

So far, I have roughly analyzed the black box of ES Modules

Of course, I refer to es-modules-a-cartoon-deep-dive , and then combine the analysis from my own understanding. If you want to know more about the implementation behind it, you can poke the link above.

ES Modules in Vite We can open a running Vite project:


As you can see from the picture above:

import { createApp } from "/node_modules/.vite/vue.js?v=2122042e";

Unlike the previous import { createApp } from "vue" , the module path introduced is rewritten here:


Vite uses modern browsers to natively support the ESM feature, omitting the packaging of modules. (This is also a very important reason for the faster project startup and hot update in the development environment)

HTTP2

Before looking at HTTP2 , let's first understand the development history of HTTP

We know that HTTP is the most important and most used protocol in the browser, and it is the communication language between the browser and the server. With the development of browsers, HTTP continues to evolve in order to adapt to new forms.

HTTP/0.9 that first appeared is relatively simple: a request-response-based model is adopted, a request is sent from the client, and the server returns data.

It can be seen from the figure that there is only one request line and the server does not return header information.

The rapid development of the World Wide Web has brought many new demands, and HTTP/0.9 is no longer suitable for the development of emerging networks, so a new protocol is needed to support the emerging networks. This is the reason for the birth of HTTP/1.0

And what is displayed in the browser is not only the HTML file, but also JavaScript , CSS , pictures, audios, and videos. Therefore, supporting multiple types of file downloads is a core requirement HTTP/1.0

In order for the client and server can communicate deeper, HTTP/1.0 introduced request headers and response headers, they are all Key-Value save form, in HTTP when the transmission request, the request will bring header information, the server returns the data, will first Return the response header information. HTTP/1.0 specific request process of 060cb39528241d, please refer to the following figure:


HTTP/1.0 once for each HTTP communication, we need to undergo establish TCP connection, transmission HTTP data and disconnect TCP three phases are connected. At that time, because the communication files were relatively small and there were not many references to each page, there was no major problem with this form of transmission. But with the popularity of browsers, more and more image files in a single page, sometimes a page may contain hundreds of externally referenced resource files, if you download each file, you need to go through the establishment of a TCP connection transmitting data and disconnecting will undoubtedly increase a lot of unnecessary overhead.

To solve this problem, HTTP/1.1 added to a lasting connection, it is characterized by a TCP connection can transmit multiple HTTP request, as long as the browser or the server is not explicitly disconnected, then the TCP connection will remain. And for the same domain name in the browser, by default, 6 TCP persistent connections are allowed to be established at the same time.

These methods have greatly improved the download speed of the page to some extent.

Before we used Webpack package the application code into a bundle.js , there is a very important reason: scattered module files will generate a large number of HTTP requests. However, a large number of HTTP requests will cause concurrent requests for resources on the browser side:

As shown in the figure above, the requests circled in red are concurrent requests, but the subsequent requests are suspended for a period of time because the number of domain name connections has exceeded the limit.

Under the HTTP1.1 standard, each request requires a separate TCP connection. After a complete communication process, it is very time-consuming. The reason for this problem is mainly caused by the following three reasons:

• Slow start of TCP
• TCP Competing for bandwidth between connections
• Head-of-line blocking

The first two problems are caused TCP itself, and the head-of-line blocking is caused by the mechanism of HTTP/1.1

In order to solve these known problems, HTTP/2 idea is a domain name using only one TCP long connection to transfer data, so the download process the entire page resource only once slow start, but also to avoid more TCP connection compete for bandwidth brought The problem.

It is often said that multiplexing, it can realize the parallel transmission of resources.

The above also mentioned that Vite uses ESM to use the module in the browser, which is to use HTTP request the module. This will generate a large number of HTTP requests, but due to the HTTP/2 of the multiplexing mechanism of 060cb3952825c2, the problem of long transmission time is solved.

ESBuild

esbuild official introduction: it is a JavaScript Bundler packaging and compression tool, it can JavaScript and TypeScript code to run on the web.

esbuild bottom layer of golang is written using 060cb395282617, which has obvious advantages compared with the packaging speed of web The speed of compiling Typescript far exceeds the official tsc .

For JSX , or TS file needs to be compiled and the like, Vite is esbuild to compile, unlike Webpack overall compiled Vite in browser requests, fishes compiled files, and then provided to the browser. Because esbuild compiles fast enough, this kind of compiling every time the page loads does not affect the acceleration speed.

Vite implementation principle

Combination of the above analysis and source sentence to be briefly Vite principles: Static Server + Compile + HMR :

• Use the current project directory as the root directory of the static file server

• Intercept some file requests

  • Process the modules import node_modules
  • Handle the vue single file component ( SFC )
• By WebSocket achieve HMR

Of course, there are already many articles about the implementation of handwritten Vite , so I won't repeat them here, the general principles are the same.

to sum up

The writing of this article brings me more thoughts. From a sharing to discover the huge ecosystem behind it and those technology black boxes that we have been using but have not understood deeply.

What's more, I sigh the ideas of the big guys, stand on the commanding heights of technology, have a higher depth and breadth, and develop some extremely useful wheels for improving productivity.

So, the article is finished, and the pace of learning is still advancing~

reference

• https://hacks.mozilla.org/2018/03/es-modules-a-cartoon-deep-dive/
• https://github.com/evanw/esbuild
• Geek Time/Luo Jianfeng/Perspective HTTP Protocol