Why is Esbuild so fast

The full text is 2500 words, and it takes about 20 minutes to read. Welcome to like, follow and forward.

What is Esbuild

Esbuild is a very new module packaging tool, which provides the Webpack, Rollup, Parcel and other tools similar resource pack ability, but has a ridiculously high performance advantages:

From top to bottom, the time-consuming gradually increased to a difference of hundreds of times. This huge performance advantage made Esbuild quickly become popular among many Node-based build tools. Especially after Vite 2.0 announced the use of Esbuild pre-build dependencies, the front-end community Its discussion has gradually increased.

So the question is, how is this done? After reading a lot of information, I summarized some key factors:

Let's talk about it in detail.

Why fast

Language advantage

Most front-end packaging tools are implemented based on JavaScript, while Esbuild chooses to use Go language. Both languages have their own good scenarios. However, in the CPU-intensive scenario of resource packaging, Go has more performance advantages, and there are gaps. How old is it? For example, calculate 50 times Fibonacci sequence, JS version:

function fibonacci(num) {
    if (num < 2) {
        return 1
    }
    return fibonacci(num - 1) + fibonacci(num - 2)
}

(() => {
    let cursor = 0;
    while (cursor < 50) {
        fibonacci(cursor++)
    }
})()

Go version:

package main

func fibonacci(num int) int{
    if num<2{
        return 1
    }
    
    return fibonacci(num-1) + fibonacci(num-2)
}

func main(){
    for i := 0; i<50; i++{
        fibonacci(i)
    }
}

The execution time of the JavaScript version is about 332.58s , and the execution time of the Go version is about 147.08s . The difference between the two is about 1.25 times. This simple experiment cannot accurately quantify the performance difference between the two languages, but the sensory It is still obvious that Go language will have better performance in CPU-intensive scenarios.

In the final analysis, although the modern JS engine has been greatly improved compared with 10 years ago, JavaScript is still an interpreted language in essence. Every time a JavaScript program is executed, the interpreter must first translate the source code into machine language. Scheduling execution; and Go is a compiled language. The source code has been translated into machine code during the compilation stage, and only these machine codes need to be executed directly at startup. This means that programs written in Go language have one less process of dynamic interpretation than JavaScript.

This language-level difference is particularly prominent in the packaging scenario. It is an exaggeration. While the JavaScript runtime is still interpreting the code, Esbuild is already parsing the user code; when the JavaScript runtime is finished explaining the code and just preparing to start, Esbuild may have already started. The packaging is complete, and the process is exited!

Therefore, at the level of compilation and operation, Go has a source code compilation process ahead of its time, which has higher execution performance than JavaScript's way of interpreting and running.

Multithreading advantage

Go is inherently capable of multi-threaded operation, and JavaScript is essentially a single-threaded language. It is not possible to implement multi-threaded operations in browsers and Node until the introduction of the WebWorker specification.

I have studied the code of Rollup and Webpack, and neither of them uses the multi-threading capabilities provided by WebWorker to the extent I know it. In contrast, Escbuild’s core selling point is performance. Its implementation algorithm has been carefully designed to use each CPU core as saturated as possible. In particular, the parsing and code generation stages of the packaging process have been fully parallelized.

In addition to the parallelism at the CPU instruction level, multiple threads of the Go language can also share the same memory space, and each thread of JavaScript has its own unique memory heap. This means that multiple processing units in Go, such as the thread parsing resource A, can directly read the running result of resource B thread, and the same operation in JavaScript requires calling the communication interface woker.postMessage to copy data between threads.

So at the runtime level, Go has natural multi-threading capabilities and more efficient memory usage, which means higher running performance.

control

Yes, yes, temperance!

Esbuild is not another Webpack. It only provides the minimal set of functions required to build a modern web application. In the future, it will not add various build features that we are already familiar with on a large scale. The main features of the latest version of Esbuild are:

• Support js, ts, jsx, css, json, text, pictures and other resources
• Incremental update
• Sourcemap
• Development server support
• Code compression
• Code split
• Tree shaking
• Plug-in support

It can be seen that the resource types and engineering features supported in this list are very few, and they are not even sufficient to support the development needs of a large-scale project. In addition to this, the official website clearly stated that there are no plans to support the following features in the future:

• Elm , Svelte , Vue , Angular and other code file formats
• Ts type check
• AST related operation API
• Hot Module Replace
• Module Federation

Moreover, the plug-in system designed by Esbuild does not intend to cover these scenarios, which means that third-party developers cannot plug-in a non-invasive way. emmm, it can be foreseen that there may be many magical modified versions in the future. .

Esbuild only solves a part of the problem, so its architecture complexity is relatively small, and the relative coding complexity will be much smaller. Compared with the unified tools such as Webpack and Rollup, it is naturally easier to achieve the ultimate performance. The modest functional design can also bring another benefit: a variety of additional tools fully customized for performance.

custom made

To recap, in tools such as Webpack and Rollup, we have to use many additional third-party plug-ins to solve various engineering needs, such as:

• Use babel to implement ES version translation
• Use eslint to implement code inspection
• Use TSC to implement ts code translation and code inspection
• Use less, stylus, sass and other css preprocessing tools

We are completely accustomed to this approach, and even think that things should be like this. Most people may not realize that there is another solution for things. Esbuild started, the choice was complete! Completely rewrite all the tools needed for the entire compilation process! This means that it needs to rewrite the loading, parsing, linking, and code generation logic of resource files such as js, ts, jsx, and json.

The development cost is very high, and it may passively fall into the risk of closure, but the benefits are also huge. It can implement the principles all the way and customize the various stages of compilation with performance as the highest priority, for example:

• Rewrite the ts translation tool, completely abandon ts type checking, and only do code conversion
• Most packaging tools disassemble lexical analysis, grammatical analysis, symbol declaration and other steps into multiple processing units with high cohesion and low coupling. Each module has clear responsibilities, high readability and maintainability. While Esbuild adheres to the performance first principle, it does not hesitate to adopt a counter-intuitive design model to mix multiple processing algorithms to reduce the performance loss caused by data flow in the compilation process.
• Consistent data structure, and derived efficient caching strategy, detailed in the next section

On the one hand, this in-depth customization reduces design costs and can maintain the consistency of the compilation chain; on the other hand, it can implement the principle of performance first to ensure the optimal performance of each link and interaction between links. Although it is accompanied by sacrifices in terms of function, readability, and maintainability, it has almost reached the extreme in terms of compilation performance.

Structural consistency

In the previous section, we mentioned that Esbuild chose to rewrite translation tools including js, ts, jsx, css and other languages, so it can better ensure the structural consistency of the source code between the compilation steps, such as using babel- in Webpack. When the loader processes JavaScript code, it may need to go through multiple data conversions:

• Webpack reads the source code, in the form of a string at this time
• Babel parses the source code and converts it to AST format
• Babel converts source code AST to lower version AST
• Babel converts the low version AST generate into the low version source code, in the form of a string
• Webpack parses low version source code
• Webpack packages multiple modules into the final product

The source code needs to go through string => AST => AST => string => AST => string , jumping repeatedly between the string and the AST.

After Esbuild rewrites most of the translation tools, it can share similar AST structures in multiple compilation stages, minimize the conversion of strings to AST structures, and improve memory efficiency.

to sum up

Purely from the perspective of compilation performance, Esbuild really beats all packaging frameworks in the world, and the gap can even be as large as a hundred times:

	time consuming	Performance difference	speed	Product size
esbuild	0.11s	1x	1198.5 kloc/s	0.97mb
esbuild (1 thread)	0.40s	4x	329.6 kloc/s	0.97mb
webpack 4	19.14s	174x	6.9 kloc/s	1.26mb
parcel 1	22.41s	204x	5.9 kloc/s	1.56mb
webpack 5	25.61s	233x	5.1 kloc/s	1.26mb
parcel 2	31.39s	285x	4.2 kloc/s	0.97mb

But this comes at a price. In addition to the natural advantages at the language level, at the functional level, it directly abandons the support for resources such as less, stylus, sass, vue, angular, and other functions such as MF, HMR, and TS type checking, just as the author said Say:

This will involve saying "no" to requests for adding major features to esbuild itself. I don't think esbuild should become an all-in-one solution for all frontend needs\!

In my opinion, Esbuild cannot replace Webpack now and in the future. It is not suitable for direct use in the production environment. It is more suitable as a low-level module packaging tool. It needs to be packaged twice on its basis to expand a set. A tool chain with both performance and complete engineering capabilities, such as Snowpack , Vite , SvelteKit , Remix Run etc.

In general, Esbuild provides a new design idea that is worth learning and understanding, but it is not suitable for direct production use for most business scenarios.