hel-micro module federation new revolution

hel-micro , module federation sdk, build-free, hot update, toolchain-independent micro-module solution, welcome to pay attention and understand

The origin of the modular federation

Since the Google chrome browser has sprung up, and was officially announced and released on September 2, 2008 v8 js engine, it has swept the Internet world with extremely high operating efficiency, and also Captured a large number of users, this unstoppable momentum made other major technology companies (apple, moliza, microsoft) feel a huge murderousness, and then everyone began to recruit horses, sharpen their swords, and prepare to kill a bloody path. Since then, the js engine has entered During the arms race, Microsoft did not hesitate to IE and began to push the new js engine with Microsoft's countless efforts behind it Chakra edge browser, It is conceivable how much everyone attaches great importance to the cake of the js engine, and the birth of v8 has catalyzed a large number of famous open source works, so that the js ecology has always maintained a very strong vitality, among which the most The famous one was born in 2009 nodejs , a server-side js runtime based on v8 , let the js language start to bloom from the foreground to the background, so that the following sentence is very early The ridicule that was born is still circulating today:

 Any application that can be written in JavaScript, willeventually be written in JavaScript.

Modular Specification

nodejs brings commonjs modularization to the server side, making large-scale js projects organized more orderly, and also bringing npm this super killer, Sharing efficiency has increased to unprecedented heights.

The front-end application has also doubled with the complexity of network applications, which has led to a period of rapid expansion of code volume. At this time, an effective modular solution is urgently needed to solve how to split modules gracefully, and how to improve code reuse and scalability. maintenance and other issues.

At this time, the two mainstream modular solutions amd and cmd started to compete in the front end and finally established a large position. Their representative implementations requirejs and seajs , I believe many friends have known or used it.

Engineering system

Although requirejs and seajs have brought the realization of the modular specification to the front end and injected a solid foundation for the large-scale js project, the modular specification alone still cannot solve how to integrate npm Ecological interoperability, how to manage increasingly complex module dependencies, if compatible with new js features and a series of issues, in the final analysis, this involves a keyword engineering system , then webpack And babel was born, their goals are very clear, and they solve the 4 major problems in the following figure

Then it became the de facto infrastructure standard for the front-end to start engineering system development.

webpack Relying on the excellent plug-in and loader mechanism, the ecology around it can continue to grow bigger and stronger, and eliminate other processes that are more tool-oriented gulp , grunt and many other opponents

The curse of npm

webpack and npm almost form a perfect partner state, but the front-end originally obtained from cdn brought the fatal resources update and deployment efficiency issues.

In some scenarios that need to be dynamically updated, this all in one npm mechanism greatly webpack the deployment efficiency of the package body. The problem is the typical scenario requirements brought by people's natural requirements for rapid iteration and express experiments in the web environment.

Note: externals itself cannot completely solve the demands of dynamic update, it is only suitable for externally linking the underlying PR dependency package to cdn

bundless is coming

At the same time webpack as the size of the project becomes larger and larger, new problems such as poor development experience (slow hot update), increased package size, and slow build speed (node\_modules black hole) are also born. At this time The new generation of development tools snow and vite began to webpack the market of ---3599ab6d8411f82e3c00649385374d28--- in the name of not being packaged.

They all take advantage of the browser's native modular ability esm , skip the required dependency analysis and packaging process of webpack , and achieve millisecond-level debugging and startup under this design.

But the extremely fast experience they bring cannot shake the whole webpack the deep siege of the ecology, in fact, everyone is debugging based on vite and the production packaging is still used webpack dual-engine drive mode, after all esm will take time to popularize.

The module federation sounded the horn of counterattack

Since everyone complains about the slow build wepack is there a way to skip the build step and allow users to combine multi-level dependency modules in their own way?

Of course there is, that is to take the pre-built path

模块联邦 so it was born. Its greatness lies in maintaining the current high-efficiency system of modularization, componentization and engineering of front-end development, allowing modules to be developed and deployed independently, and shared directly through CDN, thus breaking free The shackles that npm packages cannot be dynamically updated, thus promoting the development and operation experience of the entire front-end world to a new level.

As long as there are more modules that can be promoted to the federation, the local startup speed will be faster!

Moreover, federated modules are born with dual identities, that is, they can be module consumers or module providers, which makes a natural grid relationship between module federation applications, and module distribution efficiency, deployment efficiency, and sharing efficiency are unprecedented. Boost!

The Achilles Heel of the Modular Federation

webpack 5 Or is the implementation of module federation brought by other tools really perfect? It does solve the problem of 免构建 , 动态更新 , 跨项目共享模块

However, based on the existing compile-time plug-in mechanism, the problem of 工具链强绑定 and 编译时确定才能远程模块消费关系 cannot be avoided.

Just imagine, you need to use the technology of module federation, how many preconditions you need to do, and you need to upgrade a real toolchain! And the federation modules before different toolchains are mutually exclusive! The liquidity of modules is tied to the toolchain of your choice.

Modular Federation New Revolution

The only solution to these two major problems is to put them sdk化 , which is hel-micro a brand-new thinking on the realization of the module federation, and also the secret weapon to initiate a new revolution of the module federation.

After sdk化 , any technology stack and any tool chain can access the module federation technology losslessly and painlessly.

Module consumption relationship at runtime

The return from the tool chain to the js language itself means that the module consumption relationship is upgraded from compile time to runtime, which will greatly improve the flexibility of dynamic loading of remote modules and enable more complex services.

Dimensionality reduction strike

Compared with the module federation implemented by relying on tool plug-ins, hel-micro the implementation from the language level will cause a dimensionality reduction blow to other module federation implementations.

Compared with the traditional npm sharing method, hel-micro also has more efficient code sharing capabilities (runtime sharing)

Decrypt the sdkized core implementation

To achieve sdkization means that we must tap the hidden capabilities of the js language itself, and jump out of the traditional packaging process thinking, in order to achieve our ultimate goal

The hidden ability of asynchronous import

Usually we will use the import keyword in the header file to import other modules statically, but in fact import can be used as a function call, import a module asynchronously, and return a promise object

 const mod = await import('./some-mod');

So we can fine-tune the loading order of modules to achieve the effect of injecting new code into a module before it is statically imported by other modules

And the early injection effect brought by this asynchronous import has become the key implementation point of ---a8cfd8662f4559082ff92a1879d4958b hel-micro injecting remote runtime code into the module proxy object, so that hel-micro can provide lazy loading and preloading for users Two loading methods.

There are two core interfaces in the above figure: libReady interface is responsible for exposing modules, preFetchLib interface is responsible for pulling modules, and each module acts as a provider or consumer by calling the behavior of the interface .

Runtime Dependency Analysis

hel-micro 事件总线 、 模块池 、 样式池 、 元数据池四个数据结构，让有多Efficient, safe and orderly loading of remote modules in the dependency hierarchy.

Among them 模块池 can ensure that the module will not be repeatedly loaded and reused by the upper-layer callers.

Metadata - the soul of modules

The essence of the module is to build a collection of product files, hel-micro By providing plugins during construction, the network path of the product is collected and stored according to the protocol specified by the sdk, so that the sdk can be downloaded and executed on the network later. All remote modules.

Double build mechanism

hel-micro使用---ed82c6521d1e35ff431dbcea68584c29 rollup代理文件， webpack注入的实际运行代码， node_modules The proxy module object in node_modules gets complete type hints, so that users can get the ultimate development experience of using remote modules just like using local modules

内定了4个目录hel_dist ， hel_proxy ， hel_proxy_es ， hel_bundle来承载不同的产物，供package.json Configure different portals.

Among them, the files in the hel_proxy , hel_proxy_es directory are the entry files of the module proxy object I mentioned. We can see that the file is almost an empty shell, so it has no effect on the module The impact of the packaging size of the user can almost be ignored.

Platform and Ecology

hel-mirco sdk mainly relies on the standardized metadata format for remote module loading, so as long as any user provides the metadata of the module according to the standardized format, it can be loaded by hel-micro , which will be very beneficial to surrounding hel- The construction and development of the upper-level ecology of micro.

Supports arbitrary deployment of modules

The sdk is decoupled from the platform. By default, we provide support for and hel pack and npm unpkg , but it allows you to publish modules to any file service on the network, just know Its deployment address is sufficient.

For example, if the user saves the metadata of hel-meta.json to the background database (which can be combined with the devops pipeline), a centralized module control platform similar to hel pack can be quickly built

The centralized module control platform is very convenient for modules to implement version preview , grayscale volume , and second-level rollback , but it does not prevent the sdk from loading multi-platform packages in a de-neutral way, because the sdk inherently supports simultaneous download from When multiple different platforms pull remote modules and use them, such as loading modules from two platforms, unpkg and hel pack at the same time, the platform value will be used as a namespace to isolate modules that may have the same name on different platforms.

Upper-level ecological construction

hel-micro It only provides the ability to load remote modules. The specific ui adaptation layer also needs to be implemented in the upper encapsulation library area. The current plan is as follows

Taking hel-micro-react as an example, it provides lazy loading of remote components in the form of react hook function, and also provides shadow dom style isolation function

When to adopt hel-micro

At the moment when you encounter any of the following situations, using hel-micro is definitely worth your effort.

roadmap 2022~2023

Epilogue

The module federation can play an even more powerful role in building super-large 微容器 projects, and the module deployment efficiency and sharing efficiency of giant applications will be easily solved. The operation needs are escorted, welcome to star hel-micro , understand and use it.

_

My other open source works friend chain (welcome to follow and understand):

concent , a react data flow solution with built-in dependency collection and setup features

limu , a library for js immutable operations that is more efficient than immer