Implement dynamic import in React Native

Image credit: https://unsplash.com/photos/95YRwf6CNw8

Author of this article: ssskkk

background

With the development of the business, the amount of code of each React Native application is increasing, the volume of the bundle is expanding, and the negative impact on the application performance is becoming more and more obvious. Although we can unpack through the React Native official tool Metro, split into a basic package and a business package for a certain degree of optimization, but we can't do anything about the growing business code, we urgently need a set of solutions to reduce Smaller size of our React Native app.

multi-service package

The first thing that comes to my mind is to split multiple business packages. Since it is not enough to split into one business package, I can split it into several business packages. When a React Native application is split into multiple business packages, it is actually equivalent to splitting into multiple applications, except that the code is in the same repository. Although this can solve the problem of the continuous expansion of a single application, it has many limitations. Next, analyze them one by one:

• For link replacement, different applications require different addresses, and the replacement cost is high.
• Communication between pages is a single-page application before, and different pages can communicate directly; after splitting, communication between different applications needs to be realized by client-side bridging.
• Performance loss, opening each split business package requires a separate React Native container, which consumes memory and occupies CPU for container initialization and maintenance.
• The granularity is not enough, and the smallest dimension is also the page, and the components in the page cannot be split.
• Repeated packaging, part of the tool library shared between different pages, will be included in each business package.
• Packaging efficiency, the packaging process of each business package must go through a complete Metro packaging process, and the packaging time for splitting multiple business packages is multiplied.

dynamic import

Another solution that comes to mind as a front-end is naturally dynamic import. Based on its dynamic characteristics, this solution can be avoided for many shortcomings of multi-service packages. In addition, with dynamic import, we can achieve page on-demand loading, component lazy loading and other capabilities. However, Metro does not officially support dynamic import, so Metro needs to be deeply customized, which is also the implementation of dynamic import in React Native that will be introduced in this article.

Metro packaging principle

Before introducing the specific solution, let's take a look at Metro's packaging mechanism and its construction products.

Packaging process

As shown in the figure below, Metro packaging will go through three stages, namely Resolution, Transformation, and Serialization.

The role of Resolution is to build a dependency graph from the entry point; Transformation is executed at the same time as the Resolution phase, and its purpose is to convert all modules (a module is a module) into a language recognized by the target platform, including the conversion of advanced JavaCript syntax. (depending on BaBel), there are also special polyfills for specific platforms, such as Android. These two stages mainly produce intermediate product IR for consumption in the last stage.

Serialization combines all modules to generate bundles. Special attention should be paid to the two configurations in Serializer Options in the Metro API document:

• The signature is createModuleIdFactory and the type is () => (path: string) => number . This function generates a unique moduleId for each module, which is an auto-incrementing number by default. All dependencies rely on this moduleId.
• The signature is processModuleFilter and the type is (module: Array<Module>) => boolean . This function is used to filter modules and decide whether to enter the bundle.

bundle analysis

A typical React Native bundle can be divided into three parts from top to bottom:

• The first part is polyfills, mainly some global variables such as __DEV__ ; and some important global functions declared through IIFE, such as: __d , __r , etc.;
• The second part is the definition of each module, which starts with __d , and the business code is all in this block;

• The third part is the initialization of the application __r(react-native/Libraries/Core/InitializeCore.js moduleId) and __r(${入口 moduleId}) .
  Let's look at the analysis of specific functions

  __d function

   function define(factory, moduleId, dependencyMap) {
    const mod = {
        dependencyMap,
        factory,
        hasError: false,
        importedAll: EMPTY,
        importedDefault: EMPTY,
        isInitialized: false,
        publicModule: {
            exports: {}
        }
    };
    modules[moduleId] = mod;
  }

  __d其实就是define函数，可以看到其实现很简单，做的mode ， moduleId与mode Do a layer of mapping, so that you can get the module implementation through moduleId . Let's see __d how to use:

 __d(function (global, _$$_REQUIRE, _$$_IMPORT_DEFAULT, _$$_IMPORT_ALL, module, exports, _dependencyMap) {
    var _reactNative = _$$_REQUIRE(_dependencyMap[0], "react-native");

    var _reactNavigation = _$$_REQUIRE(_dependencyMap[1], "react-navigation");

    var _reactNavigationStack = _$$_REQUIRE(_dependencyMap[2], "react-navigation-stack");

    var _routes = _$$_REQUIRE(_dependencyMap[3], "./src/routes");

    var _appJson = _$$_REQUIRE(_dependencyMap[4], "./appJson.json");

    var AppNavigator = (0, _reactNavigationStack.createStackNavigator)(_routes.RouteConfig, (0, _routes.InitConfig)());
    var AppContiner = (0, _reactNavigation.createAppContainer)(AppNavigator);

    _reactNative.AppRegistry.registerComponent(_appJson.name, function () {
        return AppContiner;
    });
}, 0, [1, 552, 636, 664, 698], "index.android.js");

This is the only use of __d to define a module. The input parameters are explained here. The first one is a function, which is the factory function of the module. All business logic is in it. It is called after __r ; ID; the third part is the moduleId of the module it depends on; the fourth part is the file name of this module.

__r function

 function metroRequire(moduleId) {

    ...

    const moduleIdReallyIsNumber = moduleId;
    const module = modules[moduleIdReallyIsNumber];
    return module && module.isInitialized
        ? module.publicModule.exports
        : guardedLoadModule(moduleIdReallyIsNumber, module);
}

function guardedLoadModule(moduleId, module) {

    ...
    
    return loadModuleImplementation(moduleId, module);
}

function loadModuleImplementation(moduleId, module) {

    ...

    const moduleObject = module.publicModule;
    moduleObject.id = moduleId;
    factory(
        global,
        metroRequire,
        metroImportDefault,
        metroImportAll,
        moduleObject,
        moduleObject.exports,
        dependencyMap
    ); 
    return moduleObject.exports;

    ...
}

__r is actually the require function. As shown in the simplified code above, the require method first determines whether the module to be loaded already exists and the initialization is completed, if so, it returns the module directly, otherwise it calls the guardedLoadModule method, and the final call is loadModuleImplementation method. loadModuleImplementation factory passed in when the module definition is obtained and called, and finally returned.

Design

Based on the above analysis of the working principle of Metro and its product bundle, we can roughly draw the following conclusion: When React Native starts, the JS test (ie bundle) will initialize some variables first, and then declare the core method through IIFE define require ；接着通过define方法定义所有的模块，各个模块的依赖关系通moduleId ，维系的纽带就是require ; Finally, the startup is realized by the registration method of require application.

Realizing dynamic import naturally requires re-splitting and recombining the current bundles. The key point of the whole scheme is: splitting and combining, splitting is how to split bundles, what modules need to be split, when to split and split Where is the subsequent bundle stored (involving how to obtain it later); merging is how the detached bundle is obtained, and it is still executed in the correct context after it is obtained.

Minute

As mentioned earlier, there are three stages of Metro work, one of which is Resolution. The main task of this stage is to build the entire application dependency graph from the entry point, which is replaced by a tree for convenience.

Identify the entrance

As shown above, it is a dependency tree. Under normal circumstances, a bundle will be typed, including modules A, B, C, D, E, F, and G. Now I want to do dynamic imports for modules B and F. How to do it? Of course, the first step is to identify, since it is called dynamic import, we naturally think of dynamic import in JavaScript syntax.
Just need to change import A from '.A' to const A = import('A') , which requires the introduction of the Babel plugin (), in fact, the official Metro related configuration package metro-config has integrated this plugin. The official does not only do this, but also adds a unique identifier Async = true to the dynamically imported module in the Transformation stage.

In addition, Metro provides a file template named AsyncRequire.js on the final product bundle to polyfill the dynamic import syntax. The specific implementation is as follows

 const dynamicRequire = require;

module.exports = function(moduleID) {
    return Promise.resolve().then(() => dynamicRequire.importAll(moduleID));
};

Summarize how Metro handles dynamic import by default: In Transformation, use the Babel plugin to process dynamic import syntax, and add an identifier Async to the intermediate product, and use Asyncrequire.js as a template to replace the dynamic import syntax in the Serialization phase. which is

 const A = import(A);

变为

const A = function(moduleID) {
    return Promise.resolve().then(() => dynamicRequire.importAll(moduleID));
};

Asyncrequire.js is not only about how we split, but also about our final integration, which will be discussed later.

tree split

From the above, we know that a dependency tree will be generated during the construction process, and the dynamically imported modules will be identified. The next step is how to split the tree. The general processing method for trees is DFS. After DFS analysis of the dependency tree in the above figure, the following split tree can be obtained, including one main tree and two asynchronous trees. Collecting the dependencies of each tree can get the following three sets of modules: A, E, C; B, D, E, G; F, G.

Of course, in the actual scenario, the dependencies of each module are far more complicated than this, and even there are circular dependencies. In the process of doing DFS, two principles need to be followed:

• Modules that are already being processed, and then exit the loop directly when encountered

• The non-main tree modules that each asynchronous tree depends on need to be included

  bundle generation

  Three bundles can be obtained through these three sets of module sets (we call the bundle generated by the main tree the main bundle; the one generated by the asynchronous tree is called the asynchronous bundle). As for how to generate it, you can directly use the processBasicModuleFilter method in Metro mentioned above. Metro originally generated a bundle through the Serialization stage only once in a build process. Now we need to generate a bundle for each set of modules.

There are several issues to note here:

• To remove duplication, one is that the module asynchronous bundle that has been entered into the main bundle does not need to be entered; the other is the module that exists in different asynchronous trees at the same time. For this kind of module, we can mark it as dynamic import and package it separately, see The following figure
  
• In the generation order, the asynchronous bundle needs to be generated first, and then the main bundle is generated. Because it is necessary to map the information of the asynchronous bundle (such as file name, address) and the moduleId into the main bundle, so that the address information of the asynchronous bundle can be obtained through the mapping of the moduleId when it is really needed.
• Cache control, in order to ensure that each asynchronous bundle can be updated in time while enjoying the caching mechanism, it is necessary to add the content hash of the asynchronous bundle to the file name
• Storage, how to store asynchronous bundles, together with the main bundle, or separately, and get them when needed. This requires specific analysis: for those who use bundle preloading, the asynchronous bundle and the main bundle can be put together, and can be taken directly from the local when needed (the so-called preloading means that all bundles have been downloaded when the client starts, No need to download the bundle when the user opens a React Native page). Separate storage is more suitable for most of those that do not use preloading technology.

So far we have obtained the main bundle and the asynchronous bundle, the general structure is as follows:

 /* 主 bundle */

// moduleId 与 路径映射
var REMOTE_SOURCE_MAP = {${id}: ${path}, ... }

// IIFE __r 之类定义
(function (global) {
  "use strict";
  global.__r = metroRequire;
  global.__d = define;
  global.__c = clear;
  global.__registerSegment = registerSegment;
  ...
})(typeof global !== 'undefined' ? global : typeof window !== 'undefined' ? window : this);

//  业务模块
__d(function (global, _$$_REQUIRE, _$$_IMPORT_DEFAULT, _$$_IMPORT_ALL, module, exports, _dependencyMap) {
  var _reactNative = _$$_REQUIRE(_dependencyMap[0], "react-native");
  var _asyncModule = _$$_REQUIRE(_dependencyMap[4], "metro/src/lib/bundle-modules/asyncRequire")(_dependencyMap[5], "./asyncModule")
  ...
},0,[1,550,590,673,701,855],"index.ios.js");

...

// 应用启动
__r(91);
__r(0);

 /* 异步 bundle */

// 业务模块
__d(function (global, _$$_REQUIRE, _$$_IMPORT_DEFAULT, _$$_IMPORT_ALL, module, exports, _dependencyMap) {
  var _reactNative = _$$_REQUIRE(_dependencyMap[0], "react-native");
  ...
},855,[956, 1126],"asyncModule.js");

combine

In fact, most of the work has been done in this stage, and the next step is how to combine it. As mentioned earlier, the dynamic import syntax will be replaced by the template in AsyncRequire.js in the generated bundle. Careful study of its code found that it was implemented with a layer of --- Promise wrapped require(moduleId) .
Now we directly require(moduleId) must not get the real module implementation, because the asynchronous bundle has not been obtained, and the module has not been defined. But the following modifications can be made to AsyncRequire.js

 const dynamicRequire = require;
module.exports = function (moduleID) {
    return fetch(REMOTE_SOURCE_MAP[moduleID]).then(res => {  // 行1
        new Function(res)();                                 // 行2
        return dynamicRequire.importAll(moduleID)            // 行3
    });
};

The next line of analysis

• Line 1 replaces the previous mock's Promise with a real Promise request, and first obtains the bundle resource, REMOTE_SOURCE_MAP is the mapping between the moduleId written to the main bundle and the asynchronous bundle resource address during the generation phase. fetch According to the different storage methods of asynchronous bundles, choose different ways to obtain real code resources;
• Line 2 executes the obtained code through the Function method, which is the declaration of the module, so it is already defined when the module is finally returned;
• Line 3 returns the real module implementation.
  In this way, we have realized the combination , and the acquisition and execution of asynchronous bundles are all completed in AsyncRequire.js.

Summarize

So far, we have completed the transformation of React Native dynamic import. Compared with the multi-service package, because of its dynamic characteristics, all modifications are completed in the same React Native application in a closed loop, and there is no external perception, so many defects of the multi-service package do not exist. At the same time, the IR of the first production will be fully utilized during construction, so that each bundle does not need to go through the complete construction process of Metro separately.

Of course, there is one thing that must be considered, that is, after we transform Metro, whether it will affect the subsequent upgrades, so that only React Native and Metro versions can be locked. In fact, there is no need to worry about this. From the previous analysis, we can know that our transformation of the entire process can be divided into two parts: construction time and runtime. We did add a lot of capabilities during the build, such as new grouping algorithms and code generation; but the runtime is completely based on the enhancement of the existing version capabilities. This makes the runtime of dynamic import without compatibility problems. Even if you upgrade to a new version, you will still not report an error, but you will lose the ability to dynamically import before we rebuild the build again.

Finally, there are some engineering transformations that are actually used in the production environment, such as: building platform adaptation, providing quick access components, etc., and will not be described in detail here because of space limitations.

This article is published from the NetEase Cloud Music technical team, and any form of reprinting of the article is prohibited without authorization. We recruit various technical positions all year round. If you are ready to change jobs and happen to like cloud music, then join us at grp.music-fe(at)corp.netease.com!