Dry goods | Youku Mini Program Optimization Actual Combat

Author: Uprooting

Youku App is connected to the Alipay applet framework, which expands the capabilities of Youku App. However, due to the difference in the runtime environment of the Youku App and Alipay App platform during the built-in applet SDK process, the following major problems have arisen:

• The small program sdk package is large, which greatly increases the package size of Youku APP;
• After the applet container was started, the number of threads increased sharply, superimposing the threads of Youku's main APP scene, causing the crash rate to increase;
• Initializing the applet engine will affect the startup speed and memory usage of Youku APP.

In order to solve the above problems, Youku will have to deal with different aspects of package size, number of threads, memory, etc. Next, this article will introduce Youku's solutions when facing these differences and problems.

Remote SO

During the built-in process, we found that the Alipay applet framework occupies about 23MB. The ideal solution is to fully remote, so that it does not occupy the package size of Youku App, but currently Youku does not have the conditions for all remoteization in a short period of time. Through analysis, we found that so in the 23MB space occupies 7MB, and relative to the code and resource files, so is relatively independent. So our plan is as follows:

• There are dependencies between so, which need to be collected in advance and loaded in order when loading;
• Exclude these SOs during the packaging process, upload them to the server, and record SO related information. Including the remote download address and md5 value after uploading;
• When the user opens the APP and enters the applet, download these so from the server and store it in the specified directory;
• Load so in sequence according to the pre-collected dependencies.

Among them, the more complicated is the so dependency analysis. Our processing process is as follows:

1. Analyze dependencies

objdump -x *.so|grep -i needed|awk '{print $2}'

objdump is a command for disassembling object files or executable files under Linux. It allows you to learn more about the additional information that a binary file may contain in a readable format, similar to the readelf command. Some of the dependent so are in the operating system, and some are in other aar packages, and the operating system does not need to be concerned. The so in other aar packages needs to be recorded.

2. Collect dependencies:

aar package name	SO name	Rely on SO name	Effective dependence
com.AAA:aaa-build	libA1.so	libopenssl.so libandroid.so liblog.so libm.so libstdc++.so libEGL.so libGLESv2.so libOpenSLES.so libz.so libdl.so libc.so	libopenssl.so
	libA2.so	libB1.so liblog.so libandroid.so libstdc++.so libm.so libc.so libdl.so	libB1.so
com.BBB:bbb-build	libB1.so	libopenssl.so libc++_shared.so liblog.so libz.so libc.so libm.so libdl.so

You can refer to the above table to collect dependent information for each so, and finally the relationship between these so can form a mesh type. As follows:

3. Hierarchical dependency:

The design principles of hierarchical dependencies are as follows:

• The first layer: it is not allowed to rely on the so in any other aar package, and only the so that depends on the operating system or the so that does not depend on it is allowed;
• The second layer: only allow to rely on the so collected in the first layer and the so of the operating system;
• The third layer: Only the so collected by the first and second layers and the so of the operating system are allowed.

So on and so forth.

The first layer (xx)	The second layer (xx)	The third layer (xx)	The fourth layer (xx pieces)	The fifth layer (xx pieces)
A1	B1	C1
A2	B2	C2
A3	B3
A4

4. Code implementation

    private static String[] LIB_NAMES = {
            // 第一层, 不依赖其他so
            "A1",
            "A2",
            "A3",
            "A4",

            // 第二阶段, 依赖前一阶段加载完毕
            "B1",
            "B2",
            "B3",

            // 第三阶段, 依赖前两个阶段加载完毕
            "C1",
            "C2",

            // 第四阶段, 依赖前三个阶段加载完毕
            ...

            // 第五阶段, 依赖前四个阶段加载完毕
            ...
    };

    public static void ensureAllSoLoaded() {
        try {
            for (String lib_name : LIB_NAMES) {
                Log.d(TAG, "lib_name:" + lib_name);
                System.loadLibrary(lib_name);
            }
        } catch (Throwable throwable) {
            Log.e(TAG, "loadLibrary lib_name exception:");
            throwable.printStackTrace();
        }
    }

5. Summary

In the process of integrating the applet, Youku reduced the package size by 7MB through remote so. Correspondingly, when a user enters the mini program for the first time, a certain amount of waiting time is required, which affects the user experience a little.

Inject the thread pool

In some mobile phones, especially Huawei mobile phones, there is an upper limit for the number of APP threads, and if it exceeds it, it will crash. Based on this, Youku APP manages threads through a unified thread pool. When the number of threads reaches a certain number, tasks will be executed by queuing instead of continuing to add threads.

Due to different business attributes, there are a lot of threads to start on the Youku player page. Once you enter this page during use, the number of threads can reach up to 300+. If there is a small program entry on this page, once the Youku small program is started, the number of threads will increase by about 100 on the original basis. In this way, it is easy to reach the upper limit of the number of APP threads, resulting in java.lang.OutOfMemoryError.

In fact, through the online monitoring platform, there are indeed many crashes caused by this reason. So, solve this problem?

A better solution is to include the threads of the Alipay applet framework into Youku's unified thread pool for management. In this way, when the Alipay applet framework wants to execute a task, it only needs to submit the task to the Youku thread pool for execution, without actually creating a thread. Youku's thread pool decides whether to create a new thread or queue for execution according to the system's situation.

Through communication with colleagues at Alipay, they also agreed with this plan. The Alipay applet framework provides an interface to inject the Youku unified thread pool into the applet framework. When the bottom layer judges that there is already a Youku unified thread pool, it is used directly and no more is created.

The implementation is as follows:

public class MyThreadPoolManager implements IThreadPoolManager {

    private static final int NCPU = Runtime.getRuntime().availableProcessors();
    private ThreadPoolExecutor mThreadPoolExecutor;

    @Override
    public ThreadPoolExecutor createExecutor(ScheduleType scheduleType) {
        if (mThreadPoolExecutor == null) {
            mThreadPoolExecutor = YKExecutorService.from("miniappSdk", 2 * NCPU + 1, 2 * NCPU + 1 ,1000, TimeUnit.MILLISECONDS,new SynchronousQueue<Runnable>());
        }
        return mThreadPoolExecutor;
    }
}
// 支付宝小程序框架提供接口，注入优酷线程池。
TinySdk.setThreadPoolManager(new MyThreadPoolManager())

Before and after the launch, it can be seen from the following data that the effect is very significant. Almost all modules have reduced crashes by 90%. The data comparison effect is as follows:

Lazy loading

In the initial stage of the mini program's launch, the business volume is not large. If you start the Youku APP, the mini program framework will be initialized immediately, which affects the startup speed and wastes memory. Therefore, our strategy is to adopt a lazy loading mode. The process is as follows:

1. Remote dependency check

There are remote dependencies for small programs to run, and downloading these remote dependencies requires traffic and waiting time. Therefore, when we open the Mini Program business for the first time, a pop-up window will inform the user of this information and ask whether he agrees: If the user does not accept it, then do not enter the Mini Program APP and close the pop-up window; if the user accepts, download the remote dependency, and After the download is complete, the follow-up process is automatically completed.

2. Mini program operating condition check

The final loading of the applet page is the rendering kernel. When the operating conditions are checked and it is found that the operating conditions are not met, that is, the cold start of the applet, you must first determine whether the rendering kernel has been loaded (note: the rendering kernel is more than a business dependency on the applet) , It may be loaded by other businesses first).

After the rendering kernel is initialized, we continue to initialize the applet framework. When this step ends normally, the running conditions of the applet are fully available. We can load the applet page normally.

After initialization, through variable identification, when the applet exits, not all the memory will be reclaimed. So the second time and thereafter, it will be hot reloading of small programs. Since the initialization of the rendering kernel and the initialization of the applet framework are omitted, the speed will be much faster.

Summary and outlook

Youku APP mainly provides users with content services. After integrating the Alipay applet framework, it expands the scope of content services that can be provided and further expands Youku's ability as a platform. Youku will continue to work hard in the following technical directions in the future:

1. Package the overall applet framework and put it on the cloud, completely eliminating the influence of the integrated applet framework on the package size of the Youku app.
2. Add more general JSAPI to make it more convenient for business parties.
3. Idle loading replaces lazy loading, providing a better business experience and forming positive feedback.

Follow us, 3 mobile technology practices & dry goods for you to think about every week!