The impact of dex optimization on Arouter's search path

I. Introduction

Question: What is a dex file? What is dex file optimization?

What problems will dex file optimization bring to the project, and how to solve these problems?

1.1 APK compilation and packaging process

1. Pack the resource file res through aapt, and generate R.java, resources.arsc and res files (binary & non-binary keep the original code)

2. Process aidl files and generate java interface files (ignore if there is no aidl)

3. Compile R.java and java interface files through java compile to generate corresponding .class files (java compiler)

4. Use the dex command to convert the .class file and the .class file in the third-party SDK library into a classes.dex file

5. Package the CompiledResources and other resource files and classes.dex files generated by aapt to generate an apk through apkbuilder

6. Similarly, you can use the Jarsigner tool to debug or release the above apk.

The flow chart of apk compilation and packaging is as follows:

In the actual project development, there are two steps 5 and 6, and the release package can be directly generated with the help of the jenkins platform to meet the needs.

1.2 Application scenarios of dex files

Let's take a look at the commonly used scenarios of dex files. The more popular ones are: APK slimming, hot repair, plug-in, application reinforcement, Android reverse engineering, 64 K method limit.

Take the limit of the number of methods as an example. In the fourth step above, the class file is converted into a dex file. By default, only one dex file is generated. The number of methods in a single dex file cannot exceed 65536, otherwise the compilation will report an error, but we are developing the App It will definitely integrate a bunch of libraries. The number of methods is generally more than 65536. The solution to this problem is: one dex cannot be installed, use multiple dex to install, gradle adds a line of configuration: multiDexEnabled true.

There are many application scenarios of dex files on the Internet, so this article will not introduce them. Instead, it analyzes the actual problems encountered in the project, so as to have a further understanding of dex optimization.

Two, dex to vdex, odex

2.1 ART pre-optimization

What is ART (Android runtime)? It is a virtual machine for operating java code and APP. With reference to the history of Android development, Android 5.0 uses ART as the default virtual machine instead of the DVM (Dalvik VM) used in Android 4.4 and before.

Looking back at the relationship between DVM and ART and Android, let's first understand the working mechanism of several virtual machines running Java: (1) JVM: The JVM virtual machine runs Java bytecode. The process from Java files to JVM is: java -> java bytecode(class) -> java bytecode(jar)

DVM : DVM virtual machine parses and executes the dex bytecode. The process from Java files to DVM is: java -> java bytecode(class) -> dalvik bytecode(dex)

ART : ART virtual machine executes local machine code. The process from Java files to ART is: java -> java bytecode(class) -> dalvik bytecode(dex) -> optimized android runtime machine code(oat)

It can be seen that the evolution from DVM to ART is actually the transition from java files to the execution code of the virtual machine. Relatively speaking, ART has more oat processes. ART uses AOT (Ahead-Of-Time) compilation. During an installation, the bytecode is pre-compiled into machine code and stored locally. DVM is compiled using JIT (Just-In-Time). Every time the application is run, the bytecode needs to be converted to the machine immediately by the compiler. Code to continue execution. Compared with DVM, ART saves the process of parsing the bytecode every time, so the memory occupied by the runtime will be reduced, and the operating efficiency of the application will be improved.

2.2 How ART works

In the era of Android 5.0, ART claimed to be able to run the system on a 512M machine by using AOT. Starting from Android 7.0 (N for short), ART combines AOT, just-in-time (JIT) compilation and configuration file guided compilation.

These modes can be combined and configured. Taking Google's Pixel device as an example, the following compilation process is configured:

1) No AOT compilation is performed when the application is initially installed. During the first few runs of the application, the system will interpret it and JIT the frequently executed methods.

2) When the device is idle and charging, the compiling daemon will run to perform AOT compilation of common codes based on the configuration files generated during the first few runs of the application.

The next time the application is restarted, the configuration file-guided code will be used, and JIT compilation of previously compiled methods will be avoided at runtime. Methods that have been JIT-compiled during subsequent runs of the application will be added to the configuration file, and then the compilation daemon will AOT-compile these methods.

ART includes a compiler (dex2oat tool) and a runtime (libart.so) loaded to start Zygote. The dex2oat tool accepts an APK file and generates one or more compiled files, which will then be loaded during runtime. The number of files, extensions, and names will vary from version to version, but in the Android O version, the following files will be generated:

vdex: It contains the uncompressed DEX code of the APK, as well as some metadata designed to speed up verification.

odex: It contains the AOT-compiled method code in the APK.

art (optional): It contains the internal representation of ART of certain strings and classes listed in the APK, which is used to speed up application startup.

2.3 The role of vdex and odex

Decompress an APK (take the manufacturer’s system application package for example) package, you can see the following structure, does not contain any dex file

Look at the directory structure of this application on the phone again. The vdex and odex files contain all the code of the apk, and normally also contain the classes.dex file. Since vdex and odex are machine codes, there is no way to directly convert them into two-level codes that can be viewed (it may be that the tools I use are wrong).

2.4 The relationship between vdex, odex and classes.dex

It may be a bug in the system compilation, or it may be the secondary processing of the odex and vdex files after the ART file is generated. The phenomenon is like this. Try to obtain the dex file in the odex, and it prompts that there is no dex file.

In order to reconfirm whether the dex file is actually contained in odex, use 010Editor to confirm again, you can see that there is still no dex file under recent Files.

Try to get the dex file in vdex, but it is also unavailable.

Therefore, the statement that odex (or vdex) contains classes.dex is incorrect.

Three, what is Arouter

A routing component of Ali has many functions. My actual use scenario here is to perform page jumps. For specific functions, please refer to the introduction of 160bd92c33faef at the Ali Summit.

Drawing on the point mentioned in the summit as a foreshadowing, it is also a point we will talk about below. "The last thing I want to share is ARouter's future development plan. In the future, ARouter will support plug-inization and support the generation of mapping relationship documents, because plug-inization is a technical solution that will be used in many large-scale apps now, and many Dex and functions are dynamically posted. In the APP, and in this case, it is impossible to find all the Dex files, that is, for the Dex that has not been loaded, the mapping relationship inside cannot be jumped, so once the Dex file location changes, The conventional solution cannot find Dex, nor can it realize the initialization of the mapping file. This part will be supported in a later version."

The aouter path that Ali can recognize is as follows:

In other words, the aouter may not be found due to changes in the location or path of the dex file.

Four, step on the pit

4.1 Phenomenon

It is mentioned in 2.4 that the odex file does not contain dex, and the aouter search path follows the rules of grouping on-demand loading. In the final analysis, it is actually a search for class files, as shown in the following figure:

The information of the class file is recorded in the dex file, so an exception occurs. When using aouter to jump to the page, a classNotFound exception appears.

4.2 Solution

If you want to find a solution, you must know how to make odex not affect the aouter path. In this regard, it may be difficult to find a solution without relevant experience, and you can only find it a little bit. By searching the working principle of ART, I found the article "Configuring ART" , in which the article mentioned:

That is to say, by configuring the attributes of LOCAL\_DEX\_PREOPT, you can prevent odex optimization, so find Android.mk and set LOCAL\_DEX\_PREOPT := nostripping.

Both do dex optimization (generating odex files) when compiling, and do not strip dex from the apk. So there is the path comparison after the apk is generated below, and then look at the path where the dex is not stripped, and the classes.dex file is included below.

Use jadx to open the classes.dex file and find that the path file of the aouter is here, so the jumper of the aouter is normal, and the exception no longer appears.

Five, summary

odex optimizes this kind of system to do things, often there will be some unexpected results, if you are responsible for the manufacturer’s application, you often need built-in projects, you should pay attention at this time, when your application contains third-party frameworks, you should pay attention The path and resource references are all right. Although under normal circumstances, the odex file will not interfere with your path, but it is inevitable that odex will make mistakes, because for odex, the resources inside do not need to be saved, and the art file can be generated. Just run it. Reasonable use of art's configuration can help solve many problems.

Author: vivo internet client team-Xu jie