Practice of JVM Custom Class Loader in Code Extensibility

1. Background

The list management system is a system in which each module on the mobile phone configures the applications that need to be controlled into a file, and then sends it to the mobile phone for application control, such as the power consumption control of each application; the control application file of each module takes into account security issues, There are different encryption methods of their own. According to past experience, we can use template method + factory mode to obtain different encryption methods according to the type of module. The code class hierarchy is shown as follows:

Get the class structure diagram of different encryption methods

Using the factory mode and template method mode, when there is a new encryption method, we can add a new handler to meet the principle of "closed for modification, open to extension", but this method inevitably requires modification of code and re-engineering. Release version and go online. So is there a better way to solve this problem? Here is the topic we want to focus on today.

Second, the timing of class loading

A type starts from being loaded into the memory of the virtual machine, and until it is unloaded from the memory, its entire life cycle will go through Loading, Verification, Preparation, Resolution, and Initialization. , Using (Using) and unloading (Unloading) seven stages, of which the three parts of verification, preparation and analysis are collectively called Linking. The sequence of occurrence of these seven stages is shown in Figure 1.

Although the loading process of the classloader has 7 complicated steps, in fact, except for the four steps of loading, the rest are controlled by the JVM virtual machine. We have not much room for intervention except to adapt to its specifications for development. Loading is the most important means for us to control the classloader to achieve a special purpose. It is also the focus of our next introduction.

3. Loading

The "Loading" stage is a stage in the entire "Class Loading" process. During the loading phase, the Java virtual machine needs to do three things:

• Get the binary byte stream defining this class by its fully qualified name.
• Convert the static storage structure represented by this byte stream into the runtime data structure of the method area.
• A java.lang.Class object representing this class is generated in memory as an access entry for various data of this class in the method area.

The "Java Virtual Machine Specification" does not have any specific requirements for these three points, so that the flexibility of virtual machine implementation and Java application is quite large. For example, the rule "get the binary byte stream that defines this class through the fully qualified name of a class" does not specify that the binary byte stream must be obtained from a class file, rather it does not specify that the binary byte stream must be obtained from a certain class file Where to get it and how to get it. For example, we can read from ZIP archives, obtain from the network, generate at runtime, generate from other files, and read from databases. Also available from encrypted files.

From here, we can see that as long as we can obtain the .class file of the encrypted class, we can obtain the corresponding encrypted class object through the class loader, and then call the specific encryption method through reflection. Therefore, the class loader plays a crucial role in the loading process of the .class file.

Fourth, the parental delegation model

Currently, there are three class loaders in the Java virtual machine, namely, the startup class loader, the extension class loader and the application class loader; most Java programs use these three class loaders for loading.

4.1 Start the class loader

This class is implemented by C++ and is responsible for loading and storing in the \lib directory, or in the path specified by the -Xbootclasspath parameter, and is recognized by the Java virtual machine (identified by file name, such as rt.jar, tools.jar, A class library whose name does not match will not be loaded even if it is placed in the lib directory.) The class library is loaded into the memory of the virtual machine. The startup class loader cannot be directly referenced by the Java program. When the user writes a custom class loader, if he needs to delegate the loading request to the boot class loader for processing, he can directly use null instead.

4.2 Extending the class loader

This class loader is implemented as Java code in the class sun.misc.Launcher$ExtClassLoader. It is responsible for loading all class libraries in the \lib\ext directory, or in the path specified by the java.ext.dirs system variable. According to the name of "Extension Class Loader", it can be inferred that this is an extension mechanism of Java system class library. The JDK development team allows users to place generic class libraries in the ext directory to extend the functions of Java SE. , After JDK9, this extension mechanism is replaced by the natural extension ability brought by modularization. Since the extension class loader is implemented by Java code, developers can directly use the extension class loader to load the Class file in the program.

4.3 Application class loader

This class loader is implemented by sun.misc.Launcher$AppClassLoader. Since the application class loader is the return value of the getSystemClassLoader() method in the ClassLoader class, it is also called "system class loader" in some occasions. It is responsible for loading all the class libraries on the user's class path (ClassPath), and developers can also use this class loader directly in the code. If the application has not customized its own class loader, in general, this is the default class loader in the program.

Since the existing class loader loading paths have special requirements, the path stored in the .class file generated by the encrypted class compiled by ourselves is not in the paths of the three existing class loaders, so it is necessary for us to define our own class Loader.

5. Custom class loader

Except for the root class loader, all class loaders are subclasses of ClassLoader. So we can implement our own class loader by inheriting ClassLoader.

The ClassLoader class has two key methods:

• protected Class loadClass(String name, boolean resolve): name is the class name, if resolve is true, the class will be resolved during loading.
• protected Class findClass(String name) : Find a class based on the specified class name.

So, if you want to implement a custom class, you can override these two methods to implement it. However, it is recommended to override the findClass method instead of overriding the loadClass method. Overriding the loadClass method may break the parent delegation model of class loading, because the findClass method will be called inside the loadClass method.

protected Class<?> loadClass(String name, boolean resolve)
        throws ClassNotFoundException
    {
        synchronized (getClassLoadingLock(name)) {
            // First, check if the class has already been loaded
            Class<?> c = findLoadedClass(name);
            if (c == null) {
                long t0 = System.nanoTime();
                try {
                    if (parent != null) {
                        c = parent.loadClass(name, false);
                    } else {
                        c = findBootstrapClassOrNull(name);
                    }
                } catch (ClassNotFoundException e) {
                    // ClassNotFoundException thrown if class not found
                    // from the non-null parent class loader
                }
 
                if (c == null) {
                    // If still not found, then invoke findClass in order
                    // to find the class.
                    long t1 = System.nanoTime();
                    c = findClass(name);
 
                    // this is the defining class loader; record the stats
                    sun.misc.PerfCounter.getParentDelegationTime().addTime(t1 - t0);
                    sun.misc.PerfCounter.getFindClassTime().addElapsedTimeFrom(t1);
                    sun.misc.PerfCounter.getFindClasses().increment();
                }
            }
            if (resolve) {
                resolveClass(c);
            }
            return c;
        }
    }

loadClass loading method flow:

• Determine whether this class has been loaded;
• If the parent loader is not null, use the parent loader to load; otherwise, use the root loader to load;
• If none of the previous loads are successful, use the findClass method to load.

Therefore, in order not to affect the class loading process, we can simply and conveniently implement custom class loading by rewriting the findClass method.

6. Code implementation

6.1 Implementing a custom class loader

public class DynamicClassLoader extends ClassLoader {
 
    private static final String CLASS_EXTENSION = "class";
 
    @Override
    public Class<?> findClass(String encryptClassInfo) {
        EncryptClassInfo info = JSON.parseObject(encryptClassInfo, EncryptClassInfo.class);
        String filePath = info.getAbsoluteFilePath();
        String systemPath = System.getProperty("java.io.tmpdir");
        String normalizeFileName = FilenameUtils.normalize(filePath, true);
        if (StringUtils.isEmpty(normalizeFileName) || !normalizeFileName.startsWith(systemPath)
                ||getApkFileExtension(normalizeFileName) == null
                || !CLASS_EXTENSION.equals(getApkFileExtension(normalizeFileName))) {
            return null;
        }
 
        String className = info.getEncryptClassName();
        byte[] classBytes = null;
        File customEncryptFile = new File(filePath);
        try {
            Path path = Paths.get(customEncryptFile.toURI());
            classBytes = Files.readAllBytes(path);
        } catch (IOException e) {
            log.info("加密错误", e);
        }
        if (classBytes != null) {
            return defineClass(className, classBytes, 0, classBytes.length);
        }
        return null;
    }
 
    private static String getApkFileExtension(String fileName) {
        int index = fileName.lastIndexOf(".");
        if (index != -1) {
            return fileName.substring(index + 1);
        }
        return null;
    }
}

Here is mainly through integrating ClassLoader, overwriting the findClass method, obtaining the corresponding .class file information from the encrypted class information, and finally obtaining the object of the encrypted class

6.2 encrypt() method in .class file

public String encrypt(String rawString) {
        String keyString = "R.string.0x7f050001";
        byte[] enByte = encryptField(keyString, rawString.getBytes());
        return Base64.encode(enByte);
    }

6.3 Specific calls

public class EncryptStringHandler {
 
    private static final Map<String, Class<?>> classMameMap = new HashMap<>();
 
    @Autowired
    private VivofsFileHelper vivofsFileHelper;
 
    @Autowired
    private DynamicClassLoader dynamicClassLoader;
 
    public String encryptString(String fileId, String encryptClassName, String fileContent) {
        try {
            Class<?> clazz = obtainEncryptClass(fileId, encryptClassName);
            Object obj = clazz.newInstance();
            Method method = clazz.getMethod("encrypt", String.class);
            String encryptStr = (String) method.invoke(obj, fileContent);
            log.info("原字符串为:{},加密后的字符串为:{}", fileContent, encryptStr);
            return encryptStr;
        } catch (Exception e) {
            log.error("自定义加载器加载加密类异常", e);
            return null;
        }
    }
 
    private Class<?> obtainEncryptClass(String fileId, String encryptClassName) {
        Class<?> clazz = classMameMap.get(encryptClassName);
        if (clazz != null) {
            return clazz;
        }
 
        String absoluteFilePath = null;
        try {
            String domain = VivoConfigManager.getString("vivofs.host");
            String fullPath = domain + "/" + fileId;
            File classFile = vivofsFileHelper.downloadFileByUrl(fullPath);
            absoluteFilePath = classFile.getAbsolutePath();
            EncryptClassInfo encryptClassInfo = new EncryptClassInfo(encryptClassName, absoluteFilePath);
            String info = JSON.toJSONString(encryptClassInfo);
            clazz = dynamicClassLoader.findClass(info);
            //设置缓存
            Assert.notNull(clazz, "自定义类加载器加载加密类异常");
            classMameMap.put(encryptClassName, clazz);
            return clazz;
        } finally {
            if (absoluteFilePath != null) {
                FileUtils.deleteQuietly(new File(absoluteFilePath));
            }
        }
    }
}

Through the implementation of the above code, we can add the compiled .class file to the management platform, and finally call the method through the custom class loader and reflection method, so as to avoid the need for us to modify the code and reissue the version To adapt to the problem of constantly adding encryption methods.

7. Problems

When the above code is tested locally, there is no exception, but after it is deployed to the test server, a JSON parsing exception occurs, which seems to be the wrong format of the json string.

The json parsing logic mainly exists in the conversion of strings to object logic at the entrance of the DynamicClassLoader#findClass method. Why is an error reported here? We printed the input parameters at the entrance.

It is found that in addition to the correct input parameters we need (the first input parameter information is printed), there is also a full path name of Base64 cn.hutool.core.codec.Base64. This situation shows that because we have rewritten the findClass method of ClassLoader, and when Base64 is loaded, the loadClass method of the original ClassLoader class will be called to load, and the findClass method is called inside. Since the findClass has been rewritten, it will Report the above json parsing error.

protected Class<?> loadClass(String name, boolean resolve)
        throws ClassNotFoundException
    {
        synchronized (getClassLoadingLock(name)) {
            // First, check if the class has already been loaded
            Class<?> c = findLoadedClass(name);
            if (c == null) {
                long t0 = System.nanoTime();
                try {
                    if (parent != null) {
                        c = parent.loadClass(name, false);
                    } else {
                        c = findBootstrapClassOrNull(name);
                    }
                } catch (ClassNotFoundException e) {
                    // ClassNotFoundException thrown if class not found
                    // from the non-null parent class loader
                }
 
                if (c == null) {
                    // If still not found, then invoke findClass in order
                    // to find the class.
                    long t1 = System.nanoTime();
                    c = findClass(name);
 
                    // this is the defining class loader; record the stats
                    sun.misc.PerfCounter.getParentDelegationTime().addTime(t1 - t0);
                    sun.misc.PerfCounter.getFindClassTime().addElapsedTimeFrom(t1);
                    sun.misc.PerfCounter.getFindClasses().increment();
                }
            }
            if (resolve) {
                resolveClass(c);
            }
            return c;
        }
    }

But what is expected here is that in addition to the .class file used for encryption with a custom class loader, we do not want other classes to be loaded with a custom class loader. By analyzing the ClassLoader#loadClass method, we hope to be able to pass Its parent class loader is loaded into the Base64 tripartite class. Because the Bootstrap Class Loader can't be loaded into Base64, we need to display the parent class loader, but which class loader is the parent class loader set to, then we need to understand the Tomcat class loader structure .

Why Tomcat needs to build a class loading structure on the basis of JVM is mainly to solve the following problems:

• Java class libraries used by two web applications deployed on the same server can be isolated from each other;
• Java class libraries used by two web applications deployed on the same server can be shared;
• The server needs to ensure its own security as much as possible, and the class library used by the server should be independent of the class library of the application;
• A web server that supports JSP applications, large logarithms need to support the HotSwap function.

To this end, tomcat extends the Common class loader (CommonClassLoader), Catalina class loader (CatalinaClassLoader), Shared class loader (SharedClassLoader) and WebApp class loader (WebAppClassLoader), they respectively load /commons/_, /server/ _, /shared/ , and /WebApp/WEB-INF/ for the logic of the Java class library.

Through analysis, we know that the WebAppClassLoader class loader can load the dependent packages in the /WEB-INF/ directory, and the package we depend on, cn.hutool.core.codec.Base64, is located in the package hutool-all-4.6.10- sources.jar exists under the /WEB-INF/ directory, and the package vivo-namelist-platform-service-1.0.6.jar where our custom class loader is located is also under /WEB-INF/*, so since The definition class loader DynamicClassLoader is also loaded by WebAppClassLoader.

We can write a test class to test it:

@Slf4j
@Component
public class Test implements ApplicationListener<ContextRefreshedEvent> {
 
    @Override
    public void onApplicationEvent(ContextRefreshedEvent event) {
        log.info("classLoader DynamicClassLoader:" + DynamicClassLoader.class.getClassLoader().toString());
    }
}

Test Results:

So we can set the parent loader of the custom class loader DynamicClassLoader to load its own class loader:

public DynamicClassLoader() {
        super(DynamicClassLoader.class.getClassLoader());
}

When we perform the file encryption and decryption operation again, no error is found, and by adding the log, we can see that the class loader corresponding to the loaded class cn.hutool.core.codec.Base64 is indeed the class loader corresponding to the DynamicClassLoader WebAppClassLoader .

public String encrypt(String rawString) {
        log.info("classLoader Base64:{}", Base64.class.getClassLoader().toString());
        String keyString = "R.string.0x7f050001";
        byte[] enByte = encryptField(keyString, rawString.getBytes());
        return Base64.encode(enByte);
    }

Now think about it again, why it can be loaded into cn.hutool.core.codec.Base64 without setting the parent class loader of the custom class loader in the IDEA runtime environment.

Add the following print information in the IDEA runtime environment:

public String encrypt(String rawString) {
        System.out.println("类加载器详情...");
        System.out.println("classLoader EncryptStrategyHandler:" + EncryptStrategyHandlerH.class.getClassLoader().toString());
        System.out.println("classLoader EncryptStrategyHandler:" + EncryptStrategyHandlerH.class.getClassLoader().getParent().toString());
        String classPath = System.getProperty("java.class.path");
        System.out.println("classPath:" + classPath);
        System.out.println("classLoader Base64:" + Base64.class.getClassLoader().toString());
        String keyString = "R.string.0x7f050001";
        byte[] enByte = encryptField(keyString, rawString.getBytes());
        return Base64.encode(enByte);
    }

It is found that the class loader that loads the .class file is the custom class loader DynamicClassLoader, and the parent class loader of the .class loader is the application class loader AppClassLoader, and the class loader that loads cn.hutool.core.codec.Base64 is also AppClassLoader .

The specific loading process is as follows:

1) The custom class loader is first delegated to AppClassLoader;

2) AppClassLoader delegates to parent class loader ExtClassLoader;

3) ExtClassLoader is then delegated to BootStrapClassLoader, but BootClassLoader cannot be loaded, so ExtClassLoader loads itself and cannot be loaded;

4) It is then loaded by AppClassLoader;

AppClassLoader will call the findClass method of its parent class UrlClassLoader to load;

5) Finally loaded into cn.hutool.core.codec.Base64 from the user class path java.class.path.

From this, we found that in the IDEA environment, the three-party cn.hutool.core.codec.Base64 that the custom encryption class .class file depends on can be loaded through AppClassLoader.

In the Linux environment, after remote debugging, it was found that the class loader that initially loaded cn.hutool.core.codec.Base64 was DynamicClassLoader. Then it is delegated to the parent class loader AppClassLoader for loading. According to the principle of parental delegation, it will be handed over to AppClassLoader for processing. However, the class cn.hutool.core.codec.Base64 was still not found in the user path, and was finally loaded by DynamicClassLoader, which resulted in the initial JSON parsing error.

8. Summary

Since the class loading stage does not strictly limit how to obtain the binary byte stream of a class, it provides us with a possibility to dynamically load .class files through a custom class loader to achieve code extensibility. By flexibly customizing the classloader, it can also play an important role in other fields, such as implementing code encryption to avoid core code leakage, resolving conflicts caused by different services relying on different versions of the same package, and implementing hot deployment of programs to avoid frequent debugging. Restart the application.

9. References

1. "In-depth understanding of the Java virtual machine: JVM advanced features and best practices (3rd edition)"

2. the strongest in history -- the principle and application of Java class loader

Author: vivo internet server team - Wang Fei