Java High Concurrency Study Notes (3): Class Loading

1 source

• Source: "Detailed Multithreading and Architecture Design of Java High Concurrency Programming", written by Wang Wenjun
• Chapters: Chapters 9, 10, 11

Notes for the three chapters of this paper.

2 Introduction to class loading

The class loading process can be simply divided into three stages:

• Loading phase: mainly responsible for finding and loading the binary data file of the class
• Connection stage: It can be subdivided into three stages: verification, preparation, and parsing. Verification is to ensure the correctness of the class file. Preparation is to allocate memory for static variables of the class and initialize default values for them. Parsing is to refer to the symbols in the class. Convert to direct reference
• Initialization phase: assign correct initial values to static variables of the class

3 Active use and passive use

JVM specification stipulates that each class or interface needs to be initialized when it is actively used for the first time, and specifies the following six scenarios for actively using classes:

• Passing the new keyword will cause the initialization of the class
• Accessing static variables of a class
• Accessing static methods of a class
• Reflect on a class
• Initializing a subclass causes the parent class to initialize
• The startup class (that is, the class containing main() ) is also initialized

Except for the above six cases, the rest are called passive use, which will not lead to the loading and initialization of the class. For example, the static constant of the reference class will not lead to the initialization of the class.

4 Detailed explanation of class loading

As mentioned earlier, class loading can be simply divided into three stages:

• loading phase
• connection phase
• initialization phase

Let's take a look at the loading phase first.

4.1 Loading Phase

The loading phase is to read the binary data in the class file into the memory, then convert the static storage structure represented by the byte stream into the runtime data structure in the method area, and generate a java.lang.Class object of this class in the heap, As the entry to access the data structure of the method area.

The final product of class loading is the class object in heap memory. The JVM specification states that class loading is to obtain binary data streams through a fully qualified name. The sources include:

• class file: This is the most common format, which is to load the bytecode file compiled by javac
• Dynamic generation at runtime: For example, ASM can be dynamically generated, or it can be generated through dynamic proxy java.lang.Proxy , etc.
• Obtained through the network: such as RMI
• Read compressed files: such as JAR , WAR packages
• Read from database: For example, read data of MySQL field type in BLOB
• Generate class files at runtime and load them dynamically: such as serialization frameworks such as Thrift and Avro , generate several class files from a certain schema and load them

After the class loading phase is over, JVM will store these binary byte streams in the method area according to the format defined by JVM , form a specific data structure, and then instantiate a java.lang.Class object in the heap memory.

4.2 Connection Phase

This stage can be divided into three sub-stages:

• verify
• Prepare
• Parse

It should be noted that these three small stages are not actually carried out sequentially, but are carried out crosswise, that is, there will actually be a verification process during parsing.

4.2.1 Verification

The verification is to ensure that the content contained in the byte stream conforms to the JVM specification, and there is no code that compromises the security of JVM itself. When the byte stream information does not meet the requirements, an exception such as VerifyError or its sub-exception will be thrown. Verify The information includes:

• file format
• metadata
• bytecode
• Symbolic reference

4.2.1.1 Validation file format

include:

• Magic number ( 0xCAFEBABE )
• Major and minor version numbers
• Is there any missing or additional information
• Whether the constant pool constant type supports
• Whether the constant pool reference points to a nonexistent constant or an unsupported type constant
• other

4.2.1.2 Validating metadata

Metadata verification is actually a process of semantic analysis. Semantic analysis is to ensure that the byte stream meets the requirements of the JVM specification, including:

• Check whether a class has a parent class, whether it inherits an interface, whether these parent classes or interfaces are legal, or whether they exist
• Check if the class of final is inherited
• Check the abstract class, check whether the abstract method or interface method of the parent class is implemented
• Check for overloading, like same method name, same parameters but different return type, this is not allowed

4.2.1.3 Verify bytecode

Bytecode verification is mainly the control flow of the verification program, including:

• Ensure that the current thread's instructions in the program counter will not jump to illegal bytecode instructions
• Guaranteed type conversions are legal
• Ensure that the operation stack type and instruction code in the virtual machine stack can be executed correctly at any time
• Other verification

4.2.1.4 Verifying Symbol References

Verify the legality of converting symbolic references to direct references, and ensure the smooth execution of parsing actions, including:

• Whether the fully qualified name of the string described by the symbolic reference can successfully find the related class
• Whether classes, fields, and methods in symbolic references are visible to the current class
• other

4.2.2 Preparation

After verification, the preparation phase begins. This phase is relatively simple, that is, to allocate memory to the static variables of the object and set the initial value, and the memory of the class variable will be allocated to the method area. Setting the initial value is to give a default value of the relevant type when it is not set for the corresponding class variable. For example, the initial value of Int is 0, and the initial value of the reference is null .

4.2.3 Analysis

Resolution is the process of finding symbolic references to classes, fields, interfaces, and methods in the constant pool, and replacing these symbolic references with direct references. Parsing is mainly for class interfaces, fields, class methods and interface methods, including:

• Class interface analysis
• field parsing
• class method parsing
• Interface method analysis

4.3 Initialization Phase

The initialization phase is mainly the process of executing the <clinit> method. This method is generated in the compilation phase, that is to say, it is included in the bytecode file. This method includes the assignment actions of all class variables and the execution code of the static statement block. On the other hand, unlike the constructor, <clinit> does not need to explicitly call the parent class constructor, and the virtual machine ensures that the <clinit> method of the parent class is executed first.

It should also be noted that <clinit> can only be executed by the virtual machine, and the virtual machine will also ensure the security under multi-threading. Therefore, if the static code block contains the operation of loading other classes, it may cause deadlock. For example, see here .

5 class loader

5.1 Three types of core class loaders in JVM

There are three types of core class loaders in JVM , namely:

• Startup class loader: The startup class loader is the top-level class loader without a parent loader. It is written by C++ and is responsible for loading the JVM core class library, such as loading the classes in the entire java.lang package
• Extension class loader: The parent loader of the extension class loader is the startup class loader, which mainly loads the class library in the subdirectory of jre/lib/ext , which is implemented by pure Java and is a subclass of URLClassLoader
• Application class loader: also called system class loader, responsible for loading the class library under classpath , the parent loader of the application class loader is the extension class loader, and it is also the default parent loader of the custom class loader

5.2 Parental delegation mechanism

When a class loader loads a class, it does not try to load the class directly, but first passes it to the parent loader to try to load, until the top-level parent loader (startup class loader), if the parent loader fails to load , it will try to load by itself, as shown below:

6 Thread context class loader

JDK provides many SPI ( Service Provider Interface ), such as JDBC , etc. JDBC only specifies the logical relationship between these interfaces, but does not provide a specific implementation JDBC For specific implementation, the application program only needs to be interface-oriented programming. But the problem is:

• All interfaces in java.lang.sql are provided by JDK , and the class loader that loads these interfaces is the startup class loader
• Class library drivers from third-party vendors are loaded by the system class loader

Due to the parent delegation mechanism, Connections , Statement , etc. are all loaded by the startup class loader, while the implementation in the third-party JDBC driver package will not be loaded. The key to solving this problem is to use the thread context class loader to break the parent delegation mechanism.

For example, the loading process driven by MySQL is loaded through the thread context class loader.

private static Connection getConnection(String url, Properties info, Class<?> caller) throws SQLException {
        //...
        if (callerCL == null || callerCL == ClassLoader.getPlatformClassLoader()) {
            callerCL = Thread.currentThread().getContextClassLoader();
        }
        while(true) {
            //...
            if (isDriverAllowed(aDriver.driver, callerCL)) {
            }
        }
        //...
}
private static boolean isDriverAllowed(Driver driver, ClassLoader classLoader) {
    //...
    try {
        aClass = Class.forName(driver.getClass().getName(), true, classLoader);
    } catch (Exception var5) {
        result = false;
    }
    //...
    return result;
}

Through the thread context class loader, it becomes the way to start the class loader to delegate the sub-class loader to load the implementation, that is, JDK himself broke the parent delegation mechanism, and this loading method involves almost all SPI Load, including JAXB , JCE , JBI , etc.