What if the version is not compatible with the Jar package conflict?

I. Introduction

"Wife" and "Mom" fell into the water at the same time, who should be saved first?

As the saying goes: coding for five minutes, conflict resolution for two hours. As for Java development, when you first see the exceptions of ClassNotFoundException and NoSuchMethodException, the first reaction is to arrange packages. After a regular and unconventional operation, it is often found that the same Jar package introduces multiple different versions. At this time, it is generally sufficient to exclude the lower version and keep the higher version. This is because the general Jar package is backward compatible. of. However, if there is a version incompatibility, you will fall into the dilemma of "wife and mother fall into the water at the same time, who will save first". If exactly this incompatibility occurs between the middleware dependency and the business's own dependency , It's even more difficult.

As shown in the figure below, Project represents our project, Dependency A represents our business dependency, and Dependency B represents middleware dependency. If both business dependency and middleware dependency depend on the same Jar package C, but the versions are different, each is 0.1 Version and version 0.2, and the most unfortunate thing is that these two versions still conflict. Some old functions only exist in the lower version of 0.1, and some new functions only exist in the higher version of 0.2. It is really "wife and mom fell into it at the same time." In the water, you can't save anyone first."

(Picture taken from: SOFAArk official website)

As the saying goes: development that has not encountered Jar package conflict must be a fake Java development; development that has not resolved the Jar package conflict is not a qualified Java development. In a recent project, we need to use Guava's higher version of the Jar package, but found that the middleware relies on a lower version and incompatible with the higher version of the Jar package. Facing this dilemma, we must be "wife" and "mother" "Everything must be saved, so we began to seek a solution.

Two, incompatibility dependent conflict resolution solution

"wife" and "mother" must be saved, how to save?

First of all, we thought of whether we can copy the high version of Guava code that needs to be used directly into our project, but doing so will bring several problems:

• Guava is a feature-rich basic library. A certain part of the code often has a dependency relationship with many other codes, which will cause the whole body to be affected, and the workload will be much larger than expected;
• The copied code can only be maintained manually. If the official fixes the problem or refactors the code or adds functions, if we want to upgrade, we can only do it again. Therefore, we can only think of other plans, and this can only be used as a last resort plan.

Then, we are thinking that a Java class is loaded into the JVM virtual machine to be different from another Class. One is that they have different paths. They are two different classes that are unrelated to each other, but they are Loaded by different class loaders, they are still considered to be two different classes in the JVM virtual machine. Therefore, we are thinking of seeking a solution from the class loader. Inside Alibaba, there is a Pandora component, just like its name is like a magic box, it will install all the dependencies of the middleware into Pandora (called the Sar package internally), in this way, it can avoid the middleware and business The code directly presents the dilemma of "wife and mother fall into the water at the same time, who will save first".

Similarly, in similar scenarios such as application merge deployment can also exert its power. But Pandora is only used internally by Ali and is not open source. In Ant Financial, there is also such a component, and it is open sourced, called SOFAArk (official website, if you are interested, you can go to the official website to understand the principle and use of SOFAArk), we feel that we have found the Mr. Right, so we started to study SOFAArk how to use. Like Pandora, SOFAArk also uses different ClassLoader to load different versions of the third-party dependencies to isolate classes and completely solve the problem of package conflicts. This requires us to package related dependencies into Ark Plugin (see SOFAArk official documentation).

For the company, the benefits of such a solution are relatively large. After being packaged into Ark Plugin, the entire company can share it, and the business side can benefit. However, for one of our projects, adopting such a solution is undoubtedly too heavy. Therefore, we contacted middleware classmates and asked if there are plans to introduce similar isolation components to solve the dependency conflict between middleware and business code. The answer is that the company’s current package conflict is not a strong pain point, and there is no plan to introduce it for the time being. . Therefore, we can only put SOFAArk on hold for the time being and continue to look for new solutions.

Next, we were thinking that since Pandora/SOFAArk uses class loading to isolate the classes of the same path, if we change the groupId of the two conflicting repositories to be different, then even the full path of the class with the same name will be different, so There must be different classes in the JVM. If the Pandora/SOFAArk isolation method is called logical isolation, this is equivalent to physical isolation. To achieve this, it is easier to achieve this with the help of the IDE's refactoring function or the global replacement function.

When we were about to roll up our sleeves and start doing it, we couldn't help thinking that this kind of pain point should have been encountered long ago, especially for basic class libraries like Guava and Commons. Conflicts are inevitable. The predecessors should have found elegance. Tickling posture. So, we went to search for related articles. As expected, maven-shade-plugin is the elegant tickling posture. The principle of this Maven plugin is to remap the package path of the class to achieve the purpose of isolating incompatible Jar packages. .

Three, maven-shade-plugin resolve dependency conflicts

Finally, how to configure and use the maven-shade-plugin to map Guava into our own customized Jar package to achieve isolation from the middleware Guava. The whole process is relatively clear and clear. It is mainly to create a Maven project, introduce dependencies, configure the warehouse address we want to publish, introduce the compiled and packaged plug-in and maven-shade-plugin plug-in, configure the mapping rules (the part between the tags), and then Compile, package, and publish to the Maven repository. The configuration of pom.xml is as follows:

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.shaded.example</groupId>
    <artifactId>guava-wrapper</artifactId>
    <version>${guava.wrapper.version}</version>

    <name>guava-wrapper</name>
    <url>https://example.com/guava-wrapper</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
            <!- 版本与 guava 版本基本保持一致 ->
        <guava.wrapper.version>27.1-jre</guava.wrapper.version>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>27.1-jre</version>
        </dependency>  
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.3</version>
                <configuration>
                    <source>${maven.compiler.source}</source>
                    <target>${maven.compiler.target}</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <version>2.3.2</version>
                <executions>
                    <execution>
                        <id>default-jar</id>
                        <goals>
                            <goal>jar</goal>
                        </goals>
                        <phase>package</phase>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-source-plugin</artifactId>
                <version>2.4</version>
                <executions>
                    <execution>
                        <id>default-sources</id>
                        <goals>
                            <goal>jar-no-fork</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>2.4.1</version>
                <configuration>
                    <createDependencyReducedPom>false</createDependencyReducedPom>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                                          <!-- 重命名规则配置 -->
                            <relocations>
                                <relocation>
                                                      <!-- 源包路径 -->
                                    <pattern>com.google.guava</pattern>
                                                      <!-- 目标包路径 -->
                                    <shadedPattern>com.google.guava.wrapper</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>com.google.common</pattern>
                                    <shadedPattern>com.google.common.wrapper</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>com.google.thirdparty</pattern>
                                    <shadedPattern>com.google.wrapper.thirdparty</shadedPattern>
                                </relocation>
                            </relocations>
                            <transformers>
                                <transformer
                                        implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"/>
                            </transformers>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <distributionManagement>
          <!- Maven仓库配置，略 ->
    </distributionManagement>
</project>

After the project introduces this newly packaged guava-wrapper, import can choose to import the relevant classes we need from this package. as follows:

<dependency>
  <groupId>com.vivo.internet</groupId>
  <artifactId>guava-wrapper</artifactId>
  <version>27.1-jre</version>
</dependency>

Fourth, the conclusion

In order to use multiple versions of incompatible Jar packages in the same project, we first thought of maintaining the code manually, but the workload and maintenance cost were high, and then we thought of isolating through class loader (open source solution SOFAArk), but we needed to Related dependencies are packaged into Ark Plugin, the solution is undoubtedly a bit too heavy, and finally renamed and packaged through the maven-shade-plugin plug-in, elegantly solved the conflict problem of incompatible multiple versions of Jar packages in the project. From the problem, we explore the solution to the problem step by step. Although the final maven-shade-plugin plug-in solution seems to be essentially the same as the manual maintenance code, it seems to be back to the original point, but in fact the final solution is far more elegant than the beginning It is much higher, just like the road of life, spiraling upwards and advancing in a curve.

If you encounter similar scenarios where you need to support the coexistence of incompatible Jar packages, you can consider using the maven-shade-plugin plug-in. This method is relatively lightweight and can be used in scenarios where there are individual incompatible Jar package conflicts in the project. It is simple and effective. The cost is also very low. However, if the Jar package conflict phenomenon is relatively common and has become an obvious or common pain point, it is still recommended to consider the class loader isolation schemes such as Pandora and SOFAArk mentioned in the article.

Author: vivo internet server team-Zhang Wei