Analysis of the application of SPI in Spring

vivo Internet Server Team - Ma Jian

I. Overview

SPI (Service Provider Interface) is a service provider discovery mechanism built into Java, which can be used to improve the extensibility of the framework. It is mainly used in the development of frameworks, such as Dubbo. Different frameworks have slightly different implementations, but the core mechanism is the same. , and Java's SPI mechanism can find service implementations for interfaces. The SPI mechanism transfers the specific implementation of the service to the outside of the program, which provides great convenience for the extension and decoupling of the framework.

Thanks to the excellent capabilities of SPI, it provides a good support for the dynamic expansion of module functions.

This article will briefly introduce Java's built-in SPI and the SPI application in Dubbo, focusing on analyzing the SPI mechanism in Spring, comparing Spring SPI and Java's built-in SPI, and the similarities and differences with Dubbo SPI.

2. Java SPI

Java's built-in SPI uses the java.util.ServiceLoader class to parse the classPath and the file named with the fully qualified name of the interface in the META-INF/services/ directory of the jar package, and load the interface implementation class specified in the file to complete the call .

2.1 Java SPI

First through the code to understand the implementation of the Java SPI

① Create a service provider interface

 package jdk.spi;
// 接口
public interface DataBaseSPI {
    public void dataBaseOperation();
}

② Create an implementation class of the service provider interface

• MysqlDataBaseSPIImpl

Implementation class 1

 package jdk.spi.impl;
 
import jdk.spi.DataBaseSPI;
 
public class MysqlDataBaseSPIImpl implements DataBaseSPI {
 
    @Override
    public void dataBaseOperation() {
        System.out.println("Operate Mysql database!!!");
    }
}

• OracleDataBaseSPIImpl

Implementation class 2

 package jdk.spi.impl;
 
import jdk.spi.DataBaseSPI;
 
public class OracleDataBaseSPIImpl implements DataBaseSPI {
 
    @Override
    public void dataBaseOperation() {
        System.out.println("Operate Oracle database!!!");
    }
}

③ Create the jdk.spi.DataBaseSPI file in the project META-INF/services/ directory

jdk.spi.DataBaseSPI

 jdk.spi.impl.MysqlDataBaseSPIImpl
jdk.spi.impl.OracleDataBaseSPIImpl

④ Run the code:

JdkSpiTest#main()

 package jdk.spi;
 
import java.util.ServiceLoader;
 
public class JdkSpiTest {
 
    public static void main(String args[]){
        // 加载jdk.spi.DataBaseSPI文件中DataBaseSPI的实现类(懒加载)
        ServiceLoader<DataBaseSPI> dataBaseSpis = ServiceLoader.load(DataBaseSPI.class);
        // ServiceLoader实现了Iterable，故此处可以使用for循环遍历加载到的实现类
        for(DataBaseSPI spi : dataBaseSpis){
            spi.dataBaseOperation();
        }
    }
}

⑤ Running result:

 Operate Mysql database!!!
Operate Oracle database!!!

2.2 Source code analysis

The above implementation is a simple example of using Java's built-in SPI. ServiceLoader is a built-in tool class in Java for finding service-provided interfaces. The search for service-provided interfaces is realized by calling the load() method (this step is not really in the strict sense). start to look up, only do initialization), and finally traverse to access the implementation classes of the service provider interface one by one.

The above method of accessing service implementation classes is very inconvenient. For example, a service cannot be used directly, and each implementation of the service provider interface needs to be accessed through traversal. At this point, many students will have questions:

• Java built-in access can only be achieved by traversal?
• The service provider interface must be placed in the META-INF/services/ directory? Can it be placed in another directory?

Give the answer before analyzing the source code: both are true; Java's built-in SPI mechanism can only access the implementation class of the service provider interface by traversing, and the configuration file of the service provider interface can only be placed in META-INF/ services/ directory.

ServiceLoader part of the source code

 public final class ServiceLoader<S> implements Iterable<S>{
    // 服务提供接口对应文件放置目录
    private static final String PREFIX = "META-INF/services/";
 
    // The class or interface representing the service being loaded
    private final Class<S> service;
 
    // 类加载器
    private final ClassLoader loader;
 
    // The access control context taken when the ServiceLoader is created
    private final AccessControlContext acc;
 
    // 按照初始化顺序缓存服务提供接口实例
    private LinkedHashMap<String,S> providers = new LinkedHashMap<>();
 
    // 内部类，实现了Iterator接口
    private LazyIterator lookupIterator;
}

It can be found from the source code:

• The ServiceLoader class itself implements the Iterable interface and implements the iterator method. The implementation of the iterator method calls the method in the inner class LazyIterator. After parsing the service-provided interface file, the final result is returned in the Iterator, which does not support services. Provides direct access to interface implementation classes.
• All the corresponding files of the service provider interface are placed in the META-INF/services/ directory, and the final type determines that the PREFIX directory cannot be changed.

So Java's built-in SPI mechanism idea is very good, but its built-in implementation deficiencies are also obvious.

3. Dubbo SPI

Dubbo SPI follows the design idea of Java SPI, but has been greatly improved in implementation. It can not only directly access extension classes, but also greatly improve the flexibility of access and the convenience of extension.

3.1 Basic Concepts

① Extension point

A Java interface, equivalent to the service provider interface, needs to be decorated with @SPI annotation.

② Expand

The implementation class of the extension point.

③ Extension class loader: ExtensionLoader

Similar to Java SPI's ServiceLoader, it is mainly used to load and instantiate extension classes. An extension point corresponds to an extension loader.

④ Dubbo extension file loading path

The Dubbo framework supports loading extension classes from the following three paths:

• META-INF/dubbo/internal
• META-INF/dubbo
• META-INF/services

The Dubbo framework corresponds to three policy classes for extended configuration files in three different paths:

• DubboInternalLoadingStrategy
• DubboLoadingStrategy
• ServicesLoadingStrategy

There is nothing special about the extended configuration files under the three paths, in general:

• META-INF/dubbo is open to developers
• META-INF/dubbo/internal is used to load extension points inside Dubbo
• META-INF/services compatible with Java SPI

⑤ Extended configuration file

Unlike the Java SPI, the extension classes in Dubbo's extension configuration file have a name, which makes it easy to refer to them in the application.

Such as: Dubbo SPI extension configuration file

 #扩展实例名称=扩展点实现类
adaptive=org.apache.dubbo.common.compiler.support.AdaptiveCompiler
jdk=org.apache.dubbo.common.compiler.support.JdkCompiler
javassist=org.apache.dubbo.common.compiler.support.JavassistCompiler

3.2 Dubbo SPI

Let's first demonstrate the implementation of Dubbo SPI through code.

① Create an extension point (ie, service provider interface)

extension point

 package dubbo.spi;
 
import org.apache.dubbo.common.extension.SPI;
 
@SPI  // 注解标记当前接口为扩展点
public interface DataBaseSPI {
    public void dataBaseOperation();
}

② Create an extension point implementation class

• MysqlDataBaseSPIImpl

extension class 1

 package dubbo.spi.impl;
 
import dubbo.spi.DataBaseSPI;
 
public class MysqlDataBaseSPIImpl implements DataBaseSPI {
 
    @Override
    public void dataBaseOperation() {
        System.out.println("Dubbo SPI Operate Mysql database!!!");
    }
}

• OracleDataBaseSPIImpl

extension class 2

 package dubbo.spi.impl;
 
import dubbo.spi.DataBaseSPI;
 
public class OracleDataBaseSPIImpl implements DataBaseSPI {
 
    @Override
    public void dataBaseOperation() {
        System.out.println("Dubbo SPI Operate Oracle database!!!");
    }
}

③Create a dubbo.spi.DataBaseSPI file in the project META-INF/dubbo/ directory:

dubbo.spi.DataBaseSPI

 #扩展实例名称=扩展点实现类
mysql = dubbo.spi.impl.MysqlDataBaseSPIImpl
oracle = dubbo.spi.impl.OracleDataBaseSPIImpl

PS: In the file content, the left side of the equal sign is the extension instance name corresponding to the extension class, and the right side is the extension class (the content format is one extension class per line, and multiple extension classes are divided into multiple lines)

④ Run the code:

DubboSpiTest#main()

 package dubbo.spi;
 
import org.apache.dubbo.common.extension.ExtensionLoader;
 
public class DubboSpiTest {
 
    public static void main(String args[]){
        // 使用扩展类加载器加载指定扩展的实现
        ExtensionLoader<DataBaseSPI> dataBaseSpis = ExtensionLoader.getExtensionLoader(DataBaseSPI.class);
        // 根据指定的名称加载扩展实例(与dubbo.spi.DataBaseSPI中一致)
        DataBaseSPI spi = dataBaseSpis.getExtension("mysql");
        spi.dataBaseOperation();
         
        DataBaseSPI spi2 = dataBaseSpis.getExtension("oracle");
        spi2.dataBaseOperation();
    }
}

⑤ Running result:

 Dubbo SPI Operate Mysql database!!!
Dubbo SPI Operate Oracle database!!!

Intuitively from the above code implementation, Dubbo SPI is similar to Java SPI in use, but there are differences.

same:

1. The extension point is the service providing interface, the extension is the service providing the interface implementation class, and the extension configuration file is the configuration file in the services directory.
2. Both create the loader first and then access the specific service implementation class, including the deep level, when the loader is initialized, the extension configuration file is not parsed in real time to obtain the extension point implementation, but the extension point implementation is formally parsed and obtained when it is used ( i.e. lazy loading).

different:

1. The extension point must be decorated with @SPI annotation (the parsing in the source code will check this).
2. Each extension (service provider interface implementation class) in the extension configuration file in Dubbo is assigned a name.
3. Dubbo SPI is obtained directly through the name specified in the extension configuration file when obtaining the instance of the extension class, rather than the loop traversal of the Java SPI, which is more flexible in use.

3.3 Source code analysis

Using the above code implementation as the source code analysis entry, learn how Dubbo SPI is implemented.

ExtensionLoader

① Create an extension loader corresponding to the extension type through ExtensionLoader.getExtensionLoader(Classtype).

ExtensionLoader#getExtensionLoader()

 public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type) {
    if (type == null) {
        throw new IllegalArgumentException("Extension type == null");
    }
    // 校验当前类型是否为接口
    if (!type.isInterface()) {
        throw new IllegalArgumentException("Extension type (" + type + ") is not an interface!");
    }
    // 接口上是否使用了@SPI注解
    if (!withExtensionAnnotation(type)) {
        throw new IllegalArgumentException("Extension type (" + type +
                ") is not an extension, because it is NOT annotated with @" + SPI.class.getSimpleName() + "!");
    }
    // 从内存中读取该扩展点的扩展类加载器
    ExtensionLoader<T> loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
    // 内存中不存在则直接new一个扩展
    if (loader == null) {
        EXTENSION_LOADERS.putIfAbsent(type, new ExtensionLoader<T>(type));
        loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
    }
    return loader;
}

There are three important logics in the getExtensionLoader() method:

1. Determines whether the current type is an interface type.
2. Whether the current extension point is decorated with @SPI annotation.
3. EXTENSION_LOADERS is an in-memory cache of ConcurrentMap type. If an extension loader of this type exists in memory, it will be used directly. If it does not exist, create a new one and put it into the in-memory cache.

Look again at the source code of new ExtensionLoader(type)

ExtensionLoader#ExtensionLoader()

 // 私有构造器
private ExtensionLoader(Class<?> type) {
     this.type = type;
     // 创建ExtensionFactory自适应扩展
     objectFactory = (type == ExtensionFactory.class ? null : ExtensionLoader.getExtensionLoader(ExtensionFactory.class).getAdaptiveExtension());
 }

Important: The constructor is a private type, that is, the external cannot directly use the constructor to create an instance of ExtensionLoader.

Each time an ExtensionLoader instance is initialized, type and objectFactory are initialized, and type is the extension point type; objectFactory is the ExtensionFactory type.

② Use getExtension() to obtain the instance of the extension class with the specified name. getExtension is an overloaded method, namely getExtension(String name) and getExtension(String name, boolean wrap). The getExtension(String name) method finally calls getExtension(String name, boolean wrap) method.

ExtensionLoader#getExtension()

 public T getExtension(String name) {
     // 调用两个参数的getExtension方法，默认true表示需要对扩展实例做包装
     return getExtension(name, true);
 }
 
 public T getExtension(String name, boolean wrap) {
    if (StringUtils.isEmpty(name)) {
        throw new IllegalArgumentException("Extension name == null");
    }
    if ("true".equals(name)) {
        return getDefaultExtension();
    }
    // 获取Holder实例，先从ConcurrentMap类型的内存缓存中取，没值会new一个并存放到内存缓存中
    // Holder用来存放一个类型的值，这里用于存放扩展实例
    final Holder<Object> holder = getOrCreateHolder(name);
    // 从Holder读取该name对应的实例
    Object instance = holder.get();
    if (instance == null) {
       // 同步控制
       synchronized (holder) {
          instance = holder.get();
          // double check
          if (instance == null) {
             // 不存在扩展实例则解析扩展配置文件，实时创建
             instance = createExtension(name, wrap);
             holder.set(instance);
          }
        }
     }
     return (T) instance;
}

Holder class: This is used to store the specified extension instance

③ Use createExtension() to create an extension instance

ExtensionLoader#createExtension()

 // 部分createExtension代码
private T createExtension(String name, boolean wrap) {
   // 先调用getExtensionClasses()解析扩展配置文件，并生成内存缓存，
   // 然后根据扩展实例名称获取对应的扩展类
   Class<?> clazz = getExtensionClasses().get(name);
   if (clazz == null) {
       throw findException(name);
   }
   try {
       // 根据扩展类生成实例并对实例做包装(主要是进行依赖注入和初始化)
       // 优先从内存中获取该class类型的实例
       T instance = (T) EXTENSION_INSTANCES.get(clazz);
       if (instance == null) {
           // 内存中不存在则直接初始化然后放到内存中
           EXTENSION_INSTANCES.putIfAbsent(clazz, clazz.newInstance());
           instance = (T) EXTENSION_INSTANCES.get(clazz);
       }
       // 主要是注入instance中的依赖
       injectExtension(instance);
       ......
}

createExtension() method: Create an extension instance. EXTENSION_INSTANCES in the method is a memory cache of ConcurrentMap type, first fetched from memory, and there is no re-creation in memory; one of the core methods is getExtensionClasses():

ExtensionLoader#getExtensionClasses()

 private Map<String, Class<?>> getExtensionClasses() {
   // 优先从内存缓存中读
    Map<String, Class<?>> classes = cachedClasses.get();
    if (classes == null) {
        // 采用同步手段解析配置文件
        synchronized (cachedClasses) {
            // double check
            classes = cachedClasses.get();
            if (classes == null) {
                // 正式开始解析配置文件
                classes = loadExtensionClasses();
                cachedClasses.set(classes);
            }
        }
    }
    return classes;
}

cachedClasses is a memory cache of Holder<map<string, class>> type. In getExtensionClasses, the memory cache will be read first. If it does not exist in memory, the configuration file will be parsed in a synchronous way. Finally, the configuration file will be parsed in the loadExtensionClasses method to complete the extension configuration. Read the extension class from the file:

ExtensionLoader#loadExtensionClasses()

 // 在getExtensionClasses方法中是以同步的方式调用，是线程安全
private Map<String, Class<?>> loadExtensionClasses() {
   // 缓存默认扩展名称
   cacheDefaultExtensionName();
   Map<String, Class<?>> extensionClasses = new HashMap<>();
   // strategies策略类集合，分别对应dubbo的三个配置文件目录
   for (LoadingStrategy strategy : strategies) {
      loadDirectory(extensionClasses, strategy.directory(), type.getName(), strategy.preferExtensionClassLoader(), strategy.overridden(), strategy.excludedPackages());
      loadDirectory(extensionClasses, strategy.directory(), type.getName().replace("org.apache", "com.alibaba"), strategy.preferExtensionClassLoader(), strategy.overridden(),
           strategy.excludedPackages());
   }
 
   return extensionClasses;
}

The strategies in the source code are the static volatile LoadingStrategy[] strategies array, which is initialized by loading the configuration file from the META-INF/services/ directory through Java SPI. It contains three classes by default:

• DubboInternalLoadingStrategy
• DubboLoadingStrategy
• ServicesLoadingStrategy

Corresponding to the three directories of dubbo:

• META-INF/dubbo/internal
• META-INF/dubbo
• META-INF/services

The above source code analysis is only a brief introduction to Dubbo SPI. Dubbo has a wide range of applications for SPI, such as: serialized components, load balancing, etc. all use SPI technology, and many SPI functions have not been analyzed, such as: Adaptation extension, Activate activity extension, etc., interested students can study more deeply.

4. Spring SPI

Spring SPI follows the design idea of Java SPI, but there are also differences in implementation with Java SPI and Dubbo SPI. Spring implements the SPI mechanism through spring.handlers and spring.factories, which can be used without modifying the Spring source code. Do the extension development of the Spring framework.

4.1 Basic Concepts

• DefaultNamespaceHandlerResolver

Similar to the Java SPI's ServiceLoader, it is responsible for parsing the spring.handlers configuration file, generating a mapping of namespaceUri and NamespaceHandler names, and instantiating the NamespaceHandler.

• spring.handlers

Custom tag configuration file; Spring introduced spring.handlers in 2.0. By configuring the spring.handlers file to implement custom tags and using the custom tag parsing class to parse and implement dynamic expansion, the content configuration is as follows:

 http\://www.springframework.org/schema/c=org.springframework.beans.factory.xml.SimpleConstructorNamespaceHandler
http\://www.springframework.org/schema/p=org.springframework.beans.factory.xml.SimplePropertyNamespaceHandler
http\://www.springframework.org/schema/util=org.springframework.beans.factory.xml.UtilNamespaceHandler
 
spring.handlers实现的SPI是以namespaceUri作为key，NamespaceHandler作为value，建立映射关系，在解析标签时通过namespaceUri获取相应的NamespaceHandler来解析

• SpringFactoriesLoader

Similar to Java SPI's ServiceLoader, it is responsible for parsing spring.factories, and returns after instantiating all implementation classes of the specified interface.

• spring.factories

Spring introduced spring.factories in 3.2, an enhanced version of the SPI configuration file, to provide support for the implementation of Spring's SPI mechanism. The content configuration is as follows:

 # PropertySource Loaders
org.springframework.boot.env.PropertySourceLoader=\
org.springframework.boot.env.PropertiesPropertySourceLoader,\
org.springframework.boot.env.YamlPropertySourceLoader
 
# Run Listeners
org.springframework.boot.SpringApplicationRunListener=\org.springframework.boot.context.event.EventPublishingRunListener
 
spring.factories实现的SPI是以接口的全限定名作为key，接口实现类作为value，多个实现类用逗号隔开，最终返回的结果是该接口所有实现类的实例集合

• load path

The Java SPI loads the service provisioning interface configuration from the /META-INF/services directory, while Spring loads the configuration from the META-INF/spring.handlers and META-INF/spring.factories directories by default, where the path to META-INF/spring.handlers can be By re-specifying when an instance is created, META-INF/spring.factories are fixed immutable.

4.2 spring.handlers

First, the implementation of spring.handlers is initially introduced through the code.

4.2.1 spring.handlers SPI

① Create NameSpaceHandler

MysqlDataBaseHandler

 package spring.spi.handlers;
 
import org.springframework.beans.factory.config.BeanDefinition;
import org.springframework.beans.factory.xml.NamespaceHandlerSupport;
import org.springframework.beans.factory.xml.ParserContext;
import org.w3c.dom.Element;
 
// 继承抽象类
public class MysqlDataBaseHandler extends NamespaceHandlerSupport {
 
    @Override
    public void init() {   
    }
     
    @Override
    public BeanDefinition parse(Element element, ParserContext parserContext) {
        System.out.println("MysqlDataBaseHandler!!!");
        return null;
    }
}

OracleDataBaseHandler

 package spring.spi.handlers;
 
import org.springframework.beans.factory.config.BeanDefinition;
import org.springframework.beans.factory.xml.NamespaceHandlerSupport;
import org.springframework.beans.factory.xml.ParserContext;
import org.w3c.dom.Element;
 
public class OracleDataBaseHandler extends NamespaceHandlerSupport {
 
    @Override
    public void init() {
    }
 
    @Override
    public BeanDefinition parse(Element element, ParserContext parserContext) {
        System.out.println("OracleDataBaseHandler!!!");
        return null;
    }
}

② Create the spring.handlers file in the project META-INF/ directory:

document content:

spring.handlers

 #一个namespaceUri对应一个handler
http\://www.mysql.org/schema/mysql=spring.spi.handlers.MysqlDataBaseHandler
http\://www.oracle.org/schema/oracle=spring.spi.handlers.OracleDataBaseHandler

③ Run the code:

SpringSpiTest#main()

 package spring.spi;
 
import org.springframework.beans.factory.xml.DefaultNamespaceHandlerResolver;
import org.springframework.beans.factory.xml.NamespaceHandler;
 
public class SpringSpiTest {
 
    public static void main(String args[]){
        // spring中提供的默认namespace URI解析器
        DefaultNamespaceHandlerResolver resolver = new DefaultNamespaceHandlerResolver();
        // 此处假设nameSpaceUri已从xml文件中解析出来，正常流程是在项目启动的时候会解析xml文件，获取到对应的自定义标签
        // 然后根据自定义标签取得对应的nameSpaceUri
        String mysqlNameSpaceUri = "http://www.mysql.org/schema/mysql";
        NamespaceHandler  handler = resolver.resolve(mysqlNameSpaceUri);
        // 验证自定义NamespaceHandler，这里参数传null，实际使用中传具体的Element
        handler.parse(null, null);
         
        String oracleNameSpaceUri = "http://www.oracle.org/schema/oracle";
        handler = resolver.resolve(oracleNameSpaceUri);
        handler.parse(null, null);
    }
}

④ Operation result:

 MysqlDataBaseHandler!!!
OracleDataBaseHandler!!!

The above code implements dynamic parsing of custom tags by parsing spring.handlers, and uses NameSpaceURI as the key to obtain the specific NameSpaceHandler implementation class, which is different from Java SPI, where:

DefaultNamespaceHandlerResolver is the default implementation class of the NamespaceHandlerResolver interface, which is used to resolve custom tags.

• The DefaultNamespaceHandlerResolver.resolve(String namespaceUri) method takes namespaceUri as a parameter, loads the META-INF/spring.handlers configuration file in each jar package by default, and establishes the mapping between NameSpaceURI and NameSpaceHandler by parsing the spring.handlers file.
• The default path for loading configuration files is META-INF/spring.handlers, but it can be modified using the DefaultNamespaceHandlerResolver(ClassLoader, String) constructor. DefaultNamespaceHandlerResolver has multiple overloaded methods.
• The DefaultNamespaceHandlerResolver.resolve(String namespaceUri) method is mainly called by the parseCustomElement() and decorateIfRequired() methods of the BeanDefinitionParserDelegate, so the spring.handlers SPI mechanism is mainly used in the bean scanning and parsing process.

4.2.2 Source code analysis

Let's start from the above code and go deep into the source code to understand how the SPI implemented by spring handlers works.

• DefaultNamespaceHandlerResolver

① The DefaultNamespaceHandlerResolver.resolve() method itself is to obtain the corresponding namespaceHandler according to the namespaceUri to parse the label. The core source code:

DefaultNamespaceHandlerResolver#resolve()

 public NamespaceHandler resolve(String namespaceUri) {
    // 1、核心逻辑之一：获取namespaceUri和namespaceHandler映射关系
    Map<String, Object> handlerMappings = getHandlerMappings();
    // 根据namespaceUri参数取对应的namespaceHandler全限定类名or NamespaceHandler实例
    Object handlerOrClassName = handlerMappings.get(namespaceUri);
    if (handlerOrClassName == null) {
        return null;
    }
    // 2、handlerOrClassName是已初始化过的实例则直接返回
    else if (handlerOrClassName instanceof NamespaceHandler) {
        return (NamespaceHandler) handlerOrClassName;
    }else {
        String className = (String) handlerOrClassName;
        try {
            ///3、使用反射根据namespaceHandler全限定类名加载实现类
            Class<?> handlerClass = ClassUtils.forName(className, this.classLoader);
            if (!NamespaceHandler.class.isAssignableFrom(handlerClass)) {
                throw new FatalBeanException("Class [" + className + "] for namespace [" + namespaceUri +
                        "] does not implement the [" + NamespaceHandler.class.getName() + "] interface");
            }
            // 3.1、初始化namespaceHandler实例
            NamespaceHandler namespaceHandler = (NamespaceHandler) BeanUtils.instantiateClass(handlerClass);
            // 3.2、 初始化，不同的namespaceHandler实现类初始化方法逻辑有差异
            namespaceHandler.init();
            // 4、将初始化好的实例放入内存缓存中，下次解析到相同namespaceUri标签时直接返回，避免再次初始化
            handlerMappings.put(namespaceUri, namespaceHandler);
            return namespaceHandler;
        }catch (ClassNotFoundException ex) {
            throw new FatalBeanException("NamespaceHandler class [" + className + "] for namespace [" +
                    namespaceUri + "] not found", ex);
        }catch (LinkageError err) {
            throw new FatalBeanException("Invalid NamespaceHandler class [" + className + "] for namespace [" +
                    namespaceUri + "]: problem with handler class file or dependent class", err);
        }
    }
}

Step 1: getHandlerMappings() in the source code is a relatively core method. It parses spring.handlers through lazy loading and returns the mapping relationship between namespaceUri and NamespaceHandler.

Step 2: Return the fully qualified name or specific instance of the corresponding NamespaceHandler according to the namespaceUri (the name or the instance depends on whether it has been initialized, if the initialized instance will be returned directly)

Step 3: It is the fully qualified name of the NamespaceHandler implementation class. Through the third step in the above source code, use reflection to initialize.

Step 4: Put the initialized instance into the handlerMappings memory cache, which is why step 2 may be of the NamespaceHandler type.

After reading the source code of the resolve method, and then look at the call scenario of the resolve method in Spring, you can roughly understand the usage scenario of spring.handlers:

It can be seen that resolve() is mainly used in the tag parsing process, and is mainly called in the parseCustomElement and decorateIfRequired methods of BeanDefinitionParserDelegate.

② One of the core logics in the resolve() source code is the call to getHandlerMappings(), which implements the parsing of the META-INF/spring.handlers files in each jar package in getHandlerMappings(), such as:

DefaultNamespaceHandlerResolver#getHandlerMappings()

 private Map<String, Object> getHandlerMappings() {
    Map<String, Object> handlerMappings = this.handlerMappings;
    // 使用线程安全的解析逻辑，避免在并发场景下重复的解析，没必要重复解析
    // 这里在同步代码块的内外对handlerMappings == null作两次判断很有必要，采用懒汉式初始化
    if (handlerMappings == null) {
        synchronized (this) {
            handlerMappings = this.handlerMappings;
            // duble check
            if (handlerMappings == null) {
                if (logger.isDebugEnabled()) {
                    logger.debug("Loading NamespaceHandler mappings from [" + this.handlerMappingsLocation + "]");
                }
                try {
                    // 加载handlerMappingsLocation目录文件，handlerMappingsLocation路径值可变，默认是META-INF/spring.handlers
                    Properties mappings =
                            PropertiesLoaderUtils.loadAllProperties(this.handlerMappingsLocation, this.classLoader);
                    if (logger.isDebugEnabled()) {
                        logger.debug("Loaded NamespaceHandler mappings: " + mappings);
                    }
                    // 初始化内存缓存
                    handlerMappings = new ConcurrentHashMap<String, Object>(mappings.size());
                    // 将加载到的属性合并到handlerMappings中
                    CollectionUtils.mergePropertiesIntoMap(mappings, handlerMappings);
                    // 赋值内存缓存
                    this.handlerMappings = handlerMappings;
                }catch (IOException ex) {
                    throw new IllegalStateException(
                            "Unable to load NamespaceHandler mappings from location [" + this.handlerMappingsLocation + "]", ex);
                }
            }
        }
    }
    return handlerMappings;
}

This.handlerMappings in the source code is a Map-type memory cache, which stores the resolved namespaceUri and NameSpaceHandler instances.

The implementation in the body of the getHandlerMappings() method uses a thread-safe method and adds synchronization logic.

By reading the source code, you can understand that Spring's implementation of SPI logic based on spring.handlers is relatively simple, but the application is more flexible, and the support for custom tags is very convenient. It is easy to achieve access without modifying the Spring source code, as defined in Dubbo The various Dubbo tags make good use of spring.handlers.

Spring provides such flexible functionality, how is it applied? Let's take a brief look at parseCustomElement().

• BeanDefinitionParserDelegate.parseCustomElement()

As a tool type method, resolve is used in many places. Here is only a brief introduction to the application in BeanDefinitionParserDelegate.parseCustomElement().

BeanDefinitionParserDelegate#parseCustomElement()

 public BeanDefinition parseCustomElement(Element ele, BeanDefinition containingBd) {
     // 获取标签的namespaceUri
     String namespaceUri = getNamespaceURI(ele);
     // 首先获得DefaultNamespaceHandlerResolver实例在再以namespaceUri作为参数调用resolve方法解析取得NamespaceHandler
     NamespaceHandler handler = this.readerContext.getNamespaceHandlerResolver().resolve(namespaceUri);
     if (handler == null) {
         error("Unable to locate Spring NamespaceHandler for XML schema namespace [" + namespaceUri + "]", ele);
         return null;
     }
     // 调用NamespaceHandler中的parse方法开始解析标签
     return handler.parse(ele, new ParserContext(this.readerContext, this, containingBd));
 }

parseCustomElement is used as an intermediate method for parsing tags, and then look at the invocation of parseCustomElement:

Called in parseBeanDefinitions(), look at the source code of parseBeanDefinitions

DefaultBeanDefinitionDocumentReader#parseBeanDefinitions()

 protected void parseBeanDefinitions(Element root, BeanDefinitionParserDelegate delegate) {
    // spring内部定义的标签为默认标签，即非spring内部定义的标签都不是默认的namespace
    if (delegate.isDefaultNamespace(root)) {
        NodeList nl = root.getChildNodes();
        for (int i = 0; i < nl.getLength(); i++) {
            Node node = nl.item(i);
            if (node instanceof Element) {
                Element ele = (Element) node;
                // root子标签也做此判断
                if (delegate.isDefaultNamespace(ele)) {
                    parseDefaultElement(ele, delegate);
                }else{
                    // 子标签非spring默认标签(即自定义标签)也走parseCustomElement来解析
                    delegate.parseCustomElement(ele);
                }
            }
        }
    }else {
        // 非spring的默认标签(即自定义的标签)走parseCustomElement来解析
        delegate.parseCustomElement(root);
    }
}

At this point, it is very clear. Before calling, it is judged whether it is the default label of Spring. If it is not the default label, parseCustomElement is called to parse, and finally the resolve method is called.

Section 4.2.3

Since the introduction of spring.handlers in Spring 2.0, it provides more entries and means for Spring's dynamic expansion, and provides strong support for the implementation of custom tags.

Many articles focus on the implementation of spring.factories when introducing Spring SPI, and rarely mention spring.handlers, which was introduced very early. However, through personal analysis and comparison with Java SPI, spring.handlers is also an implementation of SPI. Just based on xml implementation.

Compared with Java SPI, the SPI based on spring.handlers is more flexible, does not require traversal, and can be directly mapped. It is more similar to the implementation idea of Dubbo SPI. Each class specifies a name (only in spring.handlers, namespaceUri is used as the key, The name specified in the Dubbo configuration is used as the key).

4.3 spring.factories

Also, the test code is used to introduce the logic of spring.factories to implement SPI.

4.3.1 spring.factories SPI

① Create DataBaseSPI interface

interface

 package spring.spi.factories;
 
public interface DataBaseSPI {
    public void dataBaseOperation();
}

② Create an implementation class of the DataBaseSPI interface

MysqlDataBaseImpl

 #实现类1
package spring.spi.factories.impl;
 
import spring.spi.factories.DataBaseSPI;
 
public class MysqlDataBaseImpl implements DataBaseSPI {
 
    @Override
    public void dataBaseOperation() {
        System.out.println("Mysql database test!!!!");
    }
}

MysqlDataBaseImpl

 #实现类2
package spring.spi.factories.impl;
 
import spring.spi.factories.DataBaseSPI;
 
public class OracleDataBaseImpl implements DataBaseSPI {
 
    @Override
    public void dataBaseOperation() {
        System.out.println("Oracle database test!!!!");
    }
}

③ Create the spring.factories file in the project META-INF/ directory:

document content

spring.factories

 #key是接口的全限定名，value是接口的实现类
spring.spi.factories.DataBaseSPI = spring.spi.factories.impl.MysqlDataBaseImpl,spring.spi.factories.impl.OracleDataBaseImpl

④ Run the code

SpringSpiTest#main()

 package spring.spi.factories;
 
import java.util.List;
 
import org.springframework.core.io.support.SpringFactoriesLoader;
 
public class SpringSpiTest {
 
    public static void main(String args[]){
         
        // 调用SpringFactoriesLoader.loadFactories方法加载DataBaseSPI接口所有实现类的实例
        List<DataBaseSPI> spis= SpringFactoriesLoader.loadFactories(DataBaseSPI.class, Thread.currentThread().getContextClassLoader());
         
        // 遍历DataBaseSPI接口实现类实例
        for(DataBaseSPI spi : spis){
            spi.dataBaseOperation();
        }
    }
}

⑤ Operation result

 Mysql database test!!!!
Oracle database test!!!!

It can be seen from the above sample code that the SPI implemented by spring.facotries is very similar to the Java SPI. They first obtain the implementation class of the specified interface type, and then traverse all the implementations. But there are some differences:

(1) Configuration:

Java SPI is a configuration file corresponding to a service providing interface. The configuration file stores all implementation classes of the current interface. Multiple service providing interfaces correspond to multiple configuration files, and all configurations are in the services directory;

Spring factories SPI is a spring.factories configuration file that stores multiple interfaces and corresponding implementation classes. The fully qualified name of the interface is used as the key, and the implementation class is used as the value to configure. Multiple implementation classes are separated by commas. Only one configuration of spring.factories is used. document.

(2) On the realization

Java SPI uses lazy loading mode, that is, when ServiceLoader.load() is called, only the instance of ServiceLoader is returned, and the configuration file corresponding to the interface has not been parsed. When it is used, that is, the implementation class instance of the interface provided by the service is officially parsed and returned when it is traversed. ;

Spring factories SPI has parsed the spring.facotries file when calling SpringFactoriesLoader.loadFactories() to return an instance of the interface implementation class (the implementation details are explained in the source code analysis).

4.3.2 Source code analysis

Let's start with the test code, understand the SPI implementation source code of spring.factories, and detail the implementation of spring.factories.

• The SpringFactoriesLoader test code entry directly calls the SpringFactoriesLoader.loadFactories() static method to start parsing the spring.factories file, and returns the interface type specified in the method parameters, such as the implementation class instance of the DataBaseSPI interface in the test code.

SpringFactoriesLoader#loadFactories()

 public static <T> List<T> loadFactories(Class<T> factoryClass, ClassLoader classLoader) {
    Assert.notNull(factoryClass, "'factoryClass' must not be null");
    ClassLoader classLoaderToUse = classLoader;
    // 1.确定类加载器
    if (classLoaderToUse == null) {
        classLoaderToUse = SpringFactoriesLoader.class.getClassLoader();
    }
    // 2.核心逻辑之一：解析各jar包中META-INF/spring.factories文件中factoryClass的实现类全限定名
    List<String> factoryNames = loadFactoryNames(factoryClass, classLoaderToUse);
    if (logger.isTraceEnabled()) {
        logger.trace("Loaded [" + factoryClass.getName() + "] names: " + factoryNames);
    }
    List<T> result = new ArrayList<T>(factoryNames.size());
    // 3.遍历实现类的全限定名并进行实例化
    for (String factoryName : factoryNames) {
        result.add(instantiateFactory(factoryName, factoryClass, classLoaderToUse));
    }
    // 排序
    AnnotationAwareOrderComparator.sort(result);
    // 4.返回实例化后的结果集
    return result;
}

In the source code, loadFactoryNames() is another core method. It parses the fully qualified name of the implementation class of the specified interface in the spring.factories file. See the subsequent source code for the implementation logic.

After the fully qualified name of the implementation class is obtained through the second step of parsing in the source code, the implementation class is instantiated one by one through the instantiateFactory() method in the third step.

Look at how the loadFactoryNames() source code is parsed to get the fully qualified name of the implementation class:

SpringFactoriesLoader#loadFactoryNames()

 public static List<String> loadFactoryNames(Class<?> factoryClass, ClassLoader classLoader) {
    // 1.接口全限定名
    String factoryClassName = factoryClass.getName();
    try {
        // 2.加载META-INF/spring.factories文件路径(分布在各个不同jar包里，所以这里会是多个文件路径，枚举返回)
        Enumeration<URL> urls = (classLoader != null ? classLoader.getResources(FACTORIES_RESOURCE_LOCATION) :
                ClassLoader.getSystemResources(FACTORIES_RESOURCE_LOCATION));
        List<String> result = new ArrayList<String>();
        // 3.遍历枚举集合，逐个解析spring.factories文件
        while (urls.hasMoreElements()) {
            URL url = urls.nextElement();
            Properties properties = PropertiesLoaderUtils.loadProperties(new UrlResource(url));
            String propertyValue = properties.getProperty(factoryClassName);
            // 4.spring.factories文件中一个接口的实现类有多个时会用逗号隔开，这里拆开获取实现类全限定名
            for (String factoryName : StringUtils.commaDelimitedListToStringArray(propertyValue)) {
                result.add(factoryName.trim());
            }
        }
        return result;
    }catch (IOException ex) {
        throw new IllegalArgumentException("Unable to load factories from location [" +
                FACTORIES_RESOURCE_LOCATION + "]", ex);
    }
}

The second step in the source code is to obtain the META-INF/spring.factories file path in all jar packages and return it as an enumeration value.

The third step in the source code begins to traverse the spring.factories file path, load and parse one by one, and integrate the implementation class name of the factoryClass type.

After obtaining the set of fully qualified names of the implementation classes, instantiate them one by one according to the names of the implementation classes, and continue to look at the source code of the instantiateFactory() method:

SpringFactoriesLoader#instantiateFactory()

 private static <T> T instantiateFactory(String instanceClassName, Class<T> factoryClass, ClassLoader classLoader) {
    try {
        // 1.使用classLoader类加载器加载instanceClassName类
        Class<?> instanceClass = ClassUtils.forName(instanceClassName, classLoader);
        if (!factoryClass.isAssignableFrom(instanceClass)) {
            throw new IllegalArgumentException(
                    "Class [" + instanceClassName + "] is not assignable to [" + factoryClass.getName() + "]");
        }
        // 2.instanceClassName类中的构造方法
        Constructor<?> constructor = instanceClass.getDeclaredConstructor();
        ReflectionUtils.makeAccessible(constructor);
        // 3.实例化
        return (T) constructor.newInstance();
    }
    catch (Throwable ex) {
        throw new IllegalArgumentException("Unable to instantiate factory class: " + factoryClass.getName(), ex);
    }
}

The instantiation method is a private static method, which is different from loadFactories and loadFactoryNames.

The instantiation logic is implemented by reflection as a whole, which is a more general implementation.

Through the analysis of the source code, the SPI logic implemented by Spring factories is not very complicated, and the overall implementation is easy to understand.

Spring has introduced spring.factories in 3.2, so how is spring.factories used in the Spring framework? First look at the call of the loadFactories method:

From the invocation situation, Spring has not really used the spring.factories SPI since 3.2, and it is used in less places. However, what really promotes spring.factories is in Spring Boot, and briefly understand the invocation in Spring Boot.

• getSpringFactoriesInstances() getSpringFactoriesInstances() is not a method in the Spring framework, but a private method defined in the SpringApplication class in SpringBoot, which is called in many places. The source code is as follows:

SpringApplication#getSpringFactoriesInstance()

 // 单个参数getSpringFactoriesInstances方法
private <T> Collection<T> getSpringFactoriesInstances(Class<T> type) {
    // 默认调用多参的重载方法
    return getSpringFactoriesInstances(type, new Class<?>[] {});
}
// 多个参数的getSpringFactoriesInstances方法
private <T> Collection<T> getSpringFactoriesInstances(Class<T> type,
        Class<?>[] parameterTypes, Object... args) {
    ClassLoader classLoader = getClassLoader();
    // 调用SpringFactoriesLoader中的loadFactoryNames方法加载接口实现类的全限定名
    Set<String> names = new LinkedHashSet<>(
            SpringFactoriesLoader.loadFactoryNames(type, classLoader));
    // 实例化
    List<T> instances = createSpringFactoriesInstances(type, parameterTypes,
            classLoader, args, names);
    AnnotationAwareOrderComparator.sort(instances);
    return instances;
}

SpringFactoriesLoader.loadFactoryNames() is called in getSpringFactoriesInstances() to load the fully qualified name collection of the interface implementation class, and then initialize it.

In addition to the call of the getSpringFactoriesInstances() method in SpringBoot, the methods in SpringFactoriesLoader are also widely used in other logic to achieve dynamic expansion. I will not list them one by one here. Interested students can explore it by themselves.

Section 4.3.3

The Spring framework has not been effectively used after the introduction of spring.factories in 3.2, but it provides users of the framework with another dynamic expansion capability and entry, and provides developers with a lot of freedom to play, especially in SpringBoot Widespread use is enough to prove the status of spring.factories. The introduction of spring.factories not only improves the capabilities of the Spring framework, but also exposes its shortcomings:

First of all, the implementation of spring.factories is similar to Java SPI. After loading into the implementation class of the service provider interface, it needs to be traversed to access it, which is not very convenient.

Secondly, Spring's SpringFactoriesLoader class was defined as an abstract class before version 5.0.x, but after version 5.1.0, Sping officially changed SpringFactoriesLoader to a final class, and the type change is not friendly to the compatibility of previous and previous versions.

5. Application Practice

After introducing the core source code related to the SPI mechanism in Spring, let's take a look at how the lightweight version of the sub-database sub-table SDK developed in the project uses Spring's SPI mechanism to dynamically expand the sub-database and sub-table strategy.

Based on the particularity of the project, it does not use the mature sub-library and sub-table components in the current industry, but a set of lightweight sub-library and sub-table components developed by myself based on the plug-in principle of Mybatis. In order to meet the requirements of sub-database and sub-table in different scenarios, the relevant logic of sub-database and sub-table is extracted and separated in a strategy mode. The implementation of each sub-database and sub-table corresponds to a strategy, which supports the dynamic expansion of the sub-database and sub-table strategy by the user. , and the dynamic extension here makes use of spring.factories.

First, the flow chart of the lightweight version of the sub-database and sub-table components is given, and then we conduct a detailed analysis of the places where the Spring SPI is used in the flow chart.

illustrate:

1. In the above flow chart, the initialization of the data source and sub-database sub-table strategy is generated during the project startup process. After the strategy initialization is completed, it is cached in the memory.
2. When initiating a database operation command, it is necessary to analyze whether sub-database and sub-table are required (only the process that requires sub-database and sub-table is given in the process). table to complete database operations.

As can be seen from the above flow chart, the sub-database and sub-table SDK supports dynamic loading of sub-database and sub-table strategies through spring.factories to be compatible with different usage scenarios of different projects.

The strategy class diagram of the sub-library and sub-table part:

Among them: ShardingStrategy and DBTableShardingStrategy are interfaces; BaseShardingStrategy is the default implementation class; DefaultStrategy and CountryDbSwitchStrategy are the default database and table sharding strategies implemented in the SDK based on different scenarios.

When the project is actually used, the dynamically expanded sub-database sub-table strategy only needs to inherit the BaseShardingStrategy. When the sub-database sub-table strategy is initialized in the SDK, it is dynamically loaded through SpringFactoriesLoader.loadFactories().

6. Summary

The SPI technology separates the service interface from the service implementation to achieve decoupling, which greatly improves the scalability of the program.

This article focuses on the principles and related source code of Java's built-in SPI, Dubbo SPI and Spring SPI; firstly, it demonstrates the implementation of the three SPI technologies, and then deeply reads the implementation source code of the three SPIs through the demonstration code; it focuses on the introduction of Spring There are two implementations of SPI: spring.handlers and spring.factories, and the sub-database sub-table strategy loading implemented by spring.factories. I hope that by reading this article, readers can have a deeper understanding of SPI.