MO_or breaking the bubble bun-style teaching realizes multiple data sources

I. Introduction

This article mainly introduces an elegant, safe, easy-to-use, and transaction management-supporting Spring Boot way to integrate multiple data sources. This article does not introduce what, why and when to use multiple data sources. Small partners can use it according to their own situation. Adopt as appropriate

Tips:
After some application and practice based on the following knowledge, you can better understand this article

• Lambda, ThreadLocal, stack, queue, custom annotation
• IoC, AOP, Druid, Maven, Spring Boot

Since this article mainly explains the specific implementation of the code, there are many codes and comments. If you feel that the reading experience is not good, you can use the code editor to read with the open source code.
Multiple data source Gitee addresses
The corresponding project module is hei-dynamic-datasource

Second, the general idea

1. First assemble the default data source and multiple data sources through the configuration class and yml configuration file
2. Then through custom annotations and AOP, find the target class or method, and specify the data source Key value it uses
3. Finally, by inheriting the AbstractRoutingDataSource class, return the key value of the data source processed by AOP, find the corresponding configuration from the data source assembled in the first step and apply it

3. Test Cases

You can specify different data sources by adding @DataSource("value") to a class or method

 @Service
// 方法上的注解比类上注解优先级更高
@DataSource("slave2")
public class DynamicDataSourceTestService {
    @Autowired
    private SysUserDao sysUserDao;

    @Transactional
    public void updateUser(Long id){
        SysUserEntity user = new SysUserEntity();
        user.setUserId(id);
        user.setMobile("13500000002");
        sysUserDao.updateById(user);
    }

    @Transactional
    @DataSource("slave1")
    public void updateUserBySlave1(Long id){
        SysUserEntity user = new SysUserEntity();
        user.setUserId(id);
        user.setMobile("13500000001");
        sysUserDao.updateById(user);
    }

    @DataSource("slave2")
    @Transactional
    public void updateUserBySlave2(Long id){
        SysUserEntity user = new SysUserEntity();
        user.setUserId(id);
        user.setMobile("13500000003");
        sysUserDao.updateById(user);

        // 测试事务
        int i = 1/0;
    }
}

---

 @RunWith(SpringRunner.class)
@SpringBootTest
public class DynamicDataSourceTest {
    @Autowired
    private DynamicDataSourceTestService dynamicDataSourceTestService;

    @Test
    public void test(){
        Long id = 1L;

        dynamicDataSourceTestService.updateUser(id);
        dynamicDataSourceTestService.updateUserBySlave1(id);
        dynamicDataSourceTestService.updateUserBySlave2(id);
    }

}

4. Project structure

Five, code examples and analysis

5.1, maven related dependencies

 <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>

5.2, yml configuration

 dynamic:
  datasource:
    slave1:
      driver-class-name: com.mysql.cj.jdbc.Driver
      url: jdbc:mysql://localhost:3306/hei?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
      username: root
      password: 123456
    slave2:
      driver-class-name: com.mysql.cj.jdbc.Driver
      url: jdbc:mysql://localhost:3306/hei?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai
      username: root
      password: 123456

5.3. Custom annotations (DataSource)

 // 定义作用范围为（方法、接口、类、枚举、注解）
@Target({ElementType.METHOD, ElementType.TYPE})
// 保证运行时能被JVM或使用反射的代码使用
@Retention(RetentionPolicy.RUNTIME)
// 生成Javadoc时让使用了@DataSource这个注解的地方输出@DataSource这个注解或不同内容
@Documented
// 类继承中让子类继承父类@DataSource注解
@Inherited
public @interface DataSource {
    // @DataSource注解里传的参，这里主要传配置文件中不同数据源的标识，如@DataSource("slave1")
    String value() default "";
}

5.4, the aspect class (DataSourceAspect)

 // 声明、定义切面类
@Aspect
@Component
/**
 * 让该bean的执行顺序优先级最高，并不能控制加载入IoC的顺序
 * 如果一个方法被多个 @Around 增强，那就可以使用该注解指定顺序
 */
@Order(Ordered.HIGHEST_PRECEDENCE)
public class DataSourceAspect {
    protected Logger logger = LoggerFactory.getLogger(getClass());

    // 指明通知在使用@DataSource注解标注下才触发
    @Pointcut("@annotation(io.renren.commons.dynamic.datasource.annotation.DataSource) " +
            "|| @within(io.renren.commons.dynamic.datasource.annotation.DataSource)")
    public void dataSourcePointCut() {

    }

    // 对通知方法的具体实现并采用环绕通知设定方法与切面的执行顺序，即在方法执行前和后触发
    @Around("dataSourcePointCut()")
    /**
     * ProceedingJoinPoint继承了JoinPoint，相较于JoinPoint暴露了proceed方法，该类仅配合实现around通知
     * JoinPoint类，用来获取代理类和被代理类的信息
     * 调用proceed方法，表示继续执行目标方法（即加了@DataSource注解的方法）
     */
    public Object around(ProceedingJoinPoint point) throws Throwable {

        // 通过反射获得被代理类（目标对象）
        Class targetClass = point.getTarget().getClass();
        System.out.println("targetClass:" + targetClass);
        /**
         * 获得被代理类（目标对象）的方法签名
         * signature加签是一种简单、 低成本、保障数据安全的方式
         */
        MethodSignature signature = (MethodSignature) point.getSignature();
        /**
         * 获得被代理类（目标对象）的方法
         * 这里获得方法也可以通过反射和getTarget()，但步骤更多更复杂
         */
        Method method = signature.getMethod();
        System.out.println("method:" + method);

        // 获得被代理类（目标对象）的注解对象
        DataSource targetDataSource = (DataSource) targetClass.getAnnotation(DataSource.class);
        System.out.println("targetDataSource:" + targetDataSource);
        // 获得被代理类（目标对象）的方法的注解对象
        DataSource methodDataSource = method.getAnnotation(DataSource.class);
        System.out.println("methodDataSource:" + methodDataSource);
        // 判断被代理类（目标对象）的注解对象或者被代理类（目标对象）的方法的注解对象不为空
        if (targetDataSource != null || methodDataSource != null) {
            String value;
            // 优先用被代理类（目标对象）的方法的注解对象的值进行后续赋值
            if (methodDataSource != null) {
                value = methodDataSource.value();
            } else {
                value = targetDataSource.value();
            }

            /**
             * DynamicContextHolder是自己实现的栈数据结构
             * 将注解对象的值入栈
             */
            DynamicContextHolder.push(value);
            logger.debug("set datasource is {}", value);
        }

        try {
            // 继续执行被代理类（目标对象）的方法
            return point.proceed();
        } finally {
            // 清空栈中数据
            DynamicContextHolder.poll();
            logger.debug("clean datasource");
        }
    }
}

5.5. Multi-data source context operation support class (DynamicContextHolder)

 public class DynamicContextHolder {
    /**
     * Lambda构造 本地线程变量
     * 用于避免多次创建数据库连接或者多线程使用同一个数据库连接
     * 减少数据库连接创建关闭对程序执行效率的影响与服务器压力
     *
     * 这里使用数组队列实现栈数据结构
     * 主要为了解决调用链执行顺序问题，比如这里有ABC三个方法，它们分别使用的是不同数据源
     * A调用B，B调用C，这里数据源切换时就要保证，C方法数据源要最先被清除，再是B，最后A，即后进先出LIFO
     */
    private static final ThreadLocal<Deque<String>> CONTEXT_HOLDER = ThreadLocal.withInitial(ArrayDeque::new);

    /**
     * 获得当前线程数据源
     *
     * @return 数据源名称
     */
    public static String peek() {
        return CONTEXT_HOLDER.get().peek();
    }

    /**
     * 设置当前线程数据源
     *
     * @param dataSource 数据源名称
     */
    public static void push(String dataSource) {
        CONTEXT_HOLDER.get().push(dataSource);
    }

    /**
     * 清空当前线程数据源
     */
    public static void poll() {
        Deque<String> deque = CONTEXT_HOLDER.get();
        deque.poll();
        if (deque.isEmpty()) {
            CONTEXT_HOLDER.remove();
        }
    }

}

5.6. Multiple data source classes (DynamicDataSource)

 public class DynamicDataSource extends AbstractRoutingDataSource {

    /**
     * 返回当前上下文环境的数据源key
     * 后续会根据这个key去找到对应的数据源属性
     */
    @Override
    protected Object determineCurrentLookupKey() {
        return DynamicContextHolder.peek();
    }

}

5.7. Multi-data source configuration class (DynamicDataSourceConfig)

 /**
 * 通过@EnableConfigurationProperties(DynamicDataSourceProperties.class)
 * 将DynamicDataSourceProperties.class注入到Spring容器中 
 */
@Configuration
@EnableConfigurationProperties(DynamicDataSourceProperties.class)
public class DynamicDataSourceConfig {
    // 这里properties已经包含了yml配置中所对应的多数据源的属性了
    @Autowired
    private DynamicDataSourceProperties properties;

    /**
     * 通过@ConfigurationProperties与@Bean，将yml配置文件关于druid中的属性配置，转化成bean，并将bean注入到容器中
     * 这里作用是通过autowire作为参数应用到下面的dynamicDataSource()方法中
     */
    @Bean
    @ConfigurationProperties(prefix = "spring.datasource.druid")
    public DataSourceProperties dataSourceProperties() {
        return new DataSourceProperties();
    }

    /**
     * 通过@Bean告知Spring容器，该方法会返回DynamicDataSource对象
     * 通过dynamicDataSource()配置多数据源选择逻辑，主要配置目标数据源和默认数据源
     */
    @Bean
    public DynamicDataSource dynamicDataSource(DataSourceProperties dataSourceProperties) {
        // 实例化自己实现的多数据源，其中实现了获取当前线程数据源名称的方法
        DynamicDataSource dynamicDataSource = new DynamicDataSource();
        // 设置多数据源属性
        dynamicDataSource.setTargetDataSources(getDynamicDataSource());

        // 工厂方法创建Druid数据源
        DruidDataSource defaultDataSource = DynamicDataSourceFactory.buildDruidDataSource(dataSourceProperties);
        // 设置默认数据源属性
        dynamicDataSource.setDefaultTargetDataSource(defaultDataSource);

        return dynamicDataSource;
    }

    private Map<Object, Object> getDynamicDataSource(){
        Map<String, DataSourceProperties> dataSourcePropertiesMap = properties.getDatasource();
        Map<Object, Object> targetDataSources = new HashMap<>(dataSourcePropertiesMap.size());
        dataSourcePropertiesMap.forEach((k, v) -> {
            DruidDataSource druidDataSource = DynamicDataSourceFactory.buildDruidDataSource(v);
            targetDataSources.put(k, druidDataSource);
        });

        return targetDataSources;
    }

}

5.8. Multi-data source factory class (DynamicDataSourceFactory)

 // 这里访问权限是包私有
class DynamicDataSourceFactory {

    static DruidDataSource buildDruidDataSource(DataSourceProperties properties) {
        DruidDataSource druidDataSource = new DruidDataSource();
        druidDataSource.setDriverClassName(properties.getDriverClassName());
        druidDataSource.setUrl(properties.getUrl());
        druidDataSource.setUsername(properties.getUsername());
        druidDataSource.setPassword(properties.getPassword());

        druidDataSource.setInitialSize(properties.getInitialSize());
        druidDataSource.setMaxActive(properties.getMaxActive());
        druidDataSource.setMinIdle(properties.getMinIdle());
        druidDataSource.setMaxWait(properties.getMaxWait());
        druidDataSource.setTimeBetweenEvictionRunsMillis(properties.getTimeBetweenEvictionRunsMillis());
        druidDataSource.setMinEvictableIdleTimeMillis(properties.getMinEvictableIdleTimeMillis());
        druidDataSource.setMaxEvictableIdleTimeMillis(properties.getMaxEvictableIdleTimeMillis());
        druidDataSource.setValidationQuery(properties.getValidationQuery());
        druidDataSource.setValidationQueryTimeout(properties.getValidationQueryTimeout());
        druidDataSource.setTestOnBorrow(properties.isTestOnBorrow());
        druidDataSource.setTestOnReturn(properties.isTestOnReturn());
        druidDataSource.setPoolPreparedStatements(properties.isPoolPreparedStatements());
        druidDataSource.setMaxOpenPreparedStatements(properties.getMaxOpenPreparedStatements());
        druidDataSource.setSharePreparedStatements(properties.isSharePreparedStatements());

        try {
            druidDataSource.setFilters(properties.getFilters());
            druidDataSource.init();
        } catch (SQLException e) {
            e.printStackTrace();
        }
        return druidDataSource;
    }
}

5.9, data source properties class (DataSourceProperties)

 public class DataSourceProperties {
    /**
     * 可动态配置的数据库连接属性
     */
    private String driverClassName;
    private String url;
    private String username;
    private String password;

    /**
     * Druid默认参数
     */
    private int initialSize = 2;
    private int maxActive = 10;
    private int minIdle = -1;
    private long maxWait = 60 * 1000L;
    private long timeBetweenEvictionRunsMillis = 60 * 1000L;
    private long minEvictableIdleTimeMillis = 1000L * 60L * 30L;
    private long maxEvictableIdleTimeMillis = 1000L * 60L * 60L * 7;
    private String validationQuery = "select 1";
    private int validationQueryTimeout = -1;
    private boolean testOnBorrow = false;
    private boolean testOnReturn = false;
    private boolean testWhileIdle = true;
    private boolean poolPreparedStatements = false;
    private int maxOpenPreparedStatements = -1;
    private boolean sharePreparedStatements = false;
    private String filters = "stat,wall";

    public String getDriverClassName() {
        return driverClassName;
    }

    public void setDriverClassName(String driverClassName) {
        this.driverClassName = driverClassName;
    }

    public String getUrl() {
        return url;
    }

    public void setUrl(String url) {
        this.url = url;
    }

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public String getPassword() {
        return password;
    }

    public void setPassword(String password) {
        this.password = password;
    }

    public int getInitialSize() {
        return initialSize;
    }

    public void setInitialSize(int initialSize) {
        this.initialSize = initialSize;
    }

    public int getMaxActive() {
        return maxActive;
    }

    public void setMaxActive(int maxActive) {
        this.maxActive = maxActive;
    }

    public int getMinIdle() {
        return minIdle;
    }

    public void setMinIdle(int minIdle) {
        this.minIdle = minIdle;
    }

    public long getMaxWait() {
        return maxWait;
    }

    public void setMaxWait(long maxWait) {
        this.maxWait = maxWait;
    }

    public long getTimeBetweenEvictionRunsMillis() {
        return timeBetweenEvictionRunsMillis;
    }

    public void setTimeBetweenEvictionRunsMillis(long timeBetweenEvictionRunsMillis) {
        this.timeBetweenEvictionRunsMillis = timeBetweenEvictionRunsMillis;
    }

    public long getMinEvictableIdleTimeMillis() {
        return minEvictableIdleTimeMillis;
    }

    public void setMinEvictableIdleTimeMillis(long minEvictableIdleTimeMillis) {
        this.minEvictableIdleTimeMillis = minEvictableIdleTimeMillis;
    }

    public long getMaxEvictableIdleTimeMillis() {
        return maxEvictableIdleTimeMillis;
    }

    public void setMaxEvictableIdleTimeMillis(long maxEvictableIdleTimeMillis) {
        this.maxEvictableIdleTimeMillis = maxEvictableIdleTimeMillis;
    }

    public String getValidationQuery() {
        return validationQuery;
    }

    public void setValidationQuery(String validationQuery) {
        this.validationQuery = validationQuery;
    }

    public int getValidationQueryTimeout() {
        return validationQueryTimeout;
    }

    public void setValidationQueryTimeout(int validationQueryTimeout) {
        this.validationQueryTimeout = validationQueryTimeout;
    }

    public boolean isTestOnBorrow() {
        return testOnBorrow;
    }

    public void setTestOnBorrow(boolean testOnBorrow) {
        this.testOnBorrow = testOnBorrow;
    }

    public boolean isTestOnReturn() {
        return testOnReturn;
    }

    public void setTestOnReturn(boolean testOnReturn) {
        this.testOnReturn = testOnReturn;
    }

    public boolean isTestWhileIdle() {
        return testWhileIdle;
    }

    public void setTestWhileIdle(boolean testWhileIdle) {
        this.testWhileIdle = testWhileIdle;
    }

    public boolean isPoolPreparedStatements() {
        return poolPreparedStatements;
    }

    public void setPoolPreparedStatements(boolean poolPreparedStatements) {
        this.poolPreparedStatements = poolPreparedStatements;
    }

    public int getMaxOpenPreparedStatements() {
        return maxOpenPreparedStatements;
    }

    public void setMaxOpenPreparedStatements(int maxOpenPreparedStatements) {
        this.maxOpenPreparedStatements = maxOpenPreparedStatements;
    }

    public boolean isSharePreparedStatements() {
        return sharePreparedStatements;
    }

    public void setSharePreparedStatements(boolean sharePreparedStatements) {
        this.sharePreparedStatements = sharePreparedStatements;
    }

    public String getFilters() {
        return filters;
    }

    public void setFilters(String filters) {
        this.filters = filters;
    }
}

5.10. Multiple data source properties class (DynamicDataSourceProperties)

 /**
 * 通过@ConfigurationProperties指定读取yml的前缀关键字
 * 配合setDatasource()，即读取dynamic.datasource下的配置，将配置属性转化成bean
 * 容器执行顺序是，在bean被实例化后，会调用后置处理，递归的查找属性，通过反射注入值
 *
 * 由于该类只在DynamicDataSourceConfig类中使用，没有其它地方用到，所以没有使用@Component
 * 而是在DynamicDataSourceConfig类中用@EnableConfigurationProperties定义为bean
 */
@ConfigurationProperties(prefix = "dynamic")
public class DynamicDataSourceProperties {
    private Map<String, DataSourceProperties> datasource = new LinkedHashMap<>();

    public Map<String, DataSourceProperties> getDatasource() {
        return datasource;
    }

    public void setDatasource(Map<String, DataSourceProperties> datasource) {
        this.datasource = datasource;
    }
}

6. Finally

The above code has been submitted to the open source project, and the corresponding project module is hei-dynamic-datasource
If you need it, you can click the link below to clone the code locally.
Multiple data source Gitee addresses