MyBatis source code study notes (1) first encounter

The opening of this series of articles "When we talk about looking at the source code, what are we looking at" started in October last year, and today I started to fill in the pits of this series. MyBatis is the first ORM framework I came into contact with, and it is also the ORM framework I am most familiar with. I have been using it for a long time. Today, I will try to see the internal structure of MyBatis. If you don't know MyBatis yet, you can read "Pretending to be Xiaobai's Re-learning MyBatis (1)" first.

So how to look at the source code?

I downloaded the source code of MyBatis and looked at it aimlessly? Will this be lost in the source code? I remember that when I first arrived at my current company, I looked at the code one by one, and then I felt a headache, and I didn't understand what to do in the end. After reflecting for a while, you should actually pay attention to the macro process, that is, what function does this code achieve. These codes are all to achieve this function. You don’t need to look at each method line by line, but look at the method as a unit. This method is from the whole Let's see what kind of things have been done, and don't have to pay too much attention to the internal implementation details. Looking at the code in this way, you probably have a good idea. Also in MyBatis, this is also the first code I have studied carefully, so in the first article of the MyBatis series, we first look at its implementation from a macro perspective, and slowly fill in its details in the later process. The main line of this article is how the addition, deletion, modification, and query statements we wrote in xml are executed.

After referring to a lot of MyBatis source code information, the overall architecture of MyBatis can be divided into three layers:

• Interface layer: SqlSession is the core interface that we usually interact with MyBatis (including the SqlSessionTemplte used by the subsequent integration of SpringFramework)
• Core layer: The method of SqlSession execution, the bottom layer needs to go through configuration file parsing, SQL parsing, parameter mapping, SQL execution, result set mapping when executing SQL, and there are extension plug-ins interspersed in it.
• Support layer: The function realization of the core layer is based on the coordination of various modules at the bottom layer.

Build the MyBatis environment

The environment for building MyBatis has been discussed in "Pretending to be Xiaobai's Heavy Learning MyBatis (1)", here is just a brief talk:

• Introduce Maven dependencies

 <dependency>
  <groupId>org.mybatis</groupId>
  <artifactId>mybatis</artifactId>
  <version>3.5.6</version>
 </dependency>
  <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.47</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid</artifactId>
            <version>1.2.5</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.30</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.30</version>
        </dependency>

• then a table

 CREATE TABLE `student`  (
  `id` int(11) NOT NULL COMMENT '唯一标识',
  `name` varchar(255) ,
  `number` varchar(255) ,
  `money` int(255) NULL DEFAULT NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4;

• Come to a MyBatis configuration file

 <?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration
        PUBLIC "-//mybatis.org//DTD Config 3.0//EN"
        "http://mybatis.org/dtd/mybatis-3-config.dtd">
<configuration>
    <!--加载配置文件-->
    <properties resource="jdbc.properties"/>
    <!--指定默认环境, 一般情况下,我们有三套环境,dev 开发 ,uat 测试 ,prod 生产 -->
    <environments default="development">
        <environment id="development">
            <!-- 设置事务管理器的管理方式  -->
            <transactionManager type="JDBC"/>
            <!-- 设置数据源连接的关联方式为数据池  -->
            <dataSource type="POOLED">
                <property name="driver" value="com.mysql.jdbc.Driver"/>
                <property name="url" value="jdbc:mysql://localhost:3306/studydatabase?characterEncoding=utf-8"/>
                <property name="username" value="root"/>
                <property name="password" value="root"/>
            </dataSource>
        </environment>
    </environments>
    <mappers>
        <!--设置扫描的xml,org/example/mybatis是包的全类名，StudentMapper.xml会讲-->

    <mappers>
        <!--设置扫描的xml,org/example/mybatis是包的全类名，这个BlogMapper.xml会讲
         <package name = "org.example.mybatis"/> <!-- 包下批量引入 单个注册 -->
          <mapper resource="org/example/mybatis/StudentMapper.xml"/> 
    </mappers>
    </mappers>
</configuration>

• Come to a Student class

 public class Student {
    private Long id;
    private String name;
    private String number;
    private String money;
    // 省略get set 函数
}

• Come to a Mapper.xml

 <?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper
        PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN"
        "http://mybatis.org/dtd/mybatis-3-mapper.dtd">

<mapper namespace = "org.example.mybatis.StudentMapper">
    <select id = "selectStudent" resultType = "org.example.mybatis.Student">
        SELECT * FROM STUDENT
    </select>
</mapper>

• come up with an interface

 public interface StudentMapper {
    List<Student> selectStudent();
}

• log configuration file

 log4j.rootCategory=debug, CONSOLE

# Set the enterprise logger category to FATAL and its only appender to CONSOLE.
log4j.logger.org.apache.axis.enterprise=FATAL, CONSOLE

# CONSOLE is set to be a ConsoleAppender using a PatternLayout.
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.Encoding=UTF-8
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} %-6r [%15.15t] %-5p %30.30

• Start your inquiry journey

 public class MyBatisDemo {
    public static void main(String[] args) throws Exception {
        Reader reader = Resources.getResourceAsReader("conf.xml");
        SqlSessionFactory sqlSessionFactory = new SqlSessionFactoryBuilder().build(reader);
        SqlSession sqlSession = sqlSessionFactory.openSession();
        StudentMapper studentMapper = sqlSession.getMapper(StudentMapper.class);
        List<Student> studentList = studentMapper.selectStudent();
        studentList.forEach(System.out::println);
    }
}

After execution, you can see the following output in the console:

Let's start with the SQL execution journey

Analysis of the execution process

The above execution process can be roughly divided into three steps:

• Parse the configuration file and build the SqlSessionFactory
• Get SqlSession through SqlSessionFactory, and then get proxy class
• Execute the method of the proxy class

Parse the configuration file

Parsing configuration files is performed through the build method of SqlSessionFactoryBuilder, which has several overloads:

Reader points to the conf file, environment is the environment, and properties are used for conf to get values ​​from other properties. Our configuration file is an xml, so XmlConfigBuilder is ultimately an encapsulation of the configuration file. Here we don't pay attention to how the XmlBuilder is constructed. Let's look down. After building the Xml object, call the parse method to convert it into the MyBatis Configuration object:

 // parseConfiguration 这个方法用于取xml标签的值并将其设置到Configuration上
public Configuration parse() {
    if (parsed) {
      throw new BuilderException("Each XMLConfigBuilder can only be used once.");
    }
    parsed = true;
    parseConfiguration(parser.evalNode("/configuration"));
    return configuration;
  }

 // 取标签的过程,XML->Configuration
private void parseConfiguration(XNode root) {
    try {
      // issue #117 read properties first
      propertiesElement(root.evalNode("properties"));
      Properties settings = settingsAsProperties(root.evalNode("settings"));
      loadCustomVfs(settings);
      loadCustomLogImpl(settings);
      typeAliasesElement(root.evalNode("typeAliases"));
      pluginElement(root.evalNode("plugins"));
      objectFactoryElement(root.evalNode("objectFactory"));
      objectWrapperFactoryElement(root.evalNode("objectWrapperFactory"));
      reflectorFactoryElement(root.evalNode("reflectorFactory"));
      settingsElement(settings); 
      // read it after objectFactory and objectWrapperFactory issue #631
      environmentsElement(root.evalNode("environments"));
      databaseIdProviderElement(root.evalNode("databaseIdProvider"));
      typeHandlerElement(root.evalNode("typeHandlers"));
      mapperElement(root.evalNode("mappers")); // 获取mapper方法,
    } catch (Exception e) {
      throw new BuilderException("Error parsing SQL Mapper Configuration. Cause: " + e, e);
    }
  }

• Configuration overview

• mapperElement

​ Note that the theme of our article is to focus on how the sql we write in the xml tag is executed, so here we focus on the mapperElement method of parseConfiguration. From the name, we roughly infer that this method loads the mapper.xml file. Let's click in and take a look:

 // parent 是mappers标签
private void mapperElement(XNode parent) throws Exception {
    if (parent != null) {
      for (XNode child : parent.getChildren()) { // 遍历mappers下面的结点
        if ("package".equals(child.getName())) {  // 如果是package标签则走批量引入
          String mapperPackage = child.getStringAttribute("name");
          configuration.addMappers(mapperPackage);
        } else {
          String resource = child.getStringAttribute("resource"); // 我们本次看单个引入的方式
          String url = child.getStringAttribute("url");
          String mapperClass = child.getStringAttribute("class");
          if (resource != null && url == null && mapperClass == null) {
            ErrorContext.instance().resource(resource);
            InputStream inputStream = Resources.getResourceAsStream(resource); // 加载指定文件夹下的XML
            XMLMapperBuilder mapperParser = new XMLMapperBuilder(inputStream, configuration, resource, configuration.getSqlFragments()); 
            mapperParser.parse(); // 将mapper 中的标签值映射成MyBatis的对象
          } else if (resource == null && url != null && mapperClass == null) {
            ErrorContext.instance().resource(url);
            InputStream inputStream = Resources.getUrlAsStream(url);
            XMLMapperBuilder mapperParser = new XMLMapperBuilder(inputStream, configuration, url, configuration.getSqlFragments());
            mapperParser.parse(); // 我们大致看下parse方法的实现
          } else if (resource == null && url == null && mapperClass != null) {
            Class<?> mapperInterface = Resources.classForName(mapperClass);
            configuration.addMapper(mapperInterface);
          } else {
            throw new BuilderException("A mapper element may only specify a url, resource or class, but not more than one.");
          }
        }
      }
    }
  }

The parent parameter is the mappers label, we can verify this by debugging:

 public void parse() {
  if (!configuration.isResourceLoaded(resource)) {
    configurationElement(parser.evalNode("/mapper"));
    configuration.addLoadedResource(resource);
    bindMapperForNamespace();
  }

  parsePendingResultMaps();
  parsePendingCacheRefs();
  parsePendingStatements();
}

In the introduction, Xi'an judged whether the xml has been loaded, and then parsed the tags such as additions, deletions, changes, and checks under the mapper tag. We can see this in the configurationElement.

 private void configurationElement(XNode context) {
    try {
      String namespace = context.getStringAttribute("namespace");
      if (namespace == null || namespace.isEmpty()) {
        throw new BuilderException("Mapper's namespace cannot be empty");
      }
      builderAssistant.setCurrentNamespace(namespace);
      cacheRefElement(context.evalNode("cache-ref"));
      cacheElement(context.evalNode("cache"));
      parameterMapElement(context.evalNodes("/mapper/parameterMap"));
      resultMapElements(context.evalNodes("/mapper/resultMap"));
      sqlElement(context.evalNodes("/mapper/sql"));
      buildStatementFromContext(context.evalNodes("select|insert|update|delete"));  //该方法解析标签
    } catch (Exception e) {
      throw new BuilderException("Error parsing Mapper XML. The XML location is '" + resource + "'. Cause: " + e, e);
    }
  }

 private void buildStatementFromContext(List<XNode> list) {
  if (configuration.getDatabaseId() != null) { // dataBaseId 用于指明该标签在哪个数据库下执行
    buildStatementFromContext(list, configuration.getDatabaseId());
  }
  buildStatementFromContext(list, null);
}

The parseStatementNode method is relatively long. In the end, it parses the attributes of select, insert, update, and delete of Mapper.xml, and passes the parsed attributes to the builderAssistant.addMappedStatement() method. This method has slightly more parameters. Let's take a screenshot:

At this point, we basically end the process of building the configuration. We can think that at this step, the Mybatis configuration file and Mapper.xml have been basically parsed.

Get SqlSession object

SqlSession is an interface with two main implementation classes:

What we built in the first step is actually the DefaultSqlSessionFactory:

 public SqlSessionFactory build(Configuration config) {
  return new DefaultSqlSessionFactory(config);
}

In fact, openSession is also executed by DefaultSqlSessionFactory. Let's take a look at what is roughly done in the process of openSession:

 @Override
public SqlSession openSession() {
  return openSessionFromDataSource(configuration.getDefaultExecutorType(), null, false);
}

Pay attention to this getDefaultExecutorType, this fact is the SQL executor of the core layer in the MyBatis layer, let's look down at openSessionFromDataSource:

 //  level 隔离级别, autoCommit 是否自动提交
//  ExecutorType 是一个枚举值: SIMPLE、REUSE、BATCH
private SqlSession openSessionFromDataSource(ExecutorType execType, TransactionIsolationLevel level, boolean autoCommit) {
    Transaction tx = null;
    try {
      final Environment environment = configuration.getEnvironment();
      final TransactionFactory transactionFactory = getTransactionFactoryFromEnvironment(environment);
      tx = transactionFactory.newTransaction(environment.getDataSource(), level, autoCommit);
      // 返回一个执行器,我们看下newExecutor这个方法
      final Executor executor = configuration.newExecutor(tx, execType);
      // 最后构造出来SqlSession  
      return new DefaultSqlSession(configuration, executor, autoCommit);
    } catch (Exception e) {
      closeTransaction(tx); // may have fetched a connection so lets call close()
      throw ExceptionFactory.wrapException("Error opening session.  Cause: " + e, e);
    } finally {
      ErrorContext.instance().reset();
    }
  }

 public Executor newExecutor(Transaction transaction, ExecutorType executorType) {
  executorType = executorType == null ? defaultExecutorType : executorType;
  executorType = executorType == null ? ExecutorType.SIMPLE : executorType;
  Executor executor;
  if (ExecutorType.BATCH == executorType) {
    executor = new BatchExecutor(this, transaction);
  } else if (ExecutorType.REUSE == executorType) {
    executor = new ReuseExecutor(this, transaction);
  } else {
    executor = new SimpleExecutor(this, transaction);
  }
  // 上面是根据executorType生成对应的执行器
  // 如果开启缓存，则将其执行器包装为另一种形式的执行器
  if (cacheEnabled) {
    executor = new CachingExecutor(executor);
  }
  // interceptorChain 是一个拦截器链
  // 将该执行器加入到拦截器链中增强,这事实上是MyBatis的插件开发。
  // 也是装饰器模式的应用,后面会讲。
  executor = (Executor) interceptorChain.pluginAll(executor);
  return executor;
}

Perform CRUD

Then let's see how the methods in our interface are executed,

In fact, when StudentMapper executes the selectStudent method, it should enter the object corresponding to the proxy. We enter the next step. In fact, we enter the invoke method. This invoke method actually rewrites the InvocationHandler method. InvocationHandler is a dynamic proxy interface provided by JDK. The delegated method actually goes to the invoke method, which is implemented as follows:

 public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
    try {
      if (Object.class.equals(method.getDeclaringClass())) {
        return method.invoke(this, args);
      } else {
         // 该方法会缓存该方法,如果该在缓存里面有，则无需再次产生,里面的methodCache是ConcurrentHashMap
         // 最终会返回MapperMethod对象调用invoke方法。
        // 我这里最终的MethodInvoker是PlainMethodInvoker
        return cachedInvoker(method).invoke(proxy, method, args, sqlSession); 
      }
    } catch (Throwable t) {
      throw ExceptionUtil.unwrapThrowable(t);
    }
  }

The final invoke method is shown below:

 @Override
public Object invoke(Object proxy, Method method, Object[] args, SqlSession sqlSession) throws Throwable {
  //这个execute太长,里面根据标签类型,来做下一步的操纵,这里我们放截图 
  return mapperMethod.execute(sqlSession, args);
}

Let's follow the execution of the executeForMany method:

 private <E> Object executeForMany(SqlSession sqlSession, Object[] args) {
  List<E> result;
  Object param = method.convertArgsToSqlCommandParam(args);
   // 默认的分页
    if (method.hasRowBounds()) {
    RowBounds rowBounds = method.extractRowBounds(args);
    result = sqlSession.selectList(command.getName(), param, rowBounds);
  } else {
    // 会走DefaultSqlSession的selectList下面
    result = sqlSession.selectList(command.getName(), param);
  }
  // issue #510 Collections & arrays support
  // 转换结果  
  if (!method.getReturnType().isAssignableFrom(result.getClass())) {
    if (method.getReturnType().isArray()) {
      return convertToArray(result);
    } else {
      return convertToDeclaredCollection(sqlSession.getConfiguration(), result);
    }
  }
  return result;
}

 public <E> List<E> selectList(String statement, Object parameter) {
    return this.selectList(statement, parameter, RowBounds.DEFAULT);
  }

 // 
@Override
  public <E> List<E> selectList(String statement, Object parameter, RowBounds rowBounds) {
    try {
      // 这个statement是方法引用:org.example.mybatis.StudentMapper.selectStudent
      // 通过这个key就可以从configuration获取构建的MappedStatement
      MappedStatement ms = configuration.getMappedStatement(statement);
      // query里面会判断结果是否在缓存里,我们没有引入缓存
      // 最终会走的query中的queryFromDatabase方法。
      //   queryFromDatabase 里面会调用doQuery方法
      return executor.query(ms, wrapCollection(parameter), rowBounds, Executor.NO_RESULT_HANDLER);
    } catch (Exception e) {
      throw ExceptionFactory.wrapException("Error querying database.  Cause: " + e, e);
    } finally {
      ErrorContext.instance().reset();
    }
  }

Here we focus on the doQuery method:

 @Override
  public <E> List<E> doQuery(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, BoundSql boundSql) throws SQLException {
    // 这里我们其实已经可以看到MyBatis已经准备在调用JDBC了
    // Statement 就位于JDBC中
    Statement stmt = null;
    try {
      Configuration configuration = ms.getConfiguration();
      // 根据参数处理标签中的SQL
      StatementHandler handler = configuration.newStatementHandler(wrapper, ms, parameter, rowBounds, resultHandler, boundSql);
      // 产生执行SQL的Statement
      stmt = prepareStatement(handler, ms.getStatementLog());
      // 接着调query方法. 最终会走到PreparedStatementHandler的query方法上  
      return handler.query(stmt, resultHandler);
    } finally {
      closeStatement(stmt);
    }
  }

 // 最终执行SQL
 public <E> List<E> query(Statement statement, ResultHandler resultHandler) throws SQLException {
    PreparedStatement ps = (PreparedStatement) statement;
    ps.execute();
    return resultSetHandler.handleResultSets(ps);
  }

PreparedStatement is JDBC, and it has begun to call JDBC to execute SQL. resultSetHandler is the handler for processing JDBC results.

Here we roughly sort out the Handlers encountered above:

• StatementHandler: statement handler
• ResultSetHandler: result handler, if there is a result handler, there will be a parameter handler
• ParameterHandler: parameter handler,

in conclusion

In MyBatis, the question of how the SQL statement we wrote in the xml file is executed has now been answered:

• The query statements and attributes in xml will be pre-loaded into the Configuration, and there are MappedStatements in the Configuration, which is a Map, and the key is the id of the tag.
• When we execute the corresponding Mapper, we must first execute the acquisition of the Session. In this process, we will pass through the interceptor of MyBatis. We can choose to enhance MyBatis in this process.
• When the method corresponding to the interface is called, the method of the proxy class is actually called. The proxy class will first process the parameters, obtain the MappedStatement according to the method signature, and then convert it to JDBC for processing.

Now we have a general understanding of the execution process that MyBatis already has. Maybe some methods are not too detailed, because talking about those details is not very helpful to the macro execution process.

References

• MyBatis Video Tutorial (Advanced) Video Yanqun https://www.bilibili.com/video/BV1Q4411579K?p=11
• Play MyBatis: In-depth Analysis and Customization https://juejin.cn/book/6944917557878980638