头图

MatrixOne从入门到实践——MO-Tester

从 0.5.0 版本开始,MatrixOne 引入了一个自动测试框架 MO-Tester

MO-Tester 测试框架,也可以称作为测试器,是通过 SQL 测试 MatrixOne 或其他数据库功能的。

简介

MO-Tester 是基于 Java 语言进行开发,用于 MatrixOne 的测试套件。MO-Tester 构建了一整套完整的工具链来进行 SQL 自动测试。它包含测试用例和运行结果。MO-Tester 启动后,MO-Tester 将使用 MatrixOne 运行所有 SQL 测试用例,并将所有输出 SQL 测试结果与预期结果进行比较。所有案例的结果无论成功或者失败,都将记录在报告中。

MO-Tester 相关用例、结果和报告的链接如下:

测试用例和测试结果一一对应。如需添加新的测试用例和测试结果请进入右侧所示 MatrixOne 仓库路径中进行添加:https://github.com/matrixorig...

MO-Tester 测试用例如下表所示:

测试用例描述
Benchmark/TPCHDDL and 22 Queries of TPCH Benchmark
DatabaseDDL Statements, creation/drop databases
TableDDL Statements, creation/drop tables
DMLDML Statements, including insert, select, show statements
dtypeData Types and type conversion test cases
ExpressionCase when, With(CTE), Temporal Interval
FunctionAggregate function, built-in function
ExplainExplain statement
JoinJoin statement, including Left/Right/Inner/Outer/Natural Join
OperatorIncluding +,-,*,/,MOD,%,=, >, <, IS, LIKE etc
SubqueryIncluding Select/From/Where subquery
TransactionTest of isolation level, atomicity

使用MO-Tester

环境准备

  • 安装jdk 8

    # 下载jdk压缩包
    下载地址:http://www.oracle.com/technetwork/java/javase/downloads/index.html 
    下载版本为java 8 的:
    jdk-8u131-linux-x64.tar.gz
    # 解压
    tar -zxf jdk-8u131-linux-x64.tar.gz
    # 解压后 的目录假设为
    /home/java/jdk1.8.0_131
    # 编辑/etc/profile,文件的末尾添加
    export JAVA_HOME=/home/java/jdk1.8.0_131
    export PATH=$PATH:$JAVA_HOME/bin
    # 刷新profile 
    source /etc/profile
    # 验证 jdk
    [root@motest java]# java -version
    java version "1.8.0_131"
    Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
    Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
  • 启动 MatrixOne

    请参考[MatrixOne部署]()

  • 克隆 MO-Tester 仓库

    git clone https://github.com/matrixorigin/mo-tester.git

配置 MO-Tester

MO-tester 基于 Java 语言进行开发,因此 Mo-tester 所依赖的 Java 数据库连接(JDBC,Java Database Connectivity) 驱动程序需要配置 mo.yml 文件里的参数信息:进入到 mo-tester 本地仓库,打开 mo.yml 文件,配置服务器地址、默认的数据库名称、用户名和密码等。

以下是本地独立版本 MatrixOne 的默认示例。


#jdbc
jdbc:
  driver: "com.mysql.cj.jdbc.Driver"
  server:
  - addr: "127.0.0.1:6001"
  #- addr: "127.0.0.1:3306"
  #- addr: "127.0.0.1:3306"
  database:
    default: ""
  paremeter:
    characterSetResults: "utf8"
    continueBatchOnError: "false"
    useServerPrepStmts: "true"
    alwaysSendSetIsolation: "false"
    useLocalSessionState: "true"
    zeroDateTimeBehavior: "CONVERT_TO_NULL"
    failoverReadOnly: "false"
    serverTimezone: "Asia/Shanghai"
    socketTimeout: 30000
#users
user:
  name: "dump"
  password: "111"

运行MO-Tester

运行以下所示命令行,SQL 所有测试用例将自动运行,并将报告和错误消息生成至 report/report.txtreport/error.txt 文件中。

./run.sh

如果你想调整测试范围,你可以修改 run.yml 文件中的 path 参数。或者,在执行 run.sh 命令时,你也可以指定一些参数,参数解释如下:

参数参数定义
-p设置由 MO-tester 执行的测试用例的路径。默认值可以参见 run.yml 文件中 path 的配置参数
-m设置 MO-tester 测试的方法,即直接运行或者生成新的测试结果。默认值可以参见 run.yaml 文件中 method 的配置参数
-t设置 MO-tester 执行 SQL 命令的格式类型。默认值可以参见 run.yml 文件中 type 的配置参数。
-r设置测试用例应该达到的成功率。默认值可以参见 run.yml 文件中 rate 的配置参数。
-i设置包含列表,只有路径中名称包含其中一个列表的脚本文件将被执行,如果有多个,如果没有指定,用','分隔,指的是包含的所有情况set the including list, and only script files in the path whose name contains one of the lists will be executed, if more than one, separated by ,, if not specified, refers to all cases included
-e设置排除列表,如果路径下的脚本文件的名称包含一个排除列表,则不会被执行,如果有多个,用','分隔,如果没有指定,表示不排除任何情况set the excluding list, and script files in the path whose name contains one of the lists will not be executed, if more than one, separated by ,, if not specified, refers to none of the cases excluded
-g表示带有[-- @bvt:issue#{issueNO.}]标志的 SQL 命令将不会被执行,该标志以 [-- @bvt:issue#{issueNO.}]开始,以 [-- @bvt:issue]结束。例如, -- @bvt:issue#3236 select date_add("1997-12-31 23:59:59",INTERVAL "-10000:1" HOUR_MINUTE); select date_add("1997-12-31 23:59:59",INTERVAL "-100 1" YEAR_MONTH); -- @bvt:issue 这两个 SQL 命令与问题 #3236 相关联,它们将不会在 MO-tester 测试中执行,直到问题 #3236 修复后标签移除才可以在测试中执行。
-n表示在比较结果时将忽略结果集的元数据

run.yml:

#test scritps path
path: "cases/"

#method
#run: means execute test scripts configured by para[path] completely,and generate the test reprot,execution logs
#debug: means execute  test scripts configured by para[path] completely, only print the result to console,do not generate the test reprot,execution logs
#genrs: means generate the expected result for the test scripts only
method: "run"

#rate: means the execution success rate,if the actual rate is less than this,the programe will exit with status 1
rate: 100

示例:

./run.sh -p case -m run -t script -r 100 -i select,subquery -e substring -g

使用MO-Tester完成TPCH

注意: 这里的tpch只是一个测试用例,仅用来测试数据库能够跑通tpch的22条查询语句,包含的数据条数较少。

  • 获取MatrixOne源码

    git clone https://github.com/matrixorigin/matrixone.git
  • 复制test目录到MO-Tester

    cp -r matrixone/test/cases/benchmark /home/motest/mo-tester/cases
    # 复制后,可以在MO-Tester中看到相关case 如下所示:
    root@motest cases]# pwd
    /home/motest/mo-tester/cases
    [root@motest cases]# cd benchmark/
    [root@motest benchmark]# ll
    total 0
    drwxr-xr-x. 6 root root 71 Oct  1 09:06 tpch
    [root@motest benchmark]# pwd
    /home/motest/mo-tester/cases/benchmark

    同理,将MatrixOne源码目录中test下的result也复制过来

    cp -r matrixone/test/result/benchmark /home/motest/mo-tester/result
  • 运行MO-Tester进行TPCH测试

    ./run.sh -p ./cases/benchmark/tpch -m run -t script -r 100

    最终结果显示:

    The path of the cases that need to be executed by mo-tester  : ./cases/benchmark/tpch
    The method that mo-tester will run with :run
    The type of the format that mo-tester execute the sqlcommand in : script
    The success rate that test cases should reach : 100
    2022-10-01 09:18:23 INFO  Tester:134 - The method is [run],now start to run the scripts in the path[./cases/benchmark/tpch].
    2022-10-01 09:18:23 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/01_DDL/01_create_table.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:23 INFO  Executor:186 - The script file[./cases/benchmark/tpch/01_DDL/01_create_table.sql] has been executed, and cost: 0.02s, total:11, success:11, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:23 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:23 INFO  Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] has been executed, and cost: 0.032s, total:16, success:16, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:23 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:25 INFO  Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] has been executed, and cost: 1.584s, total:602, success:602, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:25 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:25 INFO  Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] has been executed, and cost: 0.008s, total:4, success:4, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:25 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:25 INFO  Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] has been executed, and cost: 0.288s, total:151, success:151, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:25 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:25 INFO  Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] has been executed, and cost: 0.031s, total:21, success:21, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:25 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] has been executed, and cost: 0.114s, total:81, success:81, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] has been executed, and cost: 0.004s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] has been executed, and cost: 0.004s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/10_select_count.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/10_select_count.sql] has been executed, and cost: 0.016s, total:8, success:8, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q1.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q1.sql] has been executed, and cost: 0.028s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q10.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q10.sql] has been executed, and cost: 0.007s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q11.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q11.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q12.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q12.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q13.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q13.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q14.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q14.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q15.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q15.sql] has been executed, and cost: 0.004s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q16.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q16.sql] has been executed, and cost: 0.008s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q17.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q17.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q18.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q18.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q19.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q19.sql] has been executed, and cost: 0.01s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q2.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q2.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q20.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q20.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q21.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q21.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q22.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q22.sql] has been executed, and cost: 0.007s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q3.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q3.sql] has been executed, and cost: 0.008s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q4.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q4.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q5.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q5.sql] has been executed, and cost: 0.007s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q6.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q6.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q7.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q7.sql] has been executed, and cost: 0.011s, total:3, success:3, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q8.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q8.sql] has been executed, and cost: 0.008s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q9.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q9.sql] has been executed, and cost: 0.011s, total:2, success:2, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Executor:25 - Start to execute the script file[./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] now, and it will take a few moment,pleas wait......
    2022-10-01 09:18:26 INFO  Executor:186 - The script file[./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] has been executed, and cost: 0.015s, total:9, success:9, failed:0, ignored:0, abnoraml:0
    2022-10-01 09:18:26 INFO  Tester:136 - All the scripts in the path[./cases/benchmark/tpch] have been excuted.Now start to create the test report.
    2022-10-01 09:18:26 INFO  TestReport:49 - [SUMMARY] COST : 1s, TOTAL :952, SUCCESS : 952, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/01_DDL/01_create_table.sql] COST : 0.02s, TOTAL :11, SUCCESS :11, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] COST : 0.032s, TOTAL :16, SUCCESS :16, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] COST : 1.584s, TOTAL :602, SUCCESS :602, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] COST : 0.008s, TOTAL :4, SUCCESS :4, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] COST : 0.288s, TOTAL :151, SUCCESS :151, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] COST : 0.031s, TOTAL :21, SUCCESS :21, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] COST : 0.114s, TOTAL :81, SUCCESS :81, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/02_LOAD/10_select_count.sql] COST : 0.016s, TOTAL :8, SUCCESS :8, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q1.sql] COST : 0.028s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q10.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q11.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q12.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q13.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q14.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q15.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q16.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q17.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q18.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q19.sql] COST : 0.01s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q2.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q20.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q21.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q22.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q3.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q4.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q5.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q6.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q7.sql] COST : 0.011s, TOTAL :3, SUCCESS :3, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q8.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q9.sql] COST : 0.011s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  TestReport:54 - [./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] COST : 0.015s, TOTAL :9, SUCCESS :9, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    2022-10-01 09:18:26 INFO  Tester:138 - The test report has been generated in files[report.txt,report.xml].
    2022-10-01 09:18:26 INFO  Tester:144 - The execution success rate is 100%, and not less than config value 100%,this test succeed.

    我们可以看一下report.txt 观察测试用例结果

    cd ./report/ 
    [root@motest report]# cat report.txt 
    [SUMMARY] COST : 1s, TOTAL :952, SUCCESS : 952, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/01_DDL/01_create_table.sql] COST : 0.02s, TOTAL :11, SUCCESS :11, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] COST : 0.032s, TOTAL :16, SUCCESS :16, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] COST : 1.584s, TOTAL :602, SUCCESS :602, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] COST : 0.008s, TOTAL :4, SUCCESS :4, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] COST : 0.288s, TOTAL :151, SUCCESS :151, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] COST : 0.031s, TOTAL :21, SUCCESS :21, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] COST : 0.114s, TOTAL :81, SUCCESS :81, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/02_LOAD/10_select_count.sql] COST : 0.016s, TOTAL :8, SUCCESS :8, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q1.sql] COST : 0.028s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q10.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q11.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q12.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q13.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q14.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q15.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q16.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q17.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q18.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q19.sql] COST : 0.01s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q2.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q20.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q21.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q22.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q3.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q4.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q5.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q6.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q7.sql] COST : 0.011s, TOTAL :3, SUCCESS :3, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q8.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/03_QUERIES/q9.sql] COST : 0.011s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%
    [./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] COST : 0.015s, TOTAL :9, SUCCESS :9, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%

    同理,我们如果需要执行其他的测试用例,只需要将对应的cases复制过来,运行时指定对应的路径即可


MatrixOrigin
4 声望2 粉丝

新一代超融合异构数据库