Automation of Test Development - Design of Automated Test Framework

Today, I will introduce how to design an automated testing framework. The framework mentioned here is based on some mainstream class libraries, frameworks or tools, self-developed and suitable for the company's automated testing assets.

Today, there are many UnitTest test frameworks that already provide data-driven, user concurrency, assertion, reporting and other excellent features, which can be used to test functions, performance, interfaces, etc. to realise.

Here is the hierarchical structure of our proposed automated testing framework, which will be introduced one by one.

test case script

The automated test of the business is realized here, and the script at this level has the most direct correspondence with the manual test case.

public business script

Some extracted functions, class libraries or scripts that can be reused at the business level. For example, the online shopping system login, add shopping cart, order, payment and other common business operations. It can also include the implementation of some application system level or Knowledge Base with security restrictions, such as the command line tool used by the author to test the electronic payment system to send money to the global central bank, which is developed by the company's core personnel and implemented in security in a highly isolated environment.

Script parsing

It is used to parse higher-form test scripts such as keyword-driven (Data Driver), natural language understanding (NLU), etc., and convert them into executable code of a certain programming language. For example, the process of parsing the script of the QTP keyword view into VBScript code, and converting the script of RobotFramework custom keyword to Python3 code.

data management

Implement data management and data-driven automated testing. For example, everyone is used to maintaining two-dimensional tabular data in Excel, reading it in the script and then referencing it iteratively; it is also possible to write a DataProvider for the data file or ZenData data interface similar to the example in the previous article, and then use Java in the script. In the form of annotation, the data is "feed" to the business test method in a loop and one by one.

Execute scheduling

In a distributed testing framework, it is used to control what kind of environment, such as system, device or node, the test execution takes place. It can be similar to the Jenkins agent to execute tests on different remote machines; it can also be similar to ZenTao's ZAgent open source project, to execute tests on virtual machines, containers or mobile devices with specified characteristics.

result assertion

Usually, the assertion and log features of some mainstream test frameworks are encapsulated to make them more convenient to use and more suitable for the company's business. For example, while the use case is checking the verification point, it prints, stores it in the database, and pushes a more detailed message in a specified format to the log system, so as to facilitate subsequent statistical analysis.

log message

In addition to the information related to the above assertion, the expected result, actual result, pass or not output by the verification point check, it can also include: in the UI-based automated test, the control positioning result and operation result of each step; the protocol-based interface In the test, the message and status of the request and response, and so on, all kinds of logs that are useful for debugging and analysis.

Statistical Analysis

Collect and analyze the output results of test steps, checkpoints, use cases, test sets, execution plans and other levels, and generate a complete automated test statistical report.

test asset management

Centrally manage different types of test assets, including physical servers, virtual test machines, mobile phones or embedded devices, or even speech recognition (ASR) corpora in AI testing, natural language understanding (NLU) sayings, face recognition galleries.

Basic tool class library

Some commonly used, specific programming language-level class libraries. Such as string processing, file parsing, protocol request, message sending, etc.

Basic Framework and Tools

Based on third-party automated testing class libraries, frameworks or tools to implement functions such as test set organization, (distributed) execution calls, data providers, etc. Excellent open source and commercial projects such as JUnit, TestNG, Selenium, Appium, JMeter, QTP, etc. mentioned above. Most of the time, we don't need to repeat the basic wheels.

The modules and functions described above can be regarded as covering all aspects of an automated test platform. In actual work, everyone can practice as needed and gradually, and the most suitable one is always the best.

Thematic Catalog