How should performance testing be done?

Three high crowd

As a developer, what we have heard the longest is the three highs in the programming world:

High performance, high concurrency, and high availability.

It sounds very tall, but what about the performance? How should it be assessed?

This time we talk about performance testing and see what is called high performance at the bottom.

This article mainly discusses from the following aspects.

(1) What is the performance test?

(2) Why do we need performance testing?

(3) How to do performance testing?

(4) What are the tools for performance testing?

What is performance testing?

Lao Ma once said that if you want to understand a thing, you must first define it.

Here is a direct reference to the definition in the encyclopedia:

Performance testing is to test various performance indicators of the system by simulating a variety of normal, peak and abnormal load conditions through automated testing tools.

The definition of performance testing is not difficult to understand, often the definition itself explains the role of performance testing.

Why do we need performance testing?

If you are a developer and tester, and you have usually taken over a lot of requirements, you may not have much contact with performance testing.

Every need has a corresponding functional requirement and pulmonary functional requirement.

Functional requirements are the most direct and functional goals that need to be achieved in the product requirements document. Abbreviation, just use it.

Non-functional requirements are much broader. Is the architecture design reasonable? Is it easy to expand later? Is it easy to monitor? Is the code implementation elegant? Are the documentation comments complete?

Just like you wrote about a bird, the bird's head must be used as a propeller to fly, but the architectural design may be unreasonable.

A query function, the user clicks on the query, and the data is returned only after 10S. The function is satisfactory, but the performance is unacceptable.

The online transaction function is usually great in all aspects, and the system is directly paralyzed during the peak holiday season.

How to avoid these problems in production?

This requires that the corresponding performance test be done before going online to avoid problems in reproduction and cause serious production accidents.

The performance should be high, the performance should be hard, and the performance test should be high and hard!

How to do a good performance test

Before doing something, we must first determine our goals.

What is the performance test?

Some are similar to requirements analysis in the development process. Common test indicators are as follows.

Test index

Response time

Response time refers to the time it takes for a request or operation to be sent to feedback, including application server (client) processing time, network transmission time, and database server processing time.

As a user, click on the query on the page, how long it takes to get the result, this is the response time.

Users don't care how many services your backend has passed through. Slowness is the original sin.

For microservice systems, link monitoring is more important. Can help us quickly locate where is slow.

TPS/QPS

TPS (Transaction Per Second) refers to the amount of transactions processed by the system per unit of time (per second).

I think there are many similar concepts on the Internet: click volume/click rate, throughput/throughput rate, PV/UV, and I won’t repeat them here.

Personally, in essence, TPS/QPS is to pressure test the limits of your application. Can the program survive when the number of visits is large?

Two concepts are mainly involved here: high performance and high availability.

We will briefly discuss these two points later.

Test preparation

After clarifying the test indicators, you need to prepare for the test.

Environment preparation: For example, if you want to stress test the database, you need to prepare the database resources corresponding to the configuration.

Script preparation: data initialization script, calling script, etc.

This can be compared to code development in the development process.

ps: Performance stress testing is generally not very common, so the environment preparation process will be relatively long, which requires attention.

testing report

After the test, the results of the test must be given a report.

Does it pass the pressure test requirements?

What is the highest QPS?

In this way, the development can be optimized accordingly based on this report.

How to improve program performance

It is not an exaggeration to write a book about improving performance. Here are some of the most commonly used points:

(1) Slow SQL

If the response time of the general program is longer, you can first look at the slow SQL.

See if you need to add indexes or perform SQL optimization.

(2) Cache

For queries, the most significant performance improvement is the introduction of caching.

Of course, the introduction of caching will complicate the architecture, which must be combined with your actual business.

(3) Hardware upgrade

If the space for program optimization is relatively small, consider upgrading hardware resources.

For example, the server configuration is doubled, and the database configuration is doubled.

What? You said the company has no money to upgrade?

What stress test should I do if I don’t have the money to upgrade?

At this time, the function of the test report is revealed, and the data is used to speak directly.

It is straightforward to say that QPS does not meet the production requirements, and the room for program optimization is very small. It is recommended to upgrade the hardware configuration and how much to upgrade.

To be a man, we must convince people with virtue.

To do a test, you have to use data to speak.

Commonly used performance testing tools

The most commonly used tool for testing is jmeter.

In addition, there are some other tools:

LoadRunner, QALoad, SilkPerformer and Rational Performance Tester.

Here is a brief introduction to a few tools

jmeter

Apache JMeter can be used to test the performance of static and dynamic resources (Web dynamic applications).

It can be used to simulate the load on a server, server group, network, or object to test its strength or analyze the overall performance under different load types.

LoadRunner

Integrate load testing into development tools: IDE, jUnit, nUnit, Jenkins, Selenium and Microsoft Visual Studio.

Starting from version 12.55, you can run your JMeter scripts and integrate JMeter and additional script types in any performance test.

ps: This design concept is very good and can be integrated with mature tools. Standing on the shoulders of giants.

QALoad

QALoad is an automated load testing tool for client/server systems, enterprise resource allocation (ERP) and e-commerce applications.

QALoad can simulate hundreds of users to perform key business concurrently to complete the test of the application, and optimize the system performance according to the problems found to ensure the successful deployment of the application.

ps: I haven't touched this tool.

SilkPerformer

SilkPerformerV allows you to predict the behavior of an enterprise's e-commerce environment before using it—not affected by the scale and complexity of e-commerce applications.

Visual customization, visual content verification under load conditions, real-time performance monitoring and powerful management reports can help you quickly isolate problems. In this way, by minimizing test cycles, optimizing performance, and ensuring scalability, investment is accelerated Market time and guarantee the reliability of the system.

Rational Performance Tester

As part of the DevOps methodology, IBM Rational Performance Tester helps software testing teams to test earlier and more frequently.

It verifies the scalability of Web and server applications, determines the existence and causes of system performance bottlenecks, and reduces load testing.

Your software testing team can quickly perform performance tests and analyze the impact of load on applications.

ps: This tool is provided by IBM, and the quality is trustworthy.

Performance testing tools tailored for development

With so many tools available, I believe that the little friends who have read here have found their favorite testing tools.

Don't worry, the following recommends a performance testing tool for those who are doing java development.

Men have the romance of men, and of course developers must also have the romance of developers.

[Man's Wave.jpg]

junitperf

As a developer, Lao Ma uses junit the most for unit tests.

So I always wanted to find a performance stress test tool based on junit, and I did find it later.

https://github.com/houbb/junitperf is a performance testing framework designed for java developers.

Why use it?

• It fits perfectly with Junit5.
• It is easy to use, convenient for testing and practical in the process of project development.
• Provide expansion, users can customize development.

characteristic

• Support I18N
• Support a variety of report generation methods, support customization
• Junt5 is perfectly supported, easy for Java developers to use

Getting started

Introduced by maven

<dependency>
    <groupId>com.github.houbb</groupId>
    <artifactId>junitperf</artifactId>
    <version>2.0.7</version>
</dependency>

Getting started example

@JunitPerfConfig(duration = 1000)
public void helloTest() throws InterruptedException {
    Thread.sleep(100);
    System.out.println("Hello Junit5");
}

Annotation

@JunitPerfConfig specifies the attribute configuration during testing. (Required field)

Attributes	instruction	Types of	Defaults	Remark
threads	How many threads are used for execution	int	1
warmUp	Preparation time	long	0	Unit: milliseconds
duration	execution time	long	60_000 (1 minute)	Unit: milliseconds
latencyStatistics	Statistical realization	StatisticsCalculator	DefaultStatisticsCalculator
reporter	Report realization	Reporter	ConsoleReporter

Use as follows:

/**
 * 2个线程运行。
 * 准备时间：1000ms
 * 运行时间: 2000ms
 * @throws InterruptedException if any
 */
@JunitPerfConfig(threads = 2, warmUp = 1000, duration = 2000)
public void junitPerfConfigTest() throws InterruptedException {
    System.out.println("junitPerfConfigTest");
    Thread.sleep(200);
}

@JunitPerfRequire specifies the requirements that need to be met during testing. (Optional)

Attributes	instruction	Types of	Defaults	Remark
min	Optimal running time	float	-1	If the fastest running time is higher than this value, it is considered a failure. Unit: milliseconds
max	Average running time	float	-1	If the worst running time is higher than this value, it is regarded as a failure. Unit: milliseconds
average	Average running time	float	-1	If the average running time is higher than this value, it is considered a failure. Unit: milliseconds
timesPerSecond	Minimum number of executions per second	int	0	If it is less than this minimum number of executions, it is considered a failure.
percentiles	Time-consuming restrictions on execution	String[]	{}	percentiles={"20:220", "30:250"}. 20% of data execution time must not exceed 220ms; 30% of data execution time must not exceed 250ms;

Use as follows:

/**
 * 配置：2个线程运行。准备时间：1000ms。运行时间: 2000ms。
 * 要求：最快不可低于 210ms, 最慢不得低于 250ms, 平均不得低于 225ms, 每秒运行次数不得低于 4 次。
 * 20% 的数据不低于 220ms, 50% 的数据不得低于 230ms;
 *
 * @throws InterruptedException if any
 */
@JunitPerfConfig(threads = 2, warmUp = 1000, duration = 2000)
@JunitPerfRequire(min = 210, max = 250, average = 225, timesPerSecond = 4, percentiles = {"20:220", "50:230"})
public void junitPerfConfigTest() throws InterruptedException {
    System.out.println("junitPerfConfigTest");
    Thread.sleep(200);
}

testing report

Corresponding test report generation methods are also diverse, and user customization is also allowed.

Based on the console log:

[INFO] [2020-06-16 20:05:53.618] [c.g.h.j.e.HelloWorldTest.helloTest] - Started at:  2020-06-16 20:05:52.512
[INFO] [2020-06-16 20:05:53.619] [c.g.h.j.e.HelloWorldTest.helloTest] - Invocations:  9
[INFO] [2020-06-16 20:05:53.620] [c.g.h.j.e.HelloWorldTest.helloTest] - Success:  9
[INFO] [2020-06-16 20:05:53.620] [c.g.h.j.e.HelloWorldTest.helloTest] - Errors:  0
[INFO] [2020-06-16 20:05:53.621] [c.g.h.j.e.HelloWorldTest.helloTest] - Thread Count:  1
[INFO] [2020-06-16 20:05:53.623] [c.g.h.j.e.HelloWorldTest.helloTest] - Warm up:  0ms
[INFO] [2020-06-16 20:05:53.623] [c.g.h.j.e.HelloWorldTest.helloTest] - Execution time:  1000ms
[INFO] [2020-06-16 20:05:53.624] [c.g.h.j.e.HelloWorldTest.helloTest] - Throughput:  9/s (Required: -1/s) - PASSED
[INFO] [2020-06-16 20:05:53.625] [c.g.h.j.e.HelloWorldTest.helloTest] - Memory cost:  16byte
[INFO] [2020-06-16 20:05:53.635] [c.g.h.j.e.HelloWorldTest.helloTest] - Min latency:  100.191414ms (Required: -1.0ms) - PASSED
[INFO] [2020-06-16 20:05:53.635] [c.g.h.j.e.HelloWorldTest.helloTest] - Max latency:  105.2382ms (Required: -1.0ms) - PASSED
[INFO] [2020-06-16 20:05:53.636] [c.g.h.j.e.HelloWorldTest.helloTest] - Avg latency:  101.43268ms (Required: -1.0ms) - PASSED

Or based on HTML:

summary

This article gives the most basic introduction to performance testing, so that friends have a basic understanding of performance stress testing.

Testing, like development, is a time-consuming, laborious, and requires serious work to do well. The knowledge cannot be explained in a single article.

There are also many performance testing tools. This article focuses on the junitperf tool specially built for java developers.

In the next section, we will explain the implementation principle of junitperf from the perspective of source code.

I am an old horse, and I look forward to seeing you again next time.

Open source address: https://github.com/houbb/junitperf