How to initiate an MQTT performance test of 100 million connections and 10 million message throughput

Author: Yiyan

With the advent of the 5G era, the great idea of the Internet of Everything is becoming a reality. The number of connected IoT devices has already reached 12 billion in 2021, and in the next two years, there will be more than 1 billion smart water and electricity meters alone. Under such a large demand for the Internet of Things, massive device access and device management have brought great challenges to network bandwidth, communication protocols, and platform service architecture. How to do the performance test of the IoT protocol represented by MQTT is particularly important. So, how do we do MQTT performance testing?

What is the MQTT protocol

MQTT is an asynchronous communication message protocol based on the TCP/IP protocol stack. It is a lightweight publish and subscribe information transmission protocol. It can be expanded in an unreliable network environment and is suitable for scenarios with limited device hardware storage space or network bandwidth. Using the MQTT protocol, message senders and receivers are not limited by time and space.

For the IoT protocol, several key issues of IoT device communication must be addressed: its network environment is complex and unreliable, its memory and flash memory capacity is small, and its processor power is limited. The MQTT protocol occupies half of the Internet of Things protocol due to its simplicity and ease of implementation, support for QoS, and small packets.

The publish-subscribe model of MQTT

The publish-subscribe model is different from the traditional client-server model. It separates the client (publisher) that sends the message from the client (subscriber) that receives the message, and the publisher and subscriber do not need to establish direct contact. We can not only allow multiple publishers to publish messages to one subscriber, but also allow multiple subscribers to receive messages from one publisher at the same time. The essence of this lies in an intermediate role called a broker (or MQTT Broker) Responsible for all message routing and distribution. The traditional client-server model can achieve a similar effect, but it cannot be as concise and elegant as the publish-subscribe model.

The advantage of the publish-subscribe model lies in the decoupling of publishers and subscribers. This decoupling is manifested in the following two aspects:

• Spatial decoupling : Subscribers and publishers do not need to establish a direct connection, and new subscribers do not need to modify the publisher's behavior when they want to join the network.

• Time decoupling : Subscribers and publishers do not need to be online at the same time, even if there are no subscribers, it does not affect the publisher's publishing of messages.

Why do MQTT performance testing

The MQTT performance test mainly helps us to do the following:

1. Find out the core indicators of MQTT

• The end-to-end delay of messages in different network environments • The maximum number of connections that MQTT Broker maintains at the same time • TPS for MQTT sending and receiving messages

2. Assist MQTT Broker selection

There are countless MQTT Brokers available in the IoT industry. In addition to the classic Mosquitto and AWS, Azure, Baidu Cloud, Alibaba Cloud, IBM and other products that provide IoT MQTT access services, MQTT Brokers that can be used for commercial production There are many more.

However, the system performance and applicable scenarios of each MQTT Broker are different. For example, EMQ has high single-machine performance, a single machine supports millions of concurrency levels, and a cluster supports tens of millions of levels of concurrency. The advantage lies in the service capability of high concurrent connections and high throughput messages; HiveMQ single machine performance is relatively poor, with certain high concurrent connections and high throughput. The service capability of the message.

Through the MQTT performance test, we can more accurately select the appropriate MQTT Broker.

How does the industry generally do MQTT performance testing?

The difficulty of MQTT performance testing is being able to initiate and maintain a large number of connections at the same time.

At present, the most common MQTT performance test method is based on the MQTT-Xmeter plug-in of JMeter. The underlying principle of this plugin is that each thread simulates an MQTT Client and sends connection requests to the MQTT Broker. However, the threading model of JMeter (a concurrency is a thread) determines that if you want to initiate a large number of connection requests, the pressure machine must create the same number of threads, which not only causes a serious waste of resources, but also limits the number of threads that a single pressure machine can initiate. The total number of connection requests. Alibaba Cloud's performance test PTS allows a single thread to initiate and maintain a maximum of 100, a single pressure press with a maximum of 50,000, and a single pressure test with a maximum of 100 million MQTT connections.

Using PTS can help users easily initiate tens of millions or even hundreds of millions of MQTT connections, saving a lot of time and machine costs.

How to use PTS to initiate MQTT performance test of 100 million connections and 10 million message throughput

Steps for usage

1. Create a scene . We create an MQTT stress test scenario in the [Stress Test Center] -> [Create Scenario] of the PTS console;

2. Scene configuration . PTS supports four types of test nodes: establishing connections, publishing messages, subscribing messages, and closing connections. If the user only needs to test the maximum number of connections supported by the Broker, he can only create a connection establishment node.

For Alibaba Cloud's MQTT instance, users only need to turn on the signature authentication switch, fill in the AccessKey ID, AccessKey Secret, and instance ID, and PTS will dynamically generate the username and password. PTS supports specifying the number of single concurrent connections, as well as three flow models of uniform increment, step increment and fixed pressure. Users can control the growth rate of the number of MQTT connections by adjusting the number of concurrency in the stress test.

Scenario configuration example: 10,000 MQTT connections send messages as pub clients, each pub client sends 1 QoS0 message per second; 990,000 MQTT connections serve as sub clients to receive messages, each sub client receives 1 message per second QoS0 messages. Therefore, the total message publishing throughput rate is 10,000 per second, and the total message receiving throughput rate reaches 1 million per second.

Summarize

This article explains:

1. What is the MQTT protocol

2. Necessity of MQTT performance test

3. Advantages of using PTS for MQTT performance testing

4. How to use PTS to quickly initiate an MQTT performance test

At the same time, the new sales method of PTS is coming, and the price of the basic version will drop by 50%! The price of one million concurrent transactions is only 6200! There are also 0.99 trial version for new users and VPC stress test exclusive version, welcome to buy!