EMQ X and RabbitMQ message server MQTT performance comparison (below)

In last article , we use the same hardware resources, respectively the MQTT the X-news server EMQ and RabbitMQ stress tests. The results show that in the "many-to-one" scenario, there is not much difference between EMQ X and RabbitMQ; while in the "one-to-many" scenario, RabbitMQ has a more obvious gap than EMQ X.

In this issue of the article, we will further analyze this result.

There are three main reasons for the gap: the way of communication between nodes, the way of message flow architecture, and the use of queues.

Communication between nodes

RabbitMQ-delegation architecture

RabbitMQ uses the distributed connection of the Erlang language, that is, each node is connected to each other in pairs, and each node is connected to another node with a single link. In the case of the figure, three nodes are connected in sequence; when communication between nodes is required, a message needs to be sent from one node to another through this single link.

In the fan-out example, you normally need to push messages to the queues of all nodes. The optimization method used by RabbitMQ is: your message only needs to be sent once, and then its built-in proxy delegation framework will dispatch this message and send it to the queues of other nodes. In this process, the messages are sent in order, so it is guaranteed that the messages are in the same order in different queues.

But this solution is not perfect, because you will only send all the messages once, and the distribution work depends on the same delegate process. And RabbitMQ's strategy for selecting this agent process is based on the publisher's hash algorithm. So, if you have only one publisher, all messages will be pushed to a single proxy agent process.

EMQ X - Gen_RPC

There is a subtle design in EMQ X: it not only has distributed connections, but also Gen_RPC. Distributed connection and Gen_RPC perform their respective duties. The former is used to exchange Mnesia data information, and the latter is only suitable for message forwarding. Whenever you need to publish a message from one node to another node, EMQ X does not automatically regenerate a new inter-node link (default is 1 connection), and then process and push a message from one node through these new connections. Work to another node. Instead, it relies on a dedicated Gen_RPC connection specially designed for this scenario to handle this message push work. So in the fan-out (one-to-many) example, these links will be fully utilized effectively.

However, the performance of this design may be affected in a network partition environment. There is only one distributed connection between RabbitMQ nodes, so when the connection is disconnected and the brain is split, the work of healing and repairing will be easier.

Message flow

MQTT plugin

After RabbitMQ uses the MQTT plug-in, it will listen to messages published MQTT protocol After getting the message, the message is parsed, and then transformed through the AMQP protocol, and finally sent to RabbitMQ.

If you want to send a message, you need to enter mqtt_reader after the socket, and then enter all the processes shown in the following figure. However, if you want to receive the message just sent in the same channel at the same time, all the processes shown in the figure above need to be repeated again, including mqtt_reader. Among them, mqtt_reader is not only responsible for reading, but also for writing.

AMQP

The AMQP scenario is different. Each message is read by a reader and written by a writer. These two channels are independent of reading and writing. Reader is only responsible for reading content, while writer is only responsible for writing content. They perform their duties and are independent of each other. The only channel channel is a main Erlang process, which is responsible for the exchange of messages.

It can be seen that the obvious design problems of RabbitMQ in the MQTT scenario can lead to performance degradation, so what will happen if the RabbitMQ test case of the AMQP mode is introduced? Modulate RabbitMQ to use the MQTT plug-in and use a single AMQP mode, and then compare EMQ X in the stress test, it can be seen that EMQ X is still better in all tests, but in general, the AMQP mode is used RabbitMQ is better than its original results.

Many to one

In this scenario, RabbitMQ and EMQ X already have similar performance.

One to many

But if in the fan-out (one-to-many) scenario, EMQ X still has significant advantages, but the gap between RabbitMQ (AMQP) has been significantly reduced.

queue

The above tests all use QoS 1 messages. When sending QoS 1 messages, these messages must be stored on the hard disk as a durable backup each time. Therefore, the use of queue space is also particularly important.

RabbitMQ

RabbitMQ maturely uses a default queue space execution method (which can be replaced by other queues). This variable queue balances the durability of the message and the delay in sending the message to the client. But in the worst case, a message may be stored in memory. However, this also helps RabbitMQ to bring the server back online after a crash and restart, and all clients can reconnect and receive the original persistent messages.

EMQ X

EMQ X's implementation of queues is very simple, that is, priority queues are used in memory. If the sent message cannot be pushed into the receiver's queue, the message will be discarded. In EMQ X, only some other persistent plug-ins can make messages persisted. These functions are provided in the commercial version.

The original intention of EMQ X was to separate the access layer, so the problem of message persistence was left to the back end. This problem will be resolved in future versions with persistent sessions (persistence session).

Throttling

RabbitMQ-flow control

RabbitMQ uses a well-known flow control mechanism, which gives each process a credit value, as shown in the following figure. Suppose that after our server receives a message and read it by the reader, the message is sent to the channel. This process will consume the corresponding credit value of reader and channel. In this way, it is possible to achieve non-excessive transmission by keeping the credit values of the two parties matched and synchronized.

This is actually a good solution. Imagine that we have many users, that is, there are many queues. Each message sent means that the message will be distributed to many queues, which will seriously affect the RabbitMQ instance. However, this set of procedures will prevent RabbitMQ from continuing to read messages from the receive buffer because the send buffer is almost full!

EMQ X-current limit

The throttling of EMQ X is mainly achieved by limiting the flow of the reading party. First, by default, 200 messages will be read from the socket at a time. When these messages are fully received, they will be processed one by one. Once the socket reports that it has reached the maximum limit of the reading party, it will check the number of publishers and the number of bytes that have been read, and sleep for a period of time based on this value. The receiving buffer will eventually be filled, and the publisher will not publish any more content according to the requirements of the flight window in the TCP protocol.

Summarize

The above is the result and analysis of this horizontal evaluation. The ultimate winner is difficult to assert, but in terms of server performance, EMQ X is definitely slightly better. But RabbitMQ also has its unique advantages.

EMQ X design principles

In the design of EMQ X, the front-end protocol (FrontEnd) and the back-end integration (Backend) are separated first, and then the message routing plane (Flow Plane) and the monitoring management plane (Monitor/Control Plane) are separated:

1. The core problem of EMQ X is to handle massive amounts of concurrent MQTT connections and routing messages.
2. Make full use of the advantages of Erlang/OTP platform in soft real-time, low latency, high concurrency, and distributed fault tolerance.
3. Connection (Connection), session (Session), routing (Router), cluster (Cluster) layering.
4. The message routing plane (Flow Plane) is separated from the control management plane (Control Plane).
5. Support back-end database or NoSQL to achieve data persistence, disaster recovery backup and application integration.

System layering of EMQ X

1. Connection Layer: Responsible for TCP connection processing and MQTT protocol encoding and decoding.
2. Session Layer: Process the MQTT protocol publish and subscribe message interaction process.
3. Route Layer: Routing and dispatching MQTT messages within the node.
4. Distributed Layer: Route MQTT messages between distributed nodes.
5. Authentication and access control (ACL): The connection layer supports an extensible authentication and access control module.
6. Hooks and Plugins: Each layer of the system provides extensible hooks and supports plug-ins to extend the server.

RabbitMQ is more similar to Kafka's message queue cache design. It is recommended to combine the two in IoT projects.

Copyright statement: This article is EMQ original, please indicate the source for reprinting.

Original link: https://www.emqx.com/zh/blog/emqx-or-rabbitmq-part-2

Technical support: If you have any questions about this article or EMQ related products, you can visit the EMQ Q&A community https://askemq.com ask questions, and we will reply to support in time.

For more technical dry goods, please pay attention to our public account [EMQ Chinese Community].