How do Kafka and RabbitMQ choose which one to use?

Preface

We often use asynchronous messages in our work, mainly using two message modes:

• message queue
• Publish/Subscribe

message queue : multiple producers can send messages to the same message queue, but a message can only be consumed by one consumer.

Publish/Subscribe : A message can be concurrently acquired and processed by multiple subscribers.

Kafka and RabbitMQ can meet the above characteristics, so how should we choose which one to use? What is the difference between these two MQs? In what scenarios is Kafka suitable for use, and in what scenarios is RabbitMQ suitable for use? Do you have such doubts? Hope this article can help you.

how to choose?

Development language

Kafka : Scala, supports custom protocols.

RabbitMQ : Erlang, supports AMQP, MQTT, STOMP and other protocols.

Delay queue

If you have the following demand scenario:

• Send a text message to the user 60 seconds after the order is generated.
• The user has not logged in for 7 days to give the user a recall push.
• 15 minutes after the order is placed, the order is closed without payment.

Please choose RabbitMQ , the official delayed queue plug-in (x-delayed-message) has been provided, ready to use out of the box.

Message sequence

If your demand scenario needs to ensure that the messages are in order, for example: the message delivered is MySQL binlog, this kind of message is not allowed to be disordered.

Please select Kafka , it can ensure that all messages sent to the same topic partition can be processed in order.

Priority queue

If your demand scenario is to ensure the priority of message execution, for example, you need to deal with VIP customers' problems first, and then deal with ordinary customers' problems.

Please select RabbitMQ , you can set x-max-priority when creating a queue.

Message retention

If your demand scenario is that the messages after consumption are not deleted immediately, but you want to keep them for a longer period of time.

Please choose Kafka . It can configure a timeout for each topic. As long as the timeout time is not reached, the messages will be retained. Please rest assured that the performance of Kafka does not depend on the storage size. In theory, it will hardly affect the performance of storing messages.

Message filtering

If your requirement scenario is to adopt certain filtering rules to filter the received messages.

Please choose RabbitMQ because it supports message routing. But for Kafka, it can also be achieved in other ways.

Scalable row

If your demand scenario has great requirements for scalability and throughput.

Please select Kafka .

summary

This article is purely an introduction, there are problems, criticisms and corrections are welcome.

I hope that I can bring you some ideas on the choice of using the two.

Recommended reading

• Eventual Consistency Implementation Plan for Distributed Transactions
• of distributed transactions
• Answer two frequently asked code writing questions
• How do I write Git Commit message?