The message queue RocketMQ's "Six Weapons" to cope with the peak traffic of Double Eleven

Author:
review and proofreading: years,
Editing & Typesetting: Wen Yan

Behind the "498.2 billion, 583,000 transactions per second"

Catalyzed by the new crown pneumonia epidemic, digital lifestyles have gradually become the new normal. "498.2 billion, 583,000 transactions per second" is an intuitive manifestation of the first-mover advantage and huge potential of the digital economy of the 2020 Tmall Double 11 Global Carnival (abbreviated as: Tmall Double 11).

In the face of tens of millions of concurrent and trillion-level traffic peaks, behind the strong support is the official designated product of the double eleven transaction core link: the message queue RocketMQ.

Business Pain Points in Double Eleven Trading Scenarios

As Double Eleven heats up year by year, ensuring the stability of the trading scene has become the key for all companies in the Double Eleven business. The early morning of the Double Eleven event each year is the day of "Wanmin Carnival", and it is also the trading system of various companies. When tested, ensuring the business processing capabilities of the core trading system and effectively responding to hundreds of thousands of transaction orders per second becomes the top priority. Failure to buffer traffic will directly lead to the collapse of these systems. The core "secret" to avoid system crashes is the message queue RocketMQ.

How does the message queue RocketMQ help various enterprise trading systems to withstand the impact of the instantaneous tens of millions of TPS and trillions of traffic peaks, and keep the messages between applications smooth? The following introduces you the "six weapons" of the message queue RocketMQ to cope with the traffic peaks of Double Eleven.

The "six weapons" of the message queue RocketMQ

What problems will the traffic peak of Double Eleven bring to users and merchants' system business? How do the "six weapons" of the message queue RocketMQ solve these problems? The editor will take you to explore one or two:
 

Weapon 1: "Asynchronous Decoupling"

Background: On the night of Double Eleven, when users "pointed the world" on their mobile phones, they thought that behind a small shopping APP are actually a huge system. From the moment the user purchases the product, Dealing with hundreds of business systems, each transaction order data will be related to hundreds of downstream business systems, including logistics, shopping carts, points, direct charging, stream computing analysis, etc. The entire system is huge and complex, and the architecture is designed A little unreasonable will directly affect the continuity of the main station business.

Faced with such a complex and huge system, to avoid the impact of mutual coupling between system services, the "asynchronous decoupling" function of the message queue RocketMQ is used. The message queue RocketMQ realizes the loose coupling of upstream and downstream business systems. Loose coupling can reduce The complexity of the system shortens the response time of user requests (compresses the sum of the time required for multiple steps to the time required for only one message), and ensures that the failure of a downstream subsystem does not affect the entire link.

Weapon 2: "Cut the peaks and fill the valleys"

Background: After dealing with the coupling problem caused by the huge system behind the transaction business, from the user's point of view, tens of millions of users were clicking on the purchase page at the same time at 0:00 during Double Eleven. Due to the massive amount of user requests, traffic has surged. In the face of such a large amount of access traffic, the downstream notification system may not be able to carry the massive amount of calls, and may even cause problems such as system crashes and miss notifications.

To solve these problems, it is necessary to use the message queue RocketMQ's "peak cutting and valley filling" function. The message queue RocketMQ can be added between the application and the downstream notification system. RocketMQ supports high concurrent message writing with low latency, and unlimited accumulation Ability to avoid the impact of ultra-high traffic and ensure the smooth and stable operation of downstream businesses within a safe water level.

Weapon 3: "Distributed Transaction Message"

Background: Through the previous introduction, we learned that through asynchronous decoupling of messages, distributed processing of messages can be realized. In the traditional distributed transaction processing method, the user creates a new order information, accompanied by this order information. Changes, shopping carts, user tables, points, etc. in the entire business chain need to be changed, and the system needs to use distributed transaction coordination components to ensure transaction consistency for multiple business calls. Traditional distributed transaction components pursue strong consistency, low performance throughput, and complex systems. How can we achieve distributed transactions without making the system too complicated?

At this time, the "distributed transaction message" function of the message queue RocketMQ plays a key role. Through the original lightweight order flow transaction coordination ability, only one message can be sent to achieve a distributed transaction with final consistency of the message. At the same time, ensure that the order status is persisted and the downstream calls are consistent.

Weapon 4: "Message Filtering"

Background: Through the above introduction, you will find that the process from the customer placing an order to the customer receiving the product will produce a series of messages, which can be divided into transaction messages, logistics messages, shopping cart messages, etc. according to the type of message. How to ensure that each type of message is effective Delivered and consumed accurately?

At this time, you need to use the "message filtering" function of the message queue RocketMQ. You can define different attributes for different types of messages through Tag, and filter the messages according to the message attributes. Only the messages that meet the filter conditions will be delivered. Go to the consumer end for consumption. For example, define geographic attributes for logistics messages, which are divided into Hangzhou and Shanghai according to the region:

• Order message
• Logistics news
• • Logistics news and the location is Hangzhou
• • Logistics news and the location is Shanghai

Weapon Five: "Timed Messages"

Background: In addition to the problems that may occur in the above system levels, users themselves may have encountered some small details during the shopping process. For example, after clicking the purchase button, a prompt "Please complete the payment within 30 minutes" will appear , If the payment is not made for more than 30 minutes, the order will be closed automatically.

This service uses the "timed message" function of the message queue RocketMQ. The message queue RocketMQ can implement customized timed messages with second-level precision intervals, and trigger some timed tasks through messages, such as sending reminders to users at a fixed time. , And finally realize the central scheduling of massive order status changes overtime.

Weapon Six: "Sequential Sending and Receiving"

Background: During the Double 11 promotion, the buyer's business side and the transaction system itself will face many problems, and the seller side will also encounter some difficulties. For example, the buyer buys something, but the seller can't see it.

In order to solve this problem, it is generally necessary to use the sequential message synchronization capability of the message queue to synchronize the change subscription of the buyer table to the seller table. At this time, relying on RocketMQ's hotspot-free, high-performance, and highly reliable sequence messages can ensure the sequence synchronization of database changes and ensure the synchronization of orders for buyers and sellers.

Summarize

Through the above introduction, I have taken you to understand the power of the six weapons of the message queue RocketMQ on the "battlefield" of Double Eleven. The "Double Eleven" war is about to start in 2021, and the message queue RocketMQ will protect your Double Eleven business. At the same time, the platinum version can provide 99.99% service availability and 99.99999999% data reliability. Contact us and look forward to accompany your business to "ride the wind and waves" in 2021 Double Eleven.