The first issue of "Open Source People Saying" is launched! Let's take a look at "The Story Behind Ten Trillion Messages"

Introduction: How does Alibaba complete real-time message delivery with nearly a trillion-level messages every day? Hundreds of thousands of servers and thousands of applications are inseparable from distributed messaging middleware RocketMQ for smooth collaboration and real-time message delivery. For a long time, RocketMQ's easy-to-deploy, high-performance, and high-availability architecture has supported a large number of business scenarios inside and outside the group for decades. In 2016, Alibaba announced that it would donate the open source distributed messaging middleware RocketMQ to Apache, and it became an Apache top-level project (TLP) in one fell swoop. A project initiated by Chinese developers stands on a world stage and attracts the attention of enterprises and developers around the world. RocketMQ is very popular among developers and has won many industry awards. What kind of people make this project continue to burst into vigorous vitality?

Video link: https://developer.aliyun.com/live/249109

This is an open source project developed on GitHub from the first day of its birth, and it is also China's first non-Hadoop ecological top-level Apache project. It unifies the message middleware of all business lines within the Alibaba Group, and has accompanied the development of China's Internet for several iterations.

"The Story Behind Ten Trillion Messages" records the little-known founding stories behind RocketMQ from its birth, open source to development. Let's learn about the people and things behind the distributed messaging middleware Apache RocketMQ through "Open Source People Saying"!

Double 11, RocketMQ shakes, and the upper-level business " shakes three times"

As early as 2007, the Alibaba middleware team independently developed the message middleware Notify. As the predecessor of RocketMQ, Notify has supported Taobao and Tmall business as the main message middleware for a long time.

Wang Xiaorui, the founder of Apache RocketMQ, joined Taobao Middleware in 2011. The biggest challenge at that time was that Notify would go wrong every year during the "Double 11", which would delay the news for half a day or even more than a day, and merchants could not see that buyers had already purchased the goods, which had a great impact on merchant services and customer experience. After careful research and judgment, it was found that the core of the problem is how to achieve high-speed reading and writing. Wang Xiaorui's team believes that to solve this problem, it needs to be solved based on a new storage architecture. Thus, RocketMQ was born.

RocketMQ was born. It was open source and co-constructed at the beginning, and the first version was released in 3 months.

Wang Xiaorui (the founder and author of Apache RocketMQ) said: "Ali's culture is very open. When I was eating with Xiaoxie (now the head of Alibaba Cloud's Basic Products Division) in the cafeteria, I said that we wanted the product to be called RocketMQ. It can be open source from the beginning. Xiaoxie gave support at that time." Taobao's culture is also to do it right away and do it wonderfully. If you have an idea, do it quickly. So I was actually thinking about how to solve these problems in the process of dreaming. The boss did not set a strict time point, saying when you must finish writing, but my idea is that I have already figured out the whole structure and need to implement it quickly. "

From the first day, RocketMQ has been fully displayed on Github, which means that the first line of code of RocketMQ is open to users. With the joint participation of many domestic and foreign developers. RocketMQ iterates rapidly, and it took about 3 months to complete the first version.

In 2015, at the China-Japan-Korea Open Source Forum, RocketMQ, as the representative of China's open source software, won the CJK OSS Award.

On the day of "Double 11" in 2016, the volume of news reached trillions, and the peak TPS was tens of millions, creating the largest news circulation record in China and even in the world.

RocketMQ donated to become Apache's top open source project in just one year

At the beginning of Alibaba's establishment, the prototype of the entire e-commerce platform was quickly built based on open source software. Alibaba is grateful for the great contribution and value brought by the open source community, and hopes to share its own experience and give back to the community to benefit more people.

In 2016, Alibaba donated the distributed messaging middleware RocketMQ to the Apache Open Source Software Foundation. Graduated from Apache in just 10 months and became an Apache Top Level Project (TLP). This also marks that China has the first non-Hadoop ecological top-level project.

(On September 25, 2017, about 10 months from incubation to graduation, Apache officially announced that the RocketMQ donated by Alibaba to the Apache community officially graduated from the community and became a top Apache project.)

Wang Ronggang, head of Apache RocketMQ community operations, said: "We have finally completed a small phased goal, a project initiated by Chinese developers, standing on a world stage and attracting the attention of global enterprises and developers. It still feels very proud.”

In 2021, RocketMQ 5.0 will be officially released

In 2021, after the continuous efforts of many developers in the community, RocketMQ 5.0 appeared in the field of vision and sparked extensive discussions. The core of the new version includes two new highlights. First of all, the core message scene is fully expanded. RocketMQ 5.0 is no longer limited to the message decoupling scene, but will have a new layout of event-driven and message streaming scenarios. Second, the technical architecture continues to evolve, gradually forming a one-stop integrated processing technology architecture and trend .

In 2022, RocketMQ-MQTT, RocketMQ-Connect and RocketMQ-Streams will be released in batch message indexing and logical queue, completing the upgrade from the business message platform to the integrated processing platform of "message, event, and stream". Developers can implement a message store that supports multiple scenarios such as streaming computing, asynchronous delivery, and integrated driving. Realize one-stop solution to technical problems, greatly reduce technical complexity and operation and maintenance costs, and simplify enterprise application architecture.

Ding Yu, Vice Chairman of Alibaba Open Source Committee, said: "In the future, we will increase RocketMQ's open source investment and build a more prosperous community ecosystem. We hope to work with more developers and contributors to pursue the ultimate, open sharing, and realize technology universality. benefit."

It is precisely because there are such a group of open source people who love open source and are willing to fight for it that the future of open source in China is promising!

---

"Open Source People Talk" is a high-quality open source video column jointly produced by Alibaba Cloud Developer Community and InfoQ. The column focuses on four open source fields of Alibaba: cloud native, operating system, database, big data & AI, and introduces excellent open source software: RocketMQ, Dubbo, Anolis OS (Dragon Lizard Operating System), Flink, PolarDB, OceanBase, etc. The stories behind, record Spread the spirit of Ali technology's pursuit of perfection and openness and sharing.

https://developer.aliyun.com/topic/opensourcefolkstalk

Copyright statement: The content of this article is contributed by Alibaba Cloud's real-name registered users. The copyright belongs to the original author. The Alibaba Cloud developer community does not own the copyright and does not assume the corresponding legal responsibility. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find any content suspected of plagiarism in this community, fill out the infringement complaint form to report it. Once verified, this community will delete the allegedly infringing content immediately.