From news to data lake: watching the latest developments of Apache RocketMQ, Hudi, and Kyuubi

Introduction to focuses on message queue & data lake scenarios, Apache RocketMQ with Hudi & Kyuubi

Developers in Shanghai, on December 18th, Apache RocketMQ & Apache Hudi & Apache Kyuubi (Incubating) three communities Meetup came to create the strongest message transmission, real-time computing, and data into the lake integrated solution special session.

This event focuses on the combination of Apache RocketMQ and Hudi and Kyuubi data lake to help developers better deal with business challenges. The event will invite many technical experts from Himalaya, Ping An Securities, NetEase, and Alibaba Cloud to share the practice and application of Apache RocketMQ with hudi & Kyuubi.

Related project introduction

Apache RocketMQ

Apache RocketMQ is an open source messaging product by Alibaba. It has undergone many years of rigorous test of traffic peaks during Double Eleven. It has low latency, high performance, high reliability, trillion-level capacity and flexible scalability. At present, more than tens of thousands of companies around the world are using Apache RocketMQ, including not only the leading companies from the Internet such as Byte, Kuaishou, Xiaomi, Didi, and Tongcheng Yilong, but also from leading banks, brokers, insurance, and funds. A series of extremely demanding financial companies.

After several years of development, Apache RocketMQ has become the first choice for business messaging in the microservices field. With the advent of the cloud-native era and the rise of real-time computing, Apache RocketMQ 5.0, which was born in the cloud and grew longer than the cloud, came into being, and was upgraded to the cloud. The native message, event, and stream fusion processing platform helps users to more easily build next-generation event-driven and stream processing applications.

Github address:

https://github.com/apache/rocketmq

Apache Hudi

Apache Hudi is a top-level project of the Apache Software Foundation. It is a new generation of streaming data lake platform that supports data insertion, update, deletion, and incremental data processing; it can help build an efficient enterprise-level data lake.

GitHub address: _ http://github.com/apache/hudi _

Apache Kyuubi（Incubating）

Apache Kyuubi (Incubating) is a Thrift JDBC/ODBC service. It is currently connected to the Apache Spark computing framework and supports multi-tenant and distributed features. It can meet the needs of various big data scenarios such as ETL and BI reports in the enterprise. Kyuubi can provide a standardized interface for enterprise-level data lake exploration, giving users the ability to mobilize data in the entire data lake ecosystem, enabling users to process big data like ordinary data. The project has officially entered the Apache incubator on June 21, 2021. From the perspective of the current development goals of the community, its main direction is to build a serverless SQL on Lakehouse-oriented service based on its own architecture design and around various mainstream computing frameworks. Apache Kyuubi (Incubating) is a Thrift JDBC/ODBC service. It is currently connected to the Apache Spark computing framework and supports multi-tenant and distributed features. It can meet the needs of various big data scenarios such as ETL and BI reports in the enterprise. Kyuubi can provide a standardized interface for enterprise-level data lake exploration, giving users the ability to mobilize the data of the entire data lake ecology, enabling users to process big data like ordinary data. The project has officially entered the Apache incubator on June 21, 2021. From the perspective of the current development goals of the community, its main direction is to build a serverless SQL on Lakehouse-oriented service based on its own architecture design and around various mainstream computing frameworks.

Github address: _ https://github.com/apache/incubator-kyuubi _

Click here participate in the event now!

Recommended in the past

RocketMQ 5.0 POP consumption mode exploration

161a4504aac88b The strategic cooperation announced at the Open Source Gravity Summit of the 2021

Evolution Record | How do companies at different stages build a monitoring system?

---

For more information, please scan the QR code below or search for WeChat account (AlibabaCloud888) to add cloud native assistant ! Get more information!

Copyright Statement: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users, and the copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.