From news to data lake: watching the latest developments of Apache RocketMQ, Hudi, and Kyuubi

Developers in Shanghai, on December 18th, Apache RocketMQ & Apache Hudi & Apache Kyuubi (Incubating) three communities Meetup came to create the strongest message transmission, real-time computing, and data into the lake integrated solution special session.

This event focuses on the combination of Apache RocketMQ and Hudi and Kyuubi data lake to help developers better deal with business challenges. The event will invite many technical experts from Himalaya, Ping An Securities, NetEase, and Alibaba Cloud to share the practice and application of Apache RocketMQ with hudi & Kyuubi.

Related project introduction

Apache RocketMQ

Apache RocketMQ is a messaging product open sourced by Alibaba. It has undergone years of rigorous testing of double eleven traffic peaks. It has low latency, high performance, high reliability, trillion-level capacity and flexible scalability. At present, more than tens of thousands of companies around the world are using Apache RocketMQ, including not only the leading companies from the Internet such as Byte, Kuaishou, Xiaomi, Didi, and Tongcheng Yilong, but also from leading banks, brokerages, insurance, and funds. A series of extremely demanding financial companies.

After several years of development, Apache RocketMQ has become the first choice for business news in the microservices field. With the advent of the cloud-native era and the rise of real-time computing, Apache RocketMQ 5.0, which was born in the cloud and grew longer than the cloud, came into being, and was upgraded to the cloud. The native message, event, and stream fusion processing platform helps users to more easily build next-generation event-driven and stream processing applications.

Github address:
https://github.com/apache/rocketmq

Apache Hudi

Apache Hudi is a top-level project of the Apache Software Foundation. It is a new generation of streaming data lake platform that supports data insertion, update, deletion and incremental data processing; it can help build an efficient enterprise-level data lake.

GitHub address:
http://github.com/apache/hudi

Apache Kyuubi（Incubating）

Apache Kyuubi (Incubating) is a Thrift JDBC/ODBC service. It is currently connected to the Apache Spark computing framework and supports multi-tenancy and distributed features. It can meet the needs of multiple big data scenarios such as ETL and BI reports in the enterprise. Kyuubi can provide a standardized interface for enterprise-level data lake exploration, giving users the ability to mobilize data in the entire data lake ecosystem, enabling users to process big data like ordinary data. The project has officially entered the Apache incubator on June 21, 2021. From the perspective of the development goals of the community at the current stage, its main direction is to build a serverless SQL on Lakehouse-oriented service based on its own architecture design and around various mainstream computing frameworks. Apache Kyuubi (Incubating) is a Thrift JDBC/ODBC service. It is currently connected to the Apache Spark computing framework and supports multi-tenancy and distributed features. It can meet the needs of multiple big data scenarios such as ETL and BI reports in the enterprise. Kyuubi can provide a standardized interface for enterprise-level data lake exploration, giving users the ability to mobilize data in the entire data lake ecosystem, enabling users to process big data like ordinary data. The project has officially entered the Apache incubator on June 21, 2021. From the perspective of the development goals of the community at the current stage, its main direction is to build a serverless SQL on Lakehouse-oriented service based on its own architecture design and around various mainstream computing frameworks.

Github address:
https://github.com/apache/incubator-kyuubi

Click here participate in the event now!

For more information, please scan the QR code below or search for WeChat account (AlibabaCloud888) to add a cloud native assistant! Get more information!