The Apache Pulsar ecological project KoP adds a new maintainer: Wu Zhanpeng

About Apache Pulsar

Apache Pulsar is the top-level project of the Apache Software Foundation. It is the next-generation cloud-native distributed message flow platform. It integrates messaging, storage, and lightweight functional computing. It uses a separate architecture design for computing and storage to support multi-tenancy, persistent storage, Multi-computer room and cross-regional data replication, with strong consistency, high throughput and low latency, highly scalable streaming data storage characteristics.
Following the joining of Tencent Maintainer, KoP (Kafka-on-Pulsar), the top project of the Apache Software Foundation and the distributed message flow platform Apache Pulsar ecological project, added a community Maintainer. He is a senior engineer from BIGO Wu Zhanpeng (Github ID: wuzhanpeng) ). BIGO migrated its messaging system from Kafka to Pulsar in 2020. It has extensive experience in the application of KoP and has made many contributions to the community in the direction of Pulsar-Flink. The Maintainer from BIGO has joined the KoP project on behalf of BIGO’s contribution to the Pulsar project. Recognition and further investment.

What is KoP

"KoP" (Kafka on Pulsar) is jointly open sourced by StreamNative and OVHcloud, mainly to meet the strong needs of users who want to switch from Kafka applications to Pulsar.

KoP introduces the Kafka protocol processing plug-in into the Pulsar broker, thereby realizing Apache Pulsar's support for the native Apache Kafka protocol. After adding the KoP protocol processing plug-in to an existing Pulsar cluster, users can migrate existing Kafka applications and services to Pulsar without modifying the code, thereby using the powerful functions of Pulsar, such as:

• Utilize enterprise-level multi-tenant features to simplify operations;
• Avoid data relocation and simplify operations;
•Using Apache BookKeeper and tiered storage to persist the event stream;
•Using Pulsar Functions for serverless event processing.
The addition of a new maintainer

So far, Wu Zhanpeng has contributed 6 commits and 7 PRs to the KoP project; 5 commits and 6 PRs to the Pulsar project. His contribution optimizes the performance of the KoP project:

•Optimize the performance of KoP consumer side, increase the consumption throughput by 3 times, and reduce the decoding time to 1/5 of the original;
•Optimize the memory on the consumer side of KoP to prevent entry data from entering the broker heap memory and reduce the impact of GC;
Contributor's Message

I am very happy to be a Kafka-on-Pulsar Maintainer! As a user and contributor of KoP, becoming a maintainer represents the community's recognition of me. In the process of using and contributing, I have met many partners in the Pulsar community. I look forward to more and more people joining the KoP project, using, participating in contributions, and meeting like-minded engineers like me. In the future, I will devote more energy to the KoP project, contribute code and ideas, and lead more people to get to know KoP and join KoP!

I look forward to Wu Zhanpeng as the KoP project maintainer to bring new strength to the KoP maintainer team, and look forward to everyone's continued efforts to improve the functions of the KoP project and better serve the community.

At present, the Apache Pulsar community has a lot of technical needs for the KoP project. StreamNative will regularly release KoP version updates with community contributors to continuously meet the needs of the community. Warmly welcome the community to participate in the project, communicate and grow together. Hope that more and more senior contributor "players" will unlock the Maintainer role in the future.

Try KoP

•KoP uses the Apache License V2 license, the project address is: https://github.com/streamnative/kop.
• If the Pulsar cluster is already running and you want it to support the Kafka protocol, you can install the KoP protocol processing plug-in to the existing Pulsar cluster. For detailed information, please refer to the description [1].
Friends who want to switch from other Kafka applications to Pulsar are welcome to use KoP. The community has also open sourced AoP (AMQP-on-Pulsar) [2], MoP (MQTT-on-Pulsar) [3], RoP (Rocket- MQ-on-Pulsar)[4], to help users migrate RabbitMQ / MQTT/RocketMQ applications and services to Pulsar.

We look forward to your issue and PR. If you encounter problems during use, you can add 👇 Bot friends, remark "Enter the group" to enter the Pulsar exchange group, and discuss with us.

As a global open source project, as of August 4, 2021, Apache Pulsar has 430 contributors, 9.4 K+ Star, and 2.3 K+ Fork. It is precisely because of the contributions of community developers distributed around the world that the Apache Pulsar project continues to develop and advance.

If you also want to contribute to the Apache Pulsar project, to enter the Maintainer, Committer and even PMC members, please refer to the link below to join us:

Apache Pulsar official contribution guide [5]

Related Reading

• Apache Pulsar ecological project RocketMQ-on-Pulsar adds 3 Tencent Maintainers
• Apache Pulsar ecological project AoP adds two new maintainers of China Mobile!
• Tencent joins: Kafka-on-Pulsar project welcomes 2 Tencent maintainers!
Reference link

[1] Description: https://github.com/streamnative/kop#install-kop-protocol-handler
[2] AoP（AMQP-on-Pulsar）: https://github.com/streamnative/aop
[3] MoP（MQTT-on-Pulsar）: https://github.com/streamnative/mop
[4] RoP(Rocket-MQ-on-Pulsar）: https://github.com/streamnative/rop
[5] Apache Pulsar official contribution guide: http://pulsar.apache.org/en/contributing/

ApachePulsar

The top-level project of the Apache Software Foundation, the next-generation cloud-native distributed message flow platform, integrates messaging, storage, and lightweight functional computing. It uses a separate computing and storage architecture design to support multi-tenancy, persistent storage, and multi-computer rooms across regions. Data replication has streaming data storage features such as strong consistency, high throughput, low latency, and high scalability.