云原生 - Project News｜Apache Pulsar 2.7.4 Released - ApachePulsar

About Apache Pulsar
Apache Pulsar is a top-level project of the Apache Software Foundation. It is a next-generation cloud-native distributed message flow platform that integrates messaging, storage, and lightweight functional computing. Multi-machine room and cross-region data replication, with streaming data storage features such as strong consistency, high throughput, low latency, and high scalability.
GitHub address: http://github.com/apache/pulsar/

Recently, the Apache Pulsar community released Pulsar version 2.7.4! The new release includes improvements and bug fixes from 32 contributors, and 98 changes were committed.

version highlights

Upgraded Log4j to version 2.17.0 - CVE-2021-45105[1]. PR-13392[2]
OpAddEntry can be referenced correctly when recycling ManagedLedger . PR-12103[3]
NPE does not occur OpAddEntry when ManagedLedger is closed. PR-12364[4]

This blog presents the most noteworthy developments in version 2.7.4, grouped by project feature updates. For a complete list of all performance upgrades and bug fixes, please see Pulsar 2.7.4 Release Notes [5] .

Bug fixes and feature enhancements

PR-13392[6]: Upgrade log4j to version 2.17.0 - CVE-2021-45105[7]

Issue: A critical vulnerability in Log4j allows remote execution by an attacker. The vulnerability is described and tracked CVE-2021-44228[8]

solution: upgrade log4j to version 2.17.0. For details, refer to blog .

PR-12103 [. 9]: recovered OpAddEntry correctly referenced by ManagerLedger .

Issue: , after a failed write, a task was scheduled in the background to force the shutdown of the ledger and trigger the creation of a new ledger. May cause NPE or undefined behavior if the OpAddEntry

solution: ManagedLedgerImpl object reference is copied to final variable, background task will not depend on OpAddEntry instance.

PR-12123[10]: No potential race condition in BlobStoreBackedReadHandler

Issue: this release, BlobStoreBackedReadHandler entered an infinite loop while reading the offload ledger. There is a race condition between reading the entry and closing the BlobStoreBackedReadHandler.

solution: adds a status check before reading entry and makes BlobStoreBackedReadHandler exit the loop entryID greater than lastEntryID

PR-12364[11]: NPE does not occur OpAddEntry

issue: this release, the ManagedLedgerBkTest#managedLedgerClosed test closed the ManagedLedger object on some asyncAddEntry operations and failed with NPE.

Solution: Close OpAddEntry when ManagedLedger notifies OpAddEntry failure. This correctly recycles the OpAddEntry object and successfully triggers the failed callback.

PR-11294[12]: Correctly set topic policy by partition's topic name.
Issue: this release, the topic name of a partition could not be used to set topic policies.

solution: allows setting topic policies by partition's topic name by converting the partition's topic name in SystemTopicBasedTopicPoliciesService

PR-8611[13]: Schedule rate limiter in effect for consumers.
issue: this release, since acquiredPermits resets to 0 every second, the scheduling rate limiter does not take effect if all consumers start reading in the next second.

solution: DispatchRateLimiter the behavior of permits every second instead of acquiredPermits to 0. Consumers temporarily stop reading the entry, until acquiredPermits the value returned is less than permits .

PR-11310[14]: NPE does not occur when unload bundle operation is performed.
Issue: NPE occurs when unload bundle operation is performed when stress testing persistent partition topic. At the same time, the producer does not write the message.

solution: adds more security checks to fix this.

PR-11346[15]: Fix inconsistent behavior of namespace bundle cache.
Issue: this release of 161f1068fb51ce, the namespace bundle cache was not invalidated after the namespace was deleted.

Solution: The namespace policy cache is invalidated when the bundle cache is invalidated.

PR-11342[16]: Shut down the replicator and replication client after deleting the cluster.
Issue: , the replicator and replication client did not shut down after deleting the cluster. The producer of the replicator keeps trying to reconnect to the deleted cluster.

Solution: Shut down the corresponding replication machine and replication client.

PR-10384[17]: Post rate limiter works as expected.
issue: this release, if preciseTopicPublishRateLimiterEnable set to true for rate limiting, the following issues occurred:

When changing a limit from a bounded limit to an unbounded limit, the updated limit does not set the bounds.
Each topic creates a dispatch thread for each limiter instance.
Topics do not publish scheduling threads when topics are unloaded or operations are closed.
Updating the limit does not shut down the scheduling thread associated with the replaced limiter instance.

solution

Clean up previous limiter instances before creating new ones.
Use brokerService.pulsar().getExecutor() as the scheduler for the rate limiter instance.
Added resource cleanup hook for theme shutdown (uninstall).

PR-12015[18]: Clean up newly created ledger if the ZNode list cannot be updated.
Issue: When updating the ZNode list, ZooKeeper throws an exception and does not clean up the created ledger. managedLedger are not indexed into the topic 061f1068fb54e6 list and therefore cannot be kept clear as topics. Also, if a ZNode version mismatch exception is thrown, the number of ZNodes will increase in ZooKeeper.

Solution: When ZNode list update fails, regardless of exception type, delete created ledger from broker cache and BookKeeper.

participate

Use the new version

Welcome everyone download [19] and use the new version! If you encounter problems during use, you can raise questions and communicate with the community issue[20]

Join the Apache Pulsar community

The growth of the Pulsar project comes from and is rooted in the community. The preparation and release of new versions are inseparable from the contributions of community partners. Would you like to be one of them? Participating in open source can gain recognition from inside and outside the company and the community, and make friends with like-minded partners from various fields; at the same time, it can also improve personal influence and promote personal development. Participating in open source is not exclusive to coders. Community, documentation and other aspects can allow everyone to use their skills.

As a global open source project, as of now, Apache Pulsar has 480 contributors, 10.2 K+ Stars, and 2.6 K+ Forks. We have provided a participation guide for everyone, and welcome more and more small partners to help the continuous development and progress of the Apache Pulsar project.

Apache Pulsar Official Contribution Guide [21]
Apache Pulsar volunteer family

Project News｜Apache Pulsar 2.7.4 Released

About Apache Pulsar

Bug fixes and feature enhancements

participate

Use the new version

Join the Apache Pulsar community

Related Reading

Citation link

ApachePulsar

引用和评论

深入解析 Apache BookKeeper 系列：第二篇 — 写操作原理

如何画好架构图：7种常用类型与示例

Monorepo：让你的项目脱胎换骨，既能代码复用，又能独立部署！

2024 OSCAR 开源产业大会在京召开

张晋涛：KubeCon China 2024 回顾

保证Redis和数据库数据一致性的方法

2024OSCAR开源产业大会 | 开源项目社区与商业化分论坛精彩前瞻！！！