The cloud-native era message queue and stream fusion system provides a unified consumption model and supports both message queue and stream scenarios. It can not only provide enterprise-level read and write service quality and strong consistency guarantee for queue scenarios, but also provide high-quality streaming scenarios. Throughput and low latency; adopting the storage and computing separation architecture, supporting enterprise-level and financial-level functions such as large clusters, multi-tenancy, millions of topics, cross-regional data replication, persistent storage, tiered storage, and high scalability.
GitHub address: http://github.com/apache/pulsar/
Scenario Keywords :
Asynchronous decoupling peak shaving and valley filling cross-city synchronous message bus
Stream Storage Batch Stream Fusion Real-time Data Warehouse Real-time ETL Financial Risk Control
Abstract: At the beginning of 2022, the Apache Pulsar community welcomed a new Committer, Li Li (GitHub address: @urfreespace). Li Li from StreamNative is a full-stack engineer. His main contribution to the community is the construction of the Pulsar official website and documentation system. How can non-Pulsar project developers participate in open source? What role can a full stack engineer play in an open source project? Li Li will bring you the "new" ideas of participating in open source projects and obtaining open source community certification through his personal experience. \
Planning of this article: Chicken Chop, Hai Qi
- "Why do you recommend that you do open source? In addition to improving yourself, the open source project itself is a brand that endorses your ability."
- "The document is well written, it is good to use, the experience is good, and everyone is willing to use it, and it is conducive to the promotion and popularization of a project."
- "A good open source product must be complete and robust in terms of websites and documentation, and the user experience of documentation and websites that someone maintains is friendly."
———— Li Li
To sum up Li Li's style in one word, "miscellaneous" seems to be the most appropriate. The "miscellaneous" here is a compliment, and it is the epitome of Li Li's career and daily life: During his career, Li Li has been engaged in research and development, and has been involved in various fields, such as Java/Pyhton/Go server, Web front-end, iOS Client, ELK operation and maintenance and R&D work, and now he is a full-stack engineer; in life, Li Li has a wide range of hobbies: mountain climbing, carpentry, various handmade gadgets, reading, etc., and reading is also extensive.
(Let's walk into his open source world from Li Li's first point of view)
From Kafka to Pulsar, from users to developers
Before joining the Pulsar community, I worked on the operation and maintenance and R&D of ELK-related technology stacks for more than a year; I encountered some operation and maintenance pain points in the process of using Kafka, so I began to investigate Pulsar, and I have gradually deepened since then. Learn about Pulsar.
In addition, before entering the community, I have been contacting and researching the Pulsar project as a user, and I have not participated in any contribution to the main Pulsar project. Now as a full stack engineer, I'm constantly exposed to multiple aspects of Pulsar. From user to developer role, a little bit to Apache Pulsar.
The Birth of Two PIPs: Creating a Better Project Experience
My main contributions at Pulsar revolve around PIP-87 and PIP-112 :
- Upgrade the Pulsar website and revise the basic framework;
- Optimize the Release Note automation script;
- Upgrade Pulsar documentation to automate processing.
Friends who recently visited Pulsar's official website will find that the beta version of Pulsar's new website has entered the review stage. You can find relevant links on the current Pulsar website and click to view . The new version of the website is one of my contributions.
goodbye old site
In the past, the website framework version was too old, with incomplete functions and features, and weak customization capabilities (for example, the menu on the left side of the document could not be expanded), and the user felt unfriendly. Therefore, Liu Yu , a member of Pulsar PMC, and I received feedback from the community on the experience of using the website and the optimization opinions of the framework. After unified sorting, we discussed with the community many times to write a PIP document to solve the pain points of the website and upgrade and optimize the website framework. At present, the upgraded website has complete functions and features, the menu can be expanded, and it has a responsive layout; the mobile terminal is adaptable and friendly, and users can also get a comfortable experience using mobile phones.
In addition, the new website meets the needs of multi-tab switching. You can use the development language to customize the document to achieve advanced interactive effects; at the same time, some style UI layers have achieved advanced customization, which is no longer limited to the original low-level grammar, which effectively improves the documentation. flexibility.
It is worth noting that we are conducting research on the new website and collect your suggestions for the new website in the form of a questionnaire survey . You are welcome to participate in filling in.
Document automation
The manual generation of the original website Java layer code class document (Pulsar main project) is cumbersome, which greatly increases the workload of community partners. Through automated scripts, the documents are automatically generated during the construction process, and then the documents are directly embedded in the website, which can greatly reduce the tedious work, optimize the allocation of human time, and improve the execution efficiency.
When the earlier Release Management version was released, the generation of Release Notes needed to be manually organized and written into the document. Such an operation process not only wastes manpower and time, but also has the risk of errors and omissions caused by manual input of information. Therefore, the PIP-112 project has added automated script configuration to automatically pull and typeset Release Note records to complete the filing.
In addition, with the iteration of the version release, the document is getting longer and longer, it is difficult to browse, and it takes a lot of time to find the specified content. Therefore, in response to the problem of multi-component content mixing in versions, I built a framework for archiving and grading: distinguish different components, such as Broker, Bookie, Client, etc., and then separate large and small versions under each component. This makes it convenient for users to quickly browse documents according to their needs for specific components, and achieve efficient indexing.
At present, the archive rating has not been exposed, and you can get the jump link on Pulsar's new website.
User experience is really good
There are two basic ways to learn about a new project and community: one is the website and documentation, and the other is the source code. Often the source code is more difficult to read, while the documentation is easier to understand. Documentation is both a getting started guide and a great source of community contributions. Incomplete documentation and unfriendly websites can seriously affect the user and developer experience. The documentation is well written, easy to use, and comfortable to experience. Only when everyone is willing to understand it in depth can it contribute to the promotion and popularization of a project.
Perfect and robust documentation is the key to developing an excellent open source project, coupled with regular maintenance by relevant responsible personnel, to ensure a friendly user experience. Many open source projects now have a low number of contributors in terms of documentation and websites. Therefore, it is necessary to invest the necessary human time and technology in this area of work.
Why get involved in open source
I highly recommend everyone to actively participate in open source. Why get involved in open source? From my own experience, since joining the Pulsar open source community, I have met many excellent friends. The community has a strong technical atmosphere and high requirements for performance, code quality and other aspects, which drives me to set higher standards for myself and improve Myself, let me feel that the development space of technical ability is unlimited. In addition, participating in open source projects is very helpful for one's career: a high-quality project is a brand in itself, and project experience can "endorse" one's skills.
So how to participate in open source projects and communities? Newcomers who are new to the Pulsar community can start with simple tasks, and documentation is a good starting point. You can evaluate and filter according to document tags, key tags, and your own abilities in the Issue list of the Pulsar project, and choose suitable topics to try to contribute, from shallow to deep.
Becoming a Committer has a long way to go
After becoming a Committer, my sense of responsibility has increased, and I often think that what I have done is not enough. I hope I can contribute more to the Pulsar main project. Therefore, I am studying the main project contribution direction, and I look forward to bringing diversity to the Pulsar community in the future. contribute.
At the same time, I look forward to guiding and cultivating more people to join the documentation and website maintenance team, so as to solve the current problems of low number of participants and insufficient attention in these two areas, maximize the value of documents and websites, and jointly boost the later stage. Project development and user experience improvement.
Thanks
Thanks to the enthusiastic help of the community leaders. Every time I raise a PR, everyone responds positively to help standardize the code and point out deficiencies. We learn together to make progress and collide with sparks of inspiration. I think that's what being involved in open source is all about: a group of people who agree on getting together to do what they love.
Let Apache Pulsar "endorsement" for you
As a global open source project, as of now, Apache Pulsar has 514 contributors, 10.6K+ Stars, and 2.8K+ Forks. Apache Pulsar has been recognized by many users, and the scenarios are becoming more and more abundant, covering a wide range of fields, such as AI artificial intelligence, IoT, live broadcast/short video and social networking, instant retail and logistics, e-commerce, human resources, telecom operations Business, securities trading and financial technology, business intelligence, industrial Internet, basic cloud services... The main users include Zhihu, Tuya Smart, Dada Group, Ping An Securities, Huawei Cloud, Tencent's billing platform Midas, Kingsoft Cloud, Lakala, etc. .
As Li Li said, a project with high gold content can "endorsement" for your career. Participating in the Pulsar community can not only gain recognition from the company and the community, but also make friends with like-minded partners from various fields; it can also improve personal influence and promote personal development. Li Li's personal experience has also proved to everyone that there are various ways to participate in open source, and there is enough space in the community, documents, websites and other aspects for everyone to grow into senior pulsing "stars".
We have provided a participation guide for everyone, and welcome more and more small partners to help the continuous development and progress of the Apache Pulsar project.
Related Reading
- Post-95s Committer He Zhangjian: Talking about Young People's Views on Open Source
- Committer Guo Jiwei Interview: Doing open source is not about charity, and using open source is not about picking wool
- Strength and appearance coexist - Interview with Liu Yu, member of Apache Pulsar PMC
Follow the public account "Apache Pulsar" to get more technical dry goods
Join the Apache Pulsar Chinese exchange group👇🏻
Click to fill out the "New Apache Pulsar Official Website" questionnaire ~
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用。你还可以使用@来通知其他用户。