"The Apache Way" understood by Tencent Cloud senior engineer Cheng Li: Open, collaboration and Win-win.

Open, collaboration and Win-win.

———Cheng Li

ApacheCon is the official global series of conferences of the Apache Software Foundation (ASF). As a prestigious open source feast, ApacheCon has attracted much attention in the open source community and is also one of the well-known activities in the early days of the open source movement.

As early as 1998, before the establishment of ASF, ApacheCon had attracted participants at all levels to explore "tomorrow's technology" in more than 300 Apache projects and their different communities. Also in this session, developers who developed HTTPD services gathered together and decided to establish the Apache Software Foundation.

ApacheCon is held once a year, usually in Europe or North America. It is an excellent opportunity for Apache developers to communicate, discuss, and meet offline. It is also a rare occasion for sharing ideas and exciting ideas. Through hands-on practice, keynote speeches, actual case studies, training, hackathons, etc., showcase the latest development and emerging innovations of the Apache project.

This year, in order to better serve the fast-growing Apache users and contributors in the Asia-Pacific region, the ApacheCon Organizing Committee and the Apache Software Foundation are pleased to announce that the first ApacheCon online conference for the Asia-Pacific region will be held in August 2021. Held online from 6th to 8th.

In order to let everyone know more about open source and ApacheCon Asia, SegmentFault will interview some Track Chairs or conference lecturers to let you know the stories behind the preparations for the conference and the stories of these experts.

Today, what we bring to you is our interview with Cheng Li, a senior engineer of Tencent Cloud.

The following is the content of the interview with Cheng Li by SegmentFault Sifei:

I am PMC in the Apache Ozone community and Committer in the Hadoop community. I currently work on the Tencent Cloud object storage team. Once worked in AWS S3, Huawei Storage and other teams.

---

The story of Cheng Li’s first exposure to open source

The first large-scale investment in open source was in 2019. Hadoop incubated the object storage project Ozone. As an open source newcomer, he invested in the Ozone project and grew up with the project. After leading the design and delivery of major features such as MultiRaft and SCM HA, he became a part of the community. Served a more important role, and also accompanied Ozone to become the top Apache project.

---

When participating in open source and contributing to the Apache community, what are the personal and company gains?

First of all, from participating in the Apache open source community, my own code specifications, code quality, and the ability to cooperate and communicate with community members have increased significantly. A better understanding of the open source community and the meaning of open and win-win cooperation. At the same time, with the Ozone project becoming the top Apache project and Tencent's continuous investment in Hadoop and Ozone projects, some of the big data projects inside and outside the company have also been delivered. Internal and external customer use scenarios, using Ozone, a storage project, helped Tencent find a solution in the use of privatized high-density storage devices, and realized the actual value of external customers.

---

Cheng Li understands "The Apache Way"?

Open, collaboration and Win-win.

---

Cheng Li and ApacheCon Asia

This time I have two talks:

Big Data

Tencent Cloud's three-layer transparent acceleration based on Hadoop-COS builds to help the data lake

After Tencent Cloud Object Storage COS submitted the Hadoop-COS file system plug-in to the Hadoop community in 2019, it was integrated into the cloud-native storage scenario and carried the EB-level data volume of applications such as big data, AI, and containers.

This year, Tencent Cloud COS has made a comprehensive upgrade to Hadoop-COS, adding the three-layer acceleration function of the data lake, accelerating the construction and development of the data lake, and helping the integration of big data and AI architecture. The main sharing of this speech:

1. Hadoop-COS storage and computing separation under cloud native ecology
2. Hadoop-COS three-layer transparent acceleration
3. Data Lake Architecture under Big Data and AI

In this speech, the lecturer mainly shared Tencent Cloud's three-tier Hucang acceleration solution based on object storage COS, which can achieve up to 10 times faster read and write acceleration, and achieves Schema transparent acceleration based on Hadoop-COS.

At the same time, the sharer will also introduce how Tencent Cloud is based on the three-layer transparent acceleration of Hadoop-COS and integrates into the unified data lake architecture of big data + AI.

It is hoped that the audience will have an understanding of Tencent Cloud's data lake acceleration plan and understand the current specific use cases of the data lake in the big data and AI scenarios.

Big Data

How does Apache Ozone use the Raft protocol to complete high-availability solutions and greatly improve throughput?

Apache Ozone will officially become a top-level Apache project in 2020. Ozone supports multiple access methods such as HCFS, fuse, etc. Tencent engineers are outstanding in the Ozone community.

The Tencent Cloud team led the development of a number of important features in the Ozone community. This speech will focus on sharing two of them, MultiRaft and Ozone High Availability. The key features of these two Ozone communities are based on the Raft protocol.

The main sharing of this speech:

1. How to use Java Reflection and Raft protocol to complete the strong and consistent synchronization of Ozone metadata and complete the construction of Ozone high availability
2. How to use the Multi-Raft feature to improve the single-cell throughput of Ozone DataNode
3. How to use the Multi-Raft feature to adapt to high-density hard disk models and integrate Ozone into the privatized data lake solution

The audience can learn from the speech that the Tencent team used Raft and Java reflection to solve the Ozone consistency problem. At the same time, they can also obtain new ideas for optimizing DataNode write performance from the case of Multi-Raft implementation and high-density disk models.

It is expected that users will have an in-depth understanding of how the Ozone project achieves high availability of metadata and optimization of data node performance through Raft.

---

Join us in ApacheCon Asia!

Take part in the feast of the Apache community and witness the evolution of open source.

---

Tencent Cloud

The Tencent Cloud Storage team is currently vigorously developing the data lake ecology, and people with lofty ideals are welcome to join. Interested parties can contact [e-mail]

---

ApacheCon Asia full agenda address:
https://www.apachecon.com/acasia2021/zh/sessions.html

Register address now:
https://hopin.com/events/apachecon-asia-2021