Curve enters CNCF Sandbox, perfecting the unified cloud-native open source storage puzzle

On June 15, 2022, the Cloud Native Computing Foundation (CNCF) announced that the distributed storage system Curve was officially accepted as a CNCF Sandbox project. Curve is open sourced by NetEase Shufan and provides block storage and file storage capabilities. It aims to feed back the community with NetEase’s distributed architecture and cloud-native practical experience, and fill the gap of high-performance, easy-to-operate, and cloud-native open source distributed storage.

Curve's entry into the CNCF sandbox means that the world's top open source foundations recognize the evolution of NetEase Shufan cloud-native storage technology, and it also verifies NetEase Shufan's deep accumulation in the field of digital basic software and its deep insight into future technology trends. By entering the CNCF sandbox, the Curve community will attract more developers and users to participate in the co-construction, further promote the mature application of the project in cloud-native business scenarios, and deepen the implementation of cloud-native technology.

Project address: https://github.com/opencurve/curve

Curve Project Features

The research and development of Curve originated from the fact that the open-source Ceph storage system was unable to meet the simple needs of NetEase's business development, and grew out of the opportunity for cloud native to quickly land in various businesses. Looking back on 2018, NetEase has completed the full containerization of its e-commerce business, and began to use Kubernetes + Operator to run stateful applications. The improvement of cloud-native storage infrastructure has also been put on the agenda.

Even from the current CNCF Landscape, cloud-native storage projects are still far less abundant than the computing side and the network side, and open source ones are even scarcer (white background in the figure). The basic requirements for storage systems in cloud-native scenarios such as flexibility, ease of operation and maintenance, there are basically no suitable systems on the market to choose from. This is the external driving force for the healthy development of Curve.

Thanks to the Raft consistency protocol and the technical route of the innovative architecture, at present, whether using SATA SSD block storage or NVMe block storage, Curve's random read, write, and latency performance are far superior to the old open source storage system Ceph. The performance stability also has a leading degree close to the sesame relationship.

Compared with OpenEBS, another CNCF sandbox project, which uses container-based block storage open source software written in Go language, Curve covers both block storage and file storage, which is more conducive to the construction of unified digital basic software, and also requires less operation and maintenance management costs. lower.

Curve usage scenarios
Curve can be applied to various cloud native infrastructure platforms as storage bases, such as:

Connect to the OpenStack platform to provide high-performance block storage services for cloud hosts;
Connect to Kubernetes to provide it with persistent storage volumes of RWO, RWX and other types;
As cloud storage middleware, S3-compatible object storage is used as the data storage engine to provide cost-effective shared file storage for public cloud users;
Connecting with PolarFS as a high-performance storage base for cloud-native databases, it perfectly supports the storage-computation separation architecture of cloud-native databases.
In response to the current demand for autonomous and controllable domestic digital infrastructure, Curve has also done a lot of targeted adaptation work. At present, Curve fully supports the domestic Kunpeng CPU + Kirin system, and the software architecture can make full use of and exert the performance of the domestic CPU, hardware and operating system. In addition, the core modules and data structures and data communication protocols of the Curve system are independently designed and developed in China, with more than 200,000 lines of independent research and development code, and the coverage rate of the test code has reached 80%.

Curve Community Progress
Curve was officially open sourced in 2020. It has released 3 major versions in more than a year, completed more than 2,500 commits, and has more than 1,100 stars on GitHub. In addition to NetEase, it has also attracted Hyperfusion Technology Co., Ltd., Simou Technology , Chuangyun Rongda and many other enterprises use it.

In March 2022, Curve became the official ecological partner of the Alibaba Cloud PolarDB community with its excellent performance and strong ease of operation and maintenance, providing a distributed shared storage base for PolarDB for PostgreSQL based on the Shared-Storage architecture, supporting database load acquisition. The benefits of cloud-native rapid elastic scaling.

Thanks
The development of the Curve project today is inseparable from the efforts of every community participant. I would like to sincerely thank every community contributor and user for their trust, support and help, and especially to CNCF colleagues, especially Storage TAG, for their guidance and support for the development of the project. We also welcome more developers and users to participate in the construction of the Curve community. If you have any comments and suggestions, you can submit an issue to us.

Curve Future Planning <br>In the future, Curve will continue to evolve towards the goals of high performance, easy operation and maintenance, and cloud native:

High performance: Provide ultra-high-performance, ultra-low-latency storage bases for cloud-native databases and cloud middleware, helping to implement the storage-computing separation architecture. At the same time, it can also provide a high-performance storage layer for hot data for file storage, helping the implementation of cost-effective architectures.

Cloud-native: It supports the storage-computing separation architecture of various cloud-native applications, supports deployment to private cloud, public cloud, and hybrid cloud environments, and provides a more flexible and convenient storage system for cloud-native applications.

Ease of operation and maintenance: Provide simpler, easier-to-use, and easy-to-debug operation and maintenance tools and diagnostic platforms.

Curve roadmap reference: https://github.com/opencurve/curve/wiki/Roadmap