Wang Yuan was a guest on Alibaba Cloud and said that he discussed the open source and storage ecology of Dao database

What is the importance of open source for ecological docking?

What is the significance of open source ecological construction for To B customers?

Is open source a suitable path to achieve localization alternatives?

Recently, Wang Yuan, Vice President of NetEase, Executive Dean of Hangzhou Research Institute, and General Manager of NetEase Shufan, was invited to participate in Alibaba Cloud's "Big Coffee Talks * Dialogue on Open Source", and was in charge of Alibaba Cloud Database Open Source Campaign and Alibaba Cloud Intelligent Database Product Division. Qu Shan (Huang Gui), the person in charge of the architecture group, had a wonderful dialogue, taking Data Infra (data infrastructure) as an example, and conducted in-depth discussions on a series of issues such as open source ecology, commercialization and localization.

Curve open source storage, intended to support all scenarios

Wang Yuan is a senior practitioner of open source technology. In 2006, when he first arrived at NetEase, he faced the challenges brought by business growth and adopted the solution of sub-database and sub-table to promote the replacement of Oracle by open source MySQL. This predates the massive "de-IOE" movement that followed, and the technology is still working well.

Wang Yuan introduced that NetEase's open source road can be divided into three steps: making good use of open source, enhancing feedback, and independent open source. That is, first, make full use of open source technology to give full play to its value, and then contribute to the community to enhance its imperfections. The self-developed project is open source to solve the problems that the existing technical framework cannot reach.

The open source cloud-native software-defined storage system Curve is the output of the third step. It is NetEase Shufan's effort to break through the performance bottleneck of open source storage. It is also the cooperation between NetEase Shufan and Alibaba Cloud in the database field to provide distributed shared storage for the open source PolarDB for PostgreSQL. the protagonist. Qu Shan said that stability, reliability, and availability are the basic requirements for storage, and database services have higher requirements, such as supporting multi-point mount, flashback query and other capabilities for a write-on-multiple architecture. PolarDB for PostgreSQL Completed the docking with Curve.

The goal of Curve is to create a high-performance, easy-to-operate, and universal storage system that supports all scenarios, including support for PostgreSQL, MySQL and other cloud-native databases, including file storage in addition to block storage. Wang Yuan believes that this It will enable the entire technology stack to play better benefits.

Open source is necessary for the success of the underlying software

In the current field of basic software, especially Data Infra, open source is becoming more and more popular. Wang Yuan believes that open source is a necessary condition for success. Taking Curve as an example, if you want to compete with Ceph, it is impossible not to open source.

Qu Shan also believes that open source is a must for building industry benchmark products. Qu Shan said that commercial products need to pay a relatively high cost to obtain customers, and only by being open can they be connected with various ecosystems.

Qu Shan explained that the use of open standard interfaces for open source software is very important for opening up the ecosystem to realize the integration of basic software architecture systems. The docking of open source software such as PolarDB and Curve is based on standard interfaces, which does not require a lot of cost in practice. energy.

To B technology needs to consider the technology niche

The open source practice of NetEase Shufan started with serving NetEase’s business, but Xingchenhai is in the To B market and provides services to more customers through NetEase Shufan. Wang Yuan said that this requires each technology to comprehensively consider whether it can form a healthy technology ecological chain with the upstream and downstream, develop dynamically, and continuously improve and improve. Each layer has multiple technology options and multiple service providers to provide products. , and finally compose a good solution to the client.

A good technology ecology means having a thriving community with contributors from different industries and enterprises, rather than just one company playing a one-man show, so as to effectively diversify supply chain risks for enterprises.

So, the software is open source, does it affect the promotion of commercialization? Both Wang Yuan and Qu Shan believe that when complex basic software products such as databases are truly applied on a large scale in industrial-level scenarios, enterprises need professional teams to provide technical support.

Wang Yuan said that if you focus on doing a good job of products and build a large enough customer base, your business will naturally grow. "Don't worry, at least before your open source project has reached a certain state, you don't have to pay attention to this matter prematurely."

Qu Shan said that it is also a better way to commercialize products that have been verified through cloud services. Databricks and Snowflake are typical examples.

To solve the problem of stuck neck, open source is better than closed source growth

Under today's trend of replacing basic software with localization and solving the problem of stuck necks, the open source model has gained unprecedented attention, but is open source really more advantageous than the traditional closed source R&D model?

Wang Yuan, who participated in the research and development of the domestic database Shenzhou OSCAR in 2003, was deeply touched by this. He said that Shenzhou OSCAR scored far more than the second place in the evaluation of the Ministry of Science and Technology, but today it seems that the development is not ideal. Although there is still a huge gap between domestic open source databases and Data Infra technologies such as Spark and Snowflake, and it cannot be said to be particularly successful, compared with the development of non-open source domestic databases and the current open source domestic databases in the first three to five years, Wang Yuan believes that open source Even better.

"It can still be clearly seen that the growth process of open source databases at least in the early stage is much higher than the original non-open source (databases), and the probability of success will be greater if you use open source methods." Wang Yuan said.

For more exciting views, please watch the interview video in this issue, scan the QR code below or click the link to the original text.

Video link: https://talk.aliyun.com/live?id=11741877.8159532
Curve project address: https://github.com/opencurve/curve
NetEase Shufan open source homepage: https://sf.163.com/opensource