HUAWEI CLOUD GaussDB: Give full play to ecological advantages and cultivate application-oriented DBAs

Abstract: GaussDB’s primary task is to address Huawei’s business continuity needs, and at the same time to ensure the continuity of the business of customers who use GaussDB. Therefore, we insist on strategic investment, insist on building the whole from every line of code and starting from the ecology. Database system.

This article is shared from Huawei Cloud Community " Huawei Cloud GaussDB: Leveraging Ecological Advantages and Cultivating Application-oriented DBA ", author: GaussDB database.

Introduction:

With the advent of the 5G Internet era, all walks of life are gradually increasing their reliance on databases. Due to the late start of the development of the database industry in China, the market share of the database has long been controlled by Oracle, Microsoft and other American companies. However, with the continuous evolution of the domestic IT technology stack, the goal of "IOE" has changed from a corporate goal to This is the goal of the entire industry, that is to say, my country must make information system data run on self-developed systems to prevent a series of serious consequences caused by data loss. From the perspective of the international situation in the past two years, going to IOE has become the only way for the development of domestic databases.

I am very pleased to see that the domestic database has gradually formed a pattern of blossoming flowers. This time, Mr. Su Guangniu, the person in charge of Huawei cloud database business, was invited to communicate with netizens about the future development of domestic databases. At the same time, we also collected some socially discussed issues related to domestic databases that are more concerned by netizens. We hope to give the public some in-depth unique insights from the perspective of the person in charge of database products.

Su Guangniu, Head of Huawei Cloud Database Business

Responsible for the strategic formulation and development of Huawei Cloud database business, the development, operation, operation and maintenance and delivery of database cloud service products and solutions.

—The following is the full text of the interview —

Q: Introduce your company's database products and advantages, and who are the main target users?

Su Huawei database GaussDB is first positioned as a cloud database, and secondly, positioned as a distributed database. The combination of cloud and distributed is the future of the database. The primary task of GaussDB is to solve Huawei’s business continuity requirements and to use the database without any restrictions. Therefore, from the very beginning, it has been thinking about how to be completely autonomous and controllable, from every line of code, from the ecology, from the reliability, availability, and Starting from performance and other requirements, combined with cloud technology to create an enterprise-level distributed database.

Under the current situation, the above positioning of GaussDB is more in line with the needs of important industries such as finance, pan-government, telecommunications, energy, transportation, medical care, and logistics, and can truly help customers achieve digital transformation.

GaussDB is a product series that embodies the multi-mode design concept in the overall architecture design. Specifically:

• Relational Database:

1. GaussDB (foropenGauss), a distributed database based on the openGauss ecosystem, is an autonomous ecosystem distributed database
2. Based on the distributed database GaussDB (forMySQL) of the MySQL ecosystem, we are 100% compatible with MySQL, allowing more customers to migrate seamlessly. At the same time, this is also a distributed database, not a stand-alone primary and backup MySQL.

• Non-relational NoSQL database:

1. Document-based distributed database GaussDB (forMongo), 100% compatible with Mongo interface, solves the open source protocol risk, and uses 3 copies to solve the reliability of 9 copies
2. The distributed database GaussDB (forInflux) based on the timing protocol, the distributed solution solves the open source capacity problem of only active and standby
3. GaussDB (forCassandra) based on wide table Cassandra solves the serious problem of GC jitter in Java, and the data is strongly consistent
4. GaussDB (forRedis) based on the KV interface is currently 100% compatible with Redis, and can be compatible with more KV protocols in the future. The data is strong and consistent, and the ultra-large capacity 10T+ solves the problem of open source main and backup disconnection.

In addition, there are some important components, such as: data migration-DRS (DataReplicationService); application structure conversion-UGO (Database and Application MigrationUGO) (Yugong Yishan); distributed database middleware-DDM (DistributedDatabaseMiddleware), middleware this The layer is fully developed independently, so that the whole has the ability of distributed <sub-database and sub-table> to meet the demands of customers who need the MySQL ecosystem but do not have high credibility requirements.

HUAWEI CLOUD GaussDB full-stack products and solutions are designed to provide customers with peace of mind in choosing models (independent ecology, open source ecology, NoSQL, distributed and unified architecture, excellent performance), and peace of mind in migration (from online data migration to application structure conversion , So that the worries of migration no longer exist), ecological worry-free (regardless of whether it is based on an open source ecology or the requirements for an autonomous ecology, it can be met).

Q: What achievements has your company's database achieved so far?

Su GaussDB gathers global resources, relies on research institutes distributed in 7 regions around the world, about 2000+ database/data warehouse/big data experts and professionals in advanced kernel engines, algorithms, performance, etc., and continues to invest in 10 strategies. for many years:

1. Market share: In the financial, government and enterprise market, the local deployment of GaussDB (that is, the HCS solution) achieved the first market share of domestic databases (source: IDC China Relational Database Software Market Data Tracking Report in the second half of 2020); at the same time, the global DBMS market Share entered the top ten (Source: GartnerMarketShareAnalysis:DatabaseManagementSystems, Worldwide, 2020)
2. Benchmarking customers: It has obtained large-scale commercialization in 1500+ financial government enterprises and pan-Internet benchmarking major customers. Taking the financial industry as an example, GaussDB has cooperated with Industrial and Commercial Bank of China, China Construction Bank, China Merchants Bank, Postal Savings Bank, Agricultural Bank, Bank of Communications, and China CITIC Bank. , China Everbright Bank, Minsheng Bank, Hua Xia Bank, Centaline Bank, Yongan Insurance, Zhonghui Property Mutual Insurance Co., Anxin Insurance, Haitong Securities, and many other top banking, insurance and securities customers have developed extensive cooperation.
3. Academic research: Huawei has successively published 36 database papers at three top international academic conferences, SIGMOD, VLDB, and ICDE. Especially in the AI-Native direction, "LearnedCardinalityEstimationforSimilarityQueries", which was newly selected into the SIGMOD international top conference in 2021, innovatively proposed model segmentation and data segmentation solutions to solve DNN training problems.
4. University ecology: Jointly constructing a "smart base" base with the Ministry of Education, GaussDB has established cooperative relationships with 72 domestic double-class universities, and at the same time opened GaussDB database courses in 54 universities under the "New Engineering" project of the Ministry of Education. In higher vocational and technical colleges, through practical training to cultivate social-oriented applied talents. In addition to the opening of the course, Huawei Database regularly conducts teacher training every year, and publishes the GaussDB database teaching materials and teaching aids in conjunction with professors from many top universities, and continues to improve the talent training system of the GaussDB database!

In addition, the GaussDB database holds a database performance challenge competition every year, sets up an internship plan and talent recruitment plan, and provides internship opportunities for Huawei and industry partners. From theoretical teaching, hands-on experiments, competition experience, professional certification, and full-process support for talent recruitment, we will cultivate and transport GaussDB talents for a long time!

5. Developer ecology: Huawei contributed the core capabilities of GaussDB to form an open source version of openGauss and an openGauss ecosystem. There are currently 1,700+ developers participating in the openGauss open source project, 20,000+ community users, and 290,000 from 506 cities in 74 countries around the world 10 manufacturers have released their own-brand commercial database products based on the openGauss open source kernel, and 43 leading companies including Minsheng Bank and China Telecom have joined the community.
   Based on the Huawei Cloud Developer Community, it provides 39 GaussDB technical courses, releases and updates GaussDB primary and intermediate training certifications; establishes cooperative relationships with domestic 10+ databases and developer communities, actively supports and participates in online and offline technical activities, and The industry's common prosperity of the database community has empowered more than 150,000 developers.
6. Partner ecology: The development of the database industry is inseparable from the support of upstream and downstream ecological partners. After testing and certification, HUAWEI CLOUD GaussDB has formed a joint solution with 100+ partners’ products, and recommended to HUAWEI CLOUD carefully selected malls to share market opportunities , Covering:
   a) Eco-tool partners: Heterogeneous data synchronization tools and database disaster recovery tools such as DSJ, British side, Shujian, Aerospace One System, Dingjia, etc.
   b) Industry solution partners such as finance and government: such as Shenma Rongxin, Everbright Technology, etc.
   c) Consulting service partners: massive data, cloud and Enmo, etc.
   d) SaaS partners: Kingdee, Yonyou, Zhenyun, Fanwei, Zhiyuan Internet, etc.;
   e) Infrastructure partners: such as Kylin OS and Tongxin OS

Q: What are the future plans for your company’s database products, or the direction of key breakthroughs?

Su future, Huawei Cloud Database will continue to invest and deploy in the following aspects:

• do a good job in database ecology: database must follow HUAWEI CLOUD into thousands of industries, and it is inseparable from the construction of database ecology. In addition to building an autonomous and controllable ecosystem for openGauss, we are also actively embracing and fully compatible with mainstream industry ecosystems such as relational MySQL, PostgreSQL and non-relational Mongo, Redis, Cassandra and InfluxDB. In terms of partner ecosystem construction, we have certified multiple ISVs as our partners to provide database cloud solutions such as migration and disaster recovery to provide services to more customers.
• promotes joint innovation: financial industry is a typical industry with extremely stringent requirements on database stability, reliability, performance and other aspects. There are rich application scenarios such as hundreds of millions of users and high-concurrency and massive transactions. Head customers such as Huawei and ICBC insist on long-term Lianchuang helps the GaussDB database to quickly incubate and improve its capabilities through these rare harsh and rich scenarios and applications, and also help these companies complete tasks such as digital transformation.
• serves Huawei’s own business: Huawei’s GaussDB series database has already carried the services of Huawei’s terminal cloud service and internal process IT on a large scale. These two services of Huawei have demanding requirements. GaussDB has helped Huawei achieve digital transformation of both traditional and new businesses. This is a very good model.
• core competitiveness building:

1. Multi-Region, Multi-Activity: Support for nearby access to global enterprise data, global multi-activity disaster recovery, and multi-point data writing;
2. Software and hardware collaboration: Full support for diversified computing power, with E2E R&D capabilities from computing power to servers, storage, operating systems, and databases, and give play to the unique advantages of database software and hardware collaboration and performance tuning;
3. Enterprise-level distributed capabilities: Transactional data capacity ranges from M to G, and then to T, tens of T, which determines that distributed databases are the general trend. HUAWEI CLOUD database focuses on creating extreme reliability and scalability for distributed scenarios;
4. Data security and credibility: As all walks of life attach importance to data security and privacy protection, GaussDB will eliminate the risk of data leakage for customers through a fully confidential database.
5. AI-Native: Tens of thousands of servers, thousands of database instances, 100+T of data, and efficient management of data under the scale of massive nodes are problems that must be faced. Huawei is committed to building a cloud database with self-awareness, self-recovery, and self-optimization capabilities to achieve a high degree of autonomy and reduce database operation and maintenance costs

Q: Regarding the database industry, do you agree with the phrase "the DBA gains the world"?

Su essence, the meaning of this sentence is: the one who gets the ecology gets the world. DBA can be cultivated, a good ecology is easier to cultivate DBA, so ecology is the most important. The current situation of the general environment has given databases and ecological opportunities. We need to create an open, easy-to-learn and easy-to-use ecosystem. For example, openGauss. We will open source from 2020.6.30, starting with every line of code, and at the same time as much as possible Compatible with an ecology, which makes it easier for everyone to learn.

At present, 10 ISVs in China have built their own databases based on openGauss. I believe that more and more companies and individuals will feel the ecological advantages of openGauss.

Q: Are there any good learning suggestions for novices in learning GaussDB database?

Su GaussDB has launched a series of learning materials to help everyone learn and use GaussDB from scratch, including:

• Getting started with GaussDB: Register with HUAWEI CLOUD, enter HUAWEI CLOUD Academy, and find the "Database Learning Path" in the "Learning Path"; at the same time, apply for a GaussDB instance for commissioning, combine with actual combat, and be familiar with cloud database solution design and tuning;
• GaussDB Premium Course: Enter Huawei Cloud Academy and find "Database" in the "Online Course". There are videos carefully recorded by kernel experts. Each issue focuses on 1 topic and explains 1 business problem in detail.
• GaussDB MOOC: Log on to the China University MOOC website, jointly developed by the team of Professor Li Yanling from Northeast Normal University and the GaussDB expert team, "Database Principles GaussDB Cloud Database", has been launched;
• Training and certification: HUAWEI CLOUD has simultaneously launched GaussDB elementary and advanced courses and certifications. Enter HUAWEI CLOUD Academy and choose "Career Certification".
• At the same time, it is also recommended that you download openGauss and purchase "openGauss Database Core Technology" to understand the core of the database in combination with the code, contribute to the community, increase your influence, and become a master.

The Huawei Cloud Database product team continuously launches new online learning courses, holds database training camps, challenges, etc. Please follow the WeChat public account "GaussDB Database" for the latest activity information.

Q: At present, there are different opinions on database open source at home and abroad. Some domestic databases have gradually been open sourced. Is open source good? And whether open source can bring intuitive benefits.

Su Guangniu: The database is an ecological industry, which requires patient and continuous investment. Huawei actively participates in and promotes the industry ecosystem. On June 30, 2020, Huawei will open source openGauss, and build an open source ecosystem with partners, universities and developers, and encourage capable partners to develop its own-brand database products based on openGauss, as the upper layer The application provides more database choices, and the industry will prosper the database industry ecology together with the industry.

GaussDB (foropenGauss) is a commercial version of Huawei that has enhanced distributed capabilities based on the openGauss ecosystem and some cores. Through Huawei Cloud and Huawei Cloud Stack (HCS), it can meet the high performance, high reliability and high security of government and enterprise customers for distributed databases. Demand.

Open source's help to the database ecology is, overall, positive. But open source requires real open source, secure open source and compliant open source. Open source involves a large number of third-party reliances, including patent optimization and adjustments. It is not open source that is open source; therefore, in this sense, open source is also responsible. Open source is not the end, nor is it a forced response. Open source needs to help more partners to really use it.

At the same time, Huawei is also actively embracing, compatible and supporting mainstream ecosystems in the industry such as MySQL, PostgreSQL, and non-relational databases Mongo, Redis and other ecosystems. We will combine these software ecosystems with Huawei's distributed storage, software and hardware, so that customers can both To obtain ecological benefits, but also to obtain higher performance and reliability, this is what we believe is the database product that truly meets the needs of customers.

Q: What do you think are the current difficulties in database localization, or what are the main bottlenecks in the development of domestic databases?

Su One of the core difficulties of the current database industry is talents. In fact, there is a lack of talents in the root technologies of chips, operating systems, and databases. One of the reasons is that there has always been more attention to application development, such as many College students mainly learn high-level development languages such as JAVA as the main language for their future careers, but these root technologies are generally C language. These system-level software require a deep mastery of C language. Just this one makes many people not choose system level. At the same time, the entry level of these root technologies is very high. It may be possible to develop an APP within 24 hours after learning the JAVA language, but the database is not good, especially the development of the kernel may take half a year or even longer. It will make more people retreat even after entering; finally, the transaction database has been very important for a long time, and it is the core of the IT system, but the actual revenue of the database is not high. There are complicated factors in it, but it does make Many companies are discouraged because the benefits may be invisible in the future, and it is difficult to obtain talents, which makes it difficult for many companies to stick to it for a long time.

Since the beginning of its development, GaussDB has been deploying talent training and acquisition based on multiple regions around the world. At the same time, it insists on long-term strategic investment to solve the adverse effects of the above two factors.

Q: Let’s talk about your prospects and messages for the future of domestic databases.

Su in the current environment has brought opportunities to Chinese databases, so overall, it is a good thing for database practitioners. We know that the general theoretical framework and architecture of the database has actually gone through 40 years of development and is basically relatively mature. The is the difficulty of engineering, so this is the time to test endurance and perseverance. Long-term adherence to strategic investment is The most basic guarantee is ; at the same time, the current open source software provides a prototype model for the development of the database kernel, but this is directly used for some simple enhancements and hopes to become an "enterprise-level" database. In fact, there is a big gap. For the sake of customers, a lot of work needs to be done; finally, domestic databases are autonomous and controllable, and everyone in each enterprise may have a different understanding of autonomy and control. For example: through a certain open source software + one Whether middleware is considered autonomous and controllable or a distributed database at the end, it may be that the benevolent sees the benevolent and the wise sees the wisdom. However, in a responsible attitude for the service enterprise, we must ensure our business continuity (adhere to the database , 10 years and 100 years still do database), but also responsible for customers, in any case, the software developed by oneself meets the requirements of safety and compliance, the customer's business can be continuous.

The primary task of GaussDB is to address Huawei's business continuity needs, and at the same time, to ensure the continuity of the business of customers who use GaussDB. Therefore, we insist on strategic investment, insist on building the entire database system from every line of code and starting from the ecology.

Click to follow and learn about Huawei Cloud's fresh technology for the first time~