Li Feifei: Database Industry in the Era of New Technological Changes

Introduction to Cloud computing will change the database landscape

Recently, Li Feifei, head of Alibaba Cloud's Intelligent Database Division, delivered a keynote speech on "Database Industry in the Era of New Technological Transformation" at a media communication meeting.

Li Feifei said that cloud databases have become the most important development direction of databases. From the perspective of the development of the international and domestic database industry, this trend is already very obvious. In his view, in the period of great changes in new technology, China's database market is bound to undergo a series of significant changes.

Cloud's development strategy, he said that as China's number one database vendor, 16144672e852fd Alibaba Cloud is ready to fully serve the government and enterprise markets, and will continue to make efforts in this market.

The following is the core content of the speech:

The future of database: cloud native + distributed

First review the development history of the entire database.

In the past few decades, the scale of IT infrastructure has undergone tremendous changes. From the early mainframes and minicomputers, to X86 servers, to the Internet, to the evolution of cloud + terminal, the number of underlying devices has increased from dozens to several. The exponential growth of millions of units.

This change in infrastructure provides a solid foundation for the evolution of the upper-level database. Just as species evolve with changes in the environment, when the basic computing resources such as computing, storage, and networks continue to evolve, the database systems on these infrastructures also need to continue to evolve accordingly.

Different infrastructures correspond to different application scenarios and different historical stages of database development.

In the past few decades, the database has gone from the early relational database, to the later data warehouse and relational database cooperation, the subdivision of the two fields of OLTP and OLAP, to the open source database in the back, and the development process of today's cloud database. .

DB-Engines has in-depth tracking analysis on the database field. At present, there are nearly 400 different database products on this website. This number clearly shows that the database market is very competitive.

The databases on the DB-Engines website are divided into different types, including relational databases, graphs, documents, analytical databases, etc., respectively corresponding to the market segments of different industry characteristics and customer needs. In addition to traditional commercial databases, the development of databases in the open source community is also very rapid.

DB-Engines has an index-Popularity Trend, which is calculated through various indicators. The index shows that the popularity gap between open source databases and commercial databases has rapidly narrowed.

All this shows that new changes are taking place in the database industry. The authoritative market research organization Gartner elaborated several very important trends in related reports:

First, the data scale has exploded. We are in an era of big data, no doubt.

Second, the demand for real-time data production and processing is becoming stronger. This is why Alibaba Cloud today proposed cloud native database 2.0 to provide one-stop data management and services.

Third, the intelligence of production and data processing. Nowadays, the user's data processing no longer stays in the traditional transaction processing, transaction and simple or aggregate analysis and query, but hope to get more insights from the data and use data to drive intelligent business decisions. This also poses new challenges for the subsequent development of our database.

Finally, data is accelerating to the cloud. This is a very obvious industry trend. Although there are various forms such as public cloud, hybrid cloud, and private cloud, various industries are convinced of cloud deployment methods. According to Gartner's prediction, by 2023, 75% of the world's databases will be running in the cloud.

The database system architecture is also evolving rapidly. The earliest database was a single-node architecture, and both traditional commercial databases and open source databases were based on this architecture.

In actual application deployment, the advantage of this database is simple deployment and simple development. But there are also shortcomings. Firstly, it cannot provide high-availability, main-standby operation and maintenance, and high-availability switching, which is very complicated and faces very large technical challenges; secondly, it cannot quickly expand capacity. For example, in a small warehouse in my backyard, when the data volume or concurrent processing requirements exceed the capacity, there is no way to quickly expand the capacity.

In order to solve the problems of elastic expansion and high availability, the industry has derived two core technical routes, one is the cloud-native technical route, and the other is the technical route of distributed database.

What exactly is cloud native?

The essence of cloud native is to pool resources. Just like every household digs a well in the backyard, today, instead of digging a well in the backyard, it will turn on the faucet, use it according to the quantity and use it with flexible billing. There is a large pool, which is called resource pooling, which is actually the same as using computing and storing network resources in cloud computing.

The core concept of cloud native database is to use the essence of cloud computing and use virtualization technology to pool core resources in the database to solve the challenge of resource elasticity and high availability. There is also a core concept-the separation of storage and computing. By decoupling storage and computing, they can meet customer business needs flexibly and flexibly.

In addition to cloud native, another technology direction is distributed.

Through a distributed approach, computing and storage can be horizontally expanded to meet the large data volume and high concurrency requirements of the business. Of course, this also has challenges. For distributed transaction processing, the database still needs to meet the characteristics of ACID atomicity, consistency, isolation, and durability. In a distributed architecture, it is very challenging to ensure efficient processing of reads and writes.

The future development trend of databases must be to combine cloud-native capabilities and distributed capabilities into one, providing not only distributed horizontal expansion capabilities, but also cloud-native elastic high-availability capabilities. Combining the two is also the direction for Alibaba Cloud's cloud-native database represented by PolarDB to further explore and develop.

Cloud computing changes the database market pattern

Alibaba Cloud Database has continued to cultivate and accumulate in the cloud native field for many years, and we have made some progress. Gartner predicts that 75% of databases will be run in the cloud by 2023.

Gartner conducts the Magic Quadrant every year to evaluate important tracks in the cloud computing IT field. Before 2013, the database leader quadrant was basically occupied by several traditional giants, Oracle, Microsoft, IBM's DB2, and SAP.

By 2020, the market has undergone earth-shaking changes. An obvious feature in overseas markets is that in addition to traditional database giants, cloud vendors such as AWS, Google, and Microsoft have also entered the leader quadrant.

It is worth mentioning that the current Microsoft is completely different from the Microsoft of the year. Microsoft's entry into the leader quadrant this time is due to the rapid improvement of its cloud database service capabilities, which is extremely competitive. Combining the advantages of its offline database and cloud database two tracks, Microsoft has achieved a surpassing Oracle in the overall market share.

Alibaba Cloud Database entered the global database leader quadrant for the first time in 2020. This is also the high recognition given by Gartner after in-depth evaluation of product maturity, technological advancement, customer satisfaction, and market share.

Image Source: Extra Large

The figure above is a comparison of the latest global database market share ranking ten years ago and 2020. The global database market share list includes the market share of cloud databases and the market share of traditional offline databases. Combined analysis of the two, Oracle has always occupied the first place in the past. In 2020, Microsoft will surpass Oracle for the first time.

This is actually the rapid growth of Microsoft's use of SQL Server and Azure cloud database. The combination of the two tracks has achieved the overtake of Oracle. This is a very historic milestone event, which once again highlights the importance of cloud database in the new environment. Importance of vendors and database markets.

In addition, in the past few years, Alibaba Cloud's market share has continued to increase and has reached seventh place in the world. If you only look at the cloud database track, Alibaba Cloud ranks fourth in the world, behind AWS, Microsoft, and Google.

New historical opportunities in the database industry

Although the database is a track that has been developed for forty years, it is not a track that sticks to the rules. I think at this time node today, the database is still a very dynamic track, gestating a lot of new market opportunities and new explosive points.

First, the database is the largest track in the basic software category. There are many basic software, but the database is the most core category in the basic software, and it is also the category with the largest market volume.

Second, from a developer's point of view, more than half of developers need to use a database, and its importance is beyond doubt.

There are several new opportunities for the development of China's database industry, which can be called the four new-new policies, new faces, new technologies, and new opportunities.

First, the new policy. This is mainly because the state has issued a series of policies, which has raised the importance of basic software to an unprecedented level. The country strongly encourages our Chinese companies to enhance the competitiveness of domestic databases through open and fair market competition. From the perspective of technology level and market share, the country also provides a very good environment.

Second, new faces. The traditional previous generation of domestic database vendors were called the Four Little Dragons. Today, the racetrack of domestic databases is flourishing, and more new faces have emerged, including cloud service vendors represented by Alibaba Cloud, as well as vendors specializing in enterprise-level database software. Wait.

Third, new technology. The reason why we have this opportunity today to challenge the position of traditional database vendors in the market, and achieve beyond, is essentially that the environment in which we live has undergone tremendous changes.

Just like Darwin's "Theory of Evolution", the prerequisite for species to evolve rapidly is that the surrounding environment changes. The ability of new species to adapt to the environment is faster than the dominant old species, and it will evolve faster.

The emergence of new technologies, such as cloud-native, distributed, intelligent, multi-mode data processing for the Internet of Things, security and trustworthiness, new hardware, and open source. These dimensions are today and for a long time in the future. Various database vendors will A racetrack with fierce competition.

Finally, new opportunities. More and more enterprise-level users will choose the cloud deployment method, which provides us with a very good opportunity to enter the traditional government and enterprise market.

Alibaba Cloud Database is not only a leader in the public cloud market, but is also fully prepared to serve offline traditional government and enterprise markets. In fact, we have served a large number of traditional government and enterprise customers, and use cloud native and cloud computing methods. , Brings them technological dividends that were not available in traditional deployment methods before, not only reducing IT costs and ensuring stable operation, but also bringing new business value and new business opportunities.

Cloud Native Database 2.0: One-stop data management

Review the development history of Alibaba database:

From 2005 to 2009, during the period of rapid e-commerce development, Alibaba Group had the largest Oracle cluster in Asia;

From 2010 to 2013, Alibaba successfully replaced traditional commercial databases with independent open source AliSQL;

In 2014, Alibaba began to deploy its self-developed database because the business pushed us forward. For example, in the Double Eleven e-commerce scenario, there are needs for multiple activities in different places, hotspots, and seckill. We see the value of self-developed databases. So we firmly invested in the independent research and development of the basic software of database;

From 2017 to today, through years of accumulated database technology, we have provided a wealth of database services, such as cloud database hosting services RDS, cloud native database PolarDB and distributed version PolarDB-X, and cloud native data warehouse AnalyticDB, these core Products form a matrix of our database products and services.

Through technological iteration, in the era of cloud native database 2.0, we hope to provide one-stop data management and services.

Alibaba Cloud Database has come to the No. 1 position in the Chinese database market without any dispute. On the global stage, it has the ability to compete and challenge to a higher level and higher market share, and has entered the Gartner global database leader quadrant.

In addition, according to IDC's latest research report on China's relational database market share, the combination of public cloud and offline statistics shows that the market share of Alibaba Cloud database accounts for 28%, surpassing the second place by ten percentage points. In the latest Forrester data warehouse report, the cloud-native data warehouse AnalyticDB has also entered the global strong leader quadrant. In the industry's standard benchmark test, it has achieved a number of performance and cost-effective results.

We have not only made great breakthroughs and achievements in the commercial market, but have also been highly recognized by the country. The core product of cloud native, the cloud native database PolarDB, won the first prize of the Chinese Institute of Electronics for Science and Technology Progress last year, which is the highest award of the Chinese Institute of Electronics. Our database is also the only product of the selected database, which reflects the recognition of our new technology and new scientific research strength from the national level and from the perspective of scientific research.

The cloud native data warehouse AnalyticDB also won the first prize of Zhejiang Province Science and Technology Progress. We have adapted to x86, ARM and other hardware and operating systems, so that we can better serve the enterprise market and customers in China.

Looking to the future, I think that cloud-native databases have reached the stage of market development mentioned by Gartner, and they have begun to face new requirements such as real-time data, integration of online transaction analysis, and intelligent data processing.

Therefore, we propose cloud native database 2.0 to provide one-stop data management and service capabilities. Through data production and integration, to real-time data processing and storage, to data analysis and discovery, and finally to data development and management, providing the entire chain Road data life cycle management.

On this basis, we provide customers with industry-oriented solutions, that is, solutions that can generate industry value for different customers, and finally cooperate with our ecological partners to serve customers, from distribution ecology to ISV integration, to The service ecology, and then to the developers, form a complete set of data full-link management and service capabilities.

Finally, thank our customers. Our customers are distributed in all walks of life, from the pan-Internet industry to traditional government and enterprises, including transportation, energy, power, banking, insurance, securities and other industries.

We are very proud to be able to create customer value through Alibaba Cloud database products and services, as well as continuous iterative innovation. By providing customers with high-quality database services and helping them achieve business progress and success, our customers will serve thousands of households and ultimately contribute to the development of the entire society.

Copyright Notice: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users. The copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.