Yang Zhenkun: Looking at the development and rise of distributed databases from the perspective of electric vehicles

Electric vehicles and distributed databases seem to be two unrelated things, but their development paths are very similar: both were invented earlier, but neither became mainstream.

At the beginning of the 21st century, affected by factors such as environmental protection, greenhouse effect and energy shortage, electric vehicles entered people's field of vision again, but their core value is autonomous driving. The technical threshold is very high, and a very complex real-time system is required. Also at the beginning of the 21st century, due to the high concurrency and massive data processing requirements of Internet services, distributed databases were mentioned again, and an application boom was set off. However, the core value of distributed databases is that a single system can perform online transaction processing and online analysis at the same time. Processing capability (HTAP) is also a high technical threshold and requires a very complex real-time system.

If there is demand, there will be supply, and two new products appear on the market: hybrid vehicles and sub-library and sub-table databases. Hybrid vehicles seem to have the advantages of both fuel vehicles and electric vehicles, but they cannot achieve automatic driving due to the inability of the fuel engine to steplessly and smoothly adjust the speed. The sub-database sub-table middleware splits the data into multiple databases, so HTAP cannot be implemented.

This article will analyze and contrast the development of electric vehicles and distributed databases.

01. Survive through cracks

On January 29, 1886, the German Karl Benz (Karl Benz) applied for a patent for a three-wheeled vehicle (see Figure 1) [1], this day is known as the birth day of modern automobiles. In the same year, the German Gottlieb Daimler made the world's first four-wheeled car.

Figure 1: The world's first automobile patent certificate, Carl Benz, 1886

In 1963, Charles Bachman (Charles Bachman) of General Electric Company of the United States developed the network database IDS (Integrated Data Store, see Figure 2) [2], which is considered to be the first database.

Figure 2: Random Access to Data, General Electric Company, 1962

From the beginning of the 20th century to the present, for more than 100 years, fuel vehicles have been the absolute mainstream of automobiles. In fact, electric vehicles were born before gasoline vehicles. French inventor Gustave Trouvé invented the electric three-wheeled vehicle in 1881 and successfully tested it on the streets of Valois in the center of Paris (see Figure 3) [3]. Nevertheless, fuel vehicles have become the first choice in the automotive field due to the significant advantages of fuel oil and engines in terms of power-to-volume ratio and power-to-weight ratio.

Figure 3: The world's first electric car (1881)

Similarly, for nearly sixty years from the birth of database systems, centralized databases were the only option for online transaction processing (OLTP), and despite the emergence of the Ingres Star distributed database in 1987 (see Figure 4) [4], distributed databases The performance of high-speed transactions and the complexity of distributed systems make distributed databases only have some applications in online analytical processing (OLAP) scenarios with very large amounts of data in the past decade.

Figure 4: Distributed database Ingres Star system architecture

02. Time to run

As the saying goes, "When there are no storms and strong rains, one can keep the clouds open and see the moon."

Since the 21st century, human beings have faced severe challenges such as environmental pollution, atmospheric greenhouse effect and energy shortage. There are hundreds of millions of fuel vehicles in the world, and their pollutant emissions, carbon dioxide emissions and energy consumption are innumerable, prompting people to seek pollution-free, zero Emissions, low-energy vehicles.

Thanks to the development of large-capacity batteries and fast charging technology, electric vehicles have once again entered the human field of vision. However, the energy density of a battery of about 260 kWh/m3 is still far lower than the 8600 and 9600 kWh/m3 of gasoline and diesel. In layman's terms, a car needs energy to drive, assuming a certain volume of battery contains The energy can make the car travel 260 kilometers, then the energy contained in the same volume of gasoline and diesel can make the car travel 8600 kilometers and 9600 kilometers respectively, the difference can be seen. Moreover, as the ambient temperature decreases, the battery storage capacity will decrease significantly. These factors lead to the insufficient range of electric vehicles, and the battery that needs to be charged for hours is not comparable to the fuel tank that can be filled in a few minutes.

Therefore, traditional car manufacturers choose to manufacture hybrid vehicles more, that is, adding battery power to fuel vehicles as an auxiliary, using batteries as power to save energy and reduce pollutant emissions during traffic jams or low-speed driving, while normal driving or high-speed driving. Use fuel as power. In addition, some emerging car manufacturers, such as Tesla, have completely abandoned the fuel solution and developed electric vehicles that are purely powered by batteries, also known as pure electric vehicles.

At the same time, the advancement of technology and the popularization of the Internet have forced more and more enterprises to carry out digital transformation. Behind the more convenient interconnection between people and the more intelligent society, there are more and more frequent concurrency of business systems. Access, increasingly large amounts of data processing. The high cost of centralized databases and the extremely limited expansion capabilities of storage and computing are stretched, and enterprises have to seek databases with better cost-effectiveness, storage and computing expansion capabilities.

The development of distributed technology and the popularization of cloud computing have "surfaced" distributed databases. However, due to factors such as more complex distributed databases, more difficult fault location, reduced distributed transaction performance, and insufficient system maturity, traditional database vendors choose the solution of "service sub-database sub-table + middleware" , that is, based on the centralized database, the business is greatly transformed, dismantled, and split, so that each dismantled and split part is suitable for a single centralized database, which is the sub-database and sub-table database.

Of course, there are also some emerging database vendors, such as OceanBase, which have abandoned solutions based on centralized databases and developed real distributed databases, also known as native distributed databases. It is worth mentioning that the native distributed database implements integrated OLTP+OLAP, aiming at online transaction processing and analytical processing in one system, but it is extremely difficult to truly realize the "integrated" solution.

In the eyes of many people, an electric car is a car that replaces the engine and fuel with an electric motor and battery:

● Fuel car = engine + fuel + chassis + body + electrical equipment

● Electric vehicle = electric motor + battery + chassis + body + electrical equipment

If so, traditional automakers, with their extensive experience in mass vehicle production, sales, and service, don't seem to have a problem finding success in electric vehicles.

But in fact, it's not. First of all, in 2020, the market value of Tesla, the king of electric vehicles, surpassed Toyota to become the world's most valuable car company, and then in 2021, it surpassed Toyota, Volkswagen, Daimler, GM The combined market capitalization of the world's major auto companies, including Auto, Ferrari, and BMW. Let’s look at this set of data: In 2021, Tesla’s new car deliveries will exceed 930,000, with revenue of $53.823 billion and GAAP net profit of $5.519 billion; in the same period, Toyota will deliver about 10.5 million new cars, with a net profit of $21.17 billion .

Why does the capital market favor Tesla so much? Why do investors think Tesla has a lot of room to grow?

Obviously, Tesla has passed the life and death line of producing 200,000 vehicles per year, while maintaining a high gross profit margin and a rapid growth rate, which is a prerequisite for the capital market to favor Tesla. Tesla's core competencies include its important technologies such as battery management. Of course, the most fundamental reason is Tesla's Autopilot system:

● Leisure and comfort is the eternal pursuit of individuals, and automatic driving can greatly reduce the labor intensity of car driving.

● Cost reduction and efficiency increase are the never-ending pursuit of enterprises. Autonomous driving not only saves manpower, but also can significantly improve the utilization rate of vehicles (such as high-risk driving environments such as the middle of the night, plateau areas, and other scenarios that are not suitable for manual driving). This greatly reduces the total cost of vehicle ownership.

● The electric motor can steplessly and smoothly adjust the speed, which is a necessary condition for automatic driving. The engine cannot do this, so the gasoline vehicle and the hybrid vehicle containing the engine cannot realize the automatic driving.

● The automatic driving system is a very complex real-time system, which requires a long time and a large number of actual scenes to polish, and the threshold is extremely high. This extremely high threshold, as well as the huge difference between traditional in-vehicle software and autonomous driving systems, make the transition of traditional car manufacturers to pure electric vehicles and the accumulation of autonomous driving technology facing great challenges.

Similarly, in the eyes of many people, a distributed database is to replace the centralized storage and computing hardware of the database with distributed storage and computing hardware:

● Centralized database system = centralized storage + computing hardware + SQL subsystem + transaction subsystem + storage subsystem

● Distributed database system = distributed storage + computing hardware + SQL subsystem + transaction subsystem + storage subsystem

If so, it does not seem to be a problem for traditional database manufacturers to succeed in the field of distributed databases with their rich experience in database development, promotion and services.

This is not the case either:

● Leisure and comfort is the eternal pursuit of individuals. The native distributed database can meet the needs of users OLTP and OLAP at the same time with one system, namely HTAP, which avoids the duplication, conversion and loading of data between the two systems of OLTP and OLAP ( ETL), which greatly simplifies the user's operation and avoids the data consistency problem caused by ETL.

● Cost reduction and efficiency enhancement are the never-ending pursuit of enterprises. The native distributed database has one system and one storage, and the cost is significantly lower than the two storages of the OLTP and OLAP systems, which not only significantly reduces the total cost of ownership for users, but also avoids the Data aging problems caused by ETL.

● The sub-database and sub-table divide the business and data into multiple databases, so the sub-database and sub-table database cannot implement HTAP.

● Centralized database online transaction processing is already a difficult and very complex real-time system to develop. The development of distributed online transaction processing is even more complex and difficult, requiring a long period of time and a large number of actual scene polishing, and the threshold is extremely high. This extremely high threshold, as well as the significant differences between traditional centralized databases and native distributed databases in terms of SQL optimizer, transaction processing, and storage architecture, make it difficult for traditional database vendors to transform to native distributed databases and implement HTAP. .

Having said that, it has to be admitted that Tesla's autopilot system has yet to be further matured, and some people still have doubts about the safety and reliability of autopilot; the HTAP function of the native distributed database has yet to be further matured , some people still have doubts about whether HTAP can actually be applied to production systems. Therefore, the user experience becomes the best advertisement. Tesla has gained a lot of "special fans" because of its long-distance battery life, fast acceleration and new experience of autonomous driving. It is also the only electric vehicle brand in the world that has obtained the NHTSA five-star crash safety rating; the native distributed database has won many awards from banks, securities, energy, electricity, social security, etc. due to its high availability, low cost, intelligent operation and maintenance, and high machine utilization. The trust of the industry has been applied to the core business scenarios of the enterprise.

03. Open source

If the above content is to compare the growth trajectories of electric vehicles and distributed databases from the macro environment of social development and technological progress, they are so similar, then at the micro level, such as the development of a certain product, the development of electric vehicles and distributed databases The shape of the track is also very similar, and both follow the "big to small" development strategy and an open source and open mentality.

From big to small

Take Tesla as an example for electric vehicles, from sports cars to high-end SUVs, to compact SUVs and cars, from niche to public; OceanBase, the representative of the native distributed database, has gradually evolved from a cluster version to a stand-alone version. From large businesses to small businesses.

What is the development logic behind this?

When a car is just born, there is no order quantity, and it will not make profits quickly. Therefore, it is necessary to make products with less mass production and large profits. Later, when the cost is controlled, mass production of products with small profit margins. Also for databases, such as OceanBase first landed in a scenario with massive data such as Alipay. Since large businesses do not care so much about cost and machine configuration, it is easier to use large-capacity hard drives. Use a high-configuration environment to support the business, and then gradually optimize it. To achieve the goal of supporting large business even with small configuration.

open source

In 2014, Tesla announced the opening of all patents. When everyone doubted the motive behind it, Tesla had realized that autonomous driving was a disruptive innovation to the automotive industry, even if other companies had the same electric vehicle manufacturing capabilities, especially Sla is also confident that it can secure a leadership position in the electric vehicle market. At that time, people's recognition of electric vehicles was still very low, and what Tesla needed was the rise of the electric vehicle industry.

In 2021, OceanBase announced that the source code will be open source, sharing the core of the world's leading native distributed database with the outside world. Also faced with external doubts. OceanBase is committed to open source. On the one hand, it hopes to focus on user value and solve the specific problems of users. On the other hand, it hopes to build a domestic distributed database ecosystem with the outside world, and surpass "IOE" from another track.

Their attitude towards open source technology is as described in the book "Silicon Valley Iron Man": "When Musk announced in 2014 that Tesla would disclose all of its patents, analysts were trying to determine if he was just showing off. Or is there a hidden motive or a trap in it. But Musk's decision is so candid, he wants people to build and buy electric cars. Musk believes that the future of mankind depends on this. If Tesla's patents are disclosed, it means other companies Being able to build electric cars more easily then it's good for humanity and the ideas should be free. Cynics will surely laugh at his point, but Musk has a plan to do it, he's explaining his own It's sincere when it comes to thinking, and it's extremely sincere."

04. Facing the future

Today, in the automotive sector, pure electric vehicles are gaining more and more recognition:

● Tesla Model 3 standard model, the manufacturer's nominal cruising range reaches 675km, and the super charging pile can increase the cruising range by a maximum of 279km in 15 minutes. The cruising range and charging speed are close to those of fuel vehicles.

● At the end of 2021, Volkswagen announced that pure electric vehicles will account for 25% in 2026, 50% in 2030, and close to 100% in 2040.

● In 2019, BMW's R&D director was still angry about pure electric vehicles: "Europe does not need pure electric vehicles, and fuel vehicles can exist for at least 30 years." However, at the end of 2021, BMW announced that pure electric vehicles will be launched in 2030. at least 50%.

● At the end of 2021, Toyota announced that it will launch 30 pure electric vehicles by 2030.

Today, in the field of databases, native distributed databases have made great progress, and more and more benchmarks [5] are occupied by native distributed databases:

● The online transaction processing TPC-C performance list of the International Transaction Processing Performance Council TPC, its top position was occupied by the OceanBase native distributed database in May 2020, with a performance of about 707 million tpmC.

● The decision support processing TPC-DS performance list of the International Transaction Processing Performance Council TPC, the 10,000GB native distributed database was occupied by Alibaba Cloud AnalyticDB in June 2020, with a performance of about 19M QphDS; the 100,000GB top position was native to the Databricks Photon Engine The distributed database is occupied in November 2021, with a performance of about 33M QphDS.

● In the decision support processing TPC-H performance list of the International Transaction Processing Performance Committee TPC, the top positions of 10,000GB, 30,000GB and 100,000GB are all occupied by the EXASOL native distributed database in June 2021, and the performance is about 23M QphH.

In addition, native distributed databases have been increasingly recognized and widely used in the industry. In November 2020, the People's Bank of China officially released the financial industry standards for distributed databases, including "Technical Architecture of Distributed Database Technology Financial Application Specifications", "Security Technical Requirements for Distributed Database Technology Financial Application Specifications" and "Distributed Database Technology Financial Applications" Standardizing Disaster Recovery Requirements" to guide distributed database practices in the financial industry. In recent years, not only the Internet industry has adopted a large number of native distributed databases, but more and more enterprises and institutions in the banking, insurance, securities, communications, energy and social security industries have also adopted native distributed databases.

Autonomous driving is a subversive innovation of pure electric vehicles to the automobile industry, and pure electric vehicles are an inevitable choice for the development of automobiles; HTAP is a subversive innovation of native distributed databases to the database industry, and native distributed databases are an inevitable choice for database development.