China, the United States and Europe are paying attention to what is the "data infrastructure" that has become a national outlet?

Facing the east wind of "new infrastructure", among many dazzling terms and concepts such as 5G, AI, and industrial Internet, many people have found that the exposure rate of the concept of "data infrastructure" has rapidly increased from low to high, which is eye-catching. .

Especially this year, not only the Chinese government has repeatedly attached great importance to the construction of data infrastructure. The European Union, the United States and other major economies in the world have also frequently mentioned this concept in their respective plans and policies. "Data infrastructure" has become a national-level "outlet" in the world.

In the era of big data, data is a precious resource, and is even called the "fifth major factor of production". Countries, societies, companies, and even families and individuals are all in the midst of digital transformation. Data infrastructure is the inevitable result of the transformation and evolution of traditional IT infrastructure.

However, "data infrastructure", which has suddenly risen in popularity and is more or less confusing to people, is not a profound and abstract concept.

Today, let Xiao Ou come with you to explore why the data infrastructure has attracted the attention of the world. What kind of development history does it have!

The core of digital transformation that the world has attracted attention

Data infrastructure is an evolution of traditional IT infrastructure. It is a data center IT infrastructure that deeply integrates computing, storage, network, and software resources with data as the center, with the primary goal of fully tapping the value of data.

In the process of digital transformation, with the widespread application of AI, 5G, IoT and other technologies and the continuous growth of data volume, data infrastructure, as the underlying architecture foundation of the transformation, is ushering in an explosion.

According to a recent report released by Fortune Business Insights, it is predicted that under the influence of the new crown pneumonia epidemic that is sweeping the world, the adoption of data infrastructure by enterprises will accelerate. By 2027, the global data infrastructure market is expected to reach 1423.1 Billion US dollars, the compound annual growth rate during the forecast period is 5.5%.

The main body that attaches importance to data infrastructure investment also extends from Internet companies and super-large enterprises to small and medium-sized enterprises and traditional enterprises. In recent years, the latter has paid particular attention to upgrading the old IT infrastructure to a new data platform system to promote the progress of enterprise innovation.

Not only from the corporate perspective, but at the national level, the importance of data infrastructure is also increasing day by day. It is worth noting that all major economies in the world have positioned the construction of data infrastructure as the core link to promote regional economic growth.

In March, at the meeting of the Standing Committee of the Political Bureau of the CPC Central Committee, the importance of "accelerating the pace of data infrastructure construction" was raised to parallel with "increasing investment in public health services and emergency support materials" and "speeding up 5G network construction."

In the “2030 Digital Compass” plan recently proposed by the European Commission, “Building a safe, high-performance and sustainable digital infrastructure” ranks second among the four goals of the plan, second only to “cultivating a large number of highly professional Digital talent".

The four goals of the "Digital Compass" plan: talent, infrastructure, enterprises, and public services

Even the United States, which is obsessed with identity politics and the so-called "international order", has invested a considerable proportion in the new three trillion US dollar economic stimulus plan announced by the Biden administration.

From database to big data platform

The data infrastructure has ushered in an explosion and has become a world-wide "outlet" at the national level. However, it is not a very new concept, and even a little "old". It has gone through four stages of development so far.

Database stage

As the core support of digital transformation, the history of data infrastructure even dates back to the 7th and 80ths of the last century. That is when the first commercial database came out.

At that time, companies lacked a mature, grounded digital scene for front-line business personnel, and the core pain point was to solve macro-level business decision-making problems for the management. This also makes the data query dimension and digital display form of this stage relatively simple.

In this context, the early commercial databases of vendors such as Oracle and DB2 are mainly used in OLTP (online transaction processing) scenarios, which directly carry data storage and calculations from business systems and transaction systems, and generate data-oriented databases based on fixed data tables. Management’s fixed statements, etc. Therefore, these products are also called "transactional databases" or "transactional databases", and people often equate them with narrowly defined databases. This stage is also called the database stage.

Data warehouse stage

With the accumulation of the commercial and civil Internet for many years, the demand for enterprise data analysis began to gradually sink from the management to the business department in the 1990s, and business personnel began to use data analysis to solve problems more and more. Changes have forced companies to awaken a certain sense of digitalization.

In 1993, the father of relational database E. F. After Codd proposed the concept of online analytical processing (OLAP), the data infrastructure officially entered a new stage of "data warehouse". Enterprises began to separate the database carrying OLAP workloads from the transaction database of the business system, so as to avoid the interference of OLAP on core transactions.

relational database E. F. Codd

Therefore, the analytical database dedicated to OLAP, the data warehouse, was born and gradually separated from the transactional database. At the same time, in order to meet the rising demand for data storage caused by the increase in business scenarios, databases based on the MPP shared nothing architecture have gradually entered the sight of enterprises.

Big data platform stage

With the comprehensive coverage of mobile communication networks, the mobile Internet began to sprout and grow rapidly at the beginning of the 21st century. Data infrastructure around 2005, with the popularization of the mobile Internet and the further development of the Internet, the number of end users in the business system and the amount of data deposited in the enterprise have shown an explosive growth trend, ushering in the "big data" Platform" stage.

The expansion of the boundary between the Internet and the mobile Internet allows companies in a large number of fields such as finance and e-commerce to have more access to users' online data. This data is not only very diverse in data types, including structured data, semi-structured data, unstructured data, binary data, etc., but also has a large variety of complex sources and a huge amount. The demand for data storage capacity of enterprises has changed from GB, TB in the past, Quickly upgrade to PB or even EB level.

More diverse, complex, and massive data environments have once again increased the number of digital application scenarios for enterprises. Agile BI tools that can support business personnel to independently perform data analysis and self-service analysis capabilities have become a new direction for the digital display of enterprises.

At this stage, limited by the scale of node expansion, and does not support unstructured and semi-structured data, MPP data warehouses have gradually begun to fail to meet the needs of enterprises. Big data technology represented by Hadoop has gradually become one of the core technologies of data infrastructure.

The big data platform based on the Hadoop ecology, while compatible with the MPP data warehouse built in the previous stage, provides a SQL-on-Hadoop-based data warehouse, as well as large-scale data warehouses including NoSQL databases, stream processing, batch processing, and distributed storage. Data suite. It realizes the independence of storage nodes and computing nodes, and independent expansion of memory computing and storage.

Big data technology represented by Hadoop provides technical possibilities for enterprises to uniformly collect, store and process various types of data.

Promote the digital future of enterprises

The concept of "cloud computing" was put forward by Eric Schmidt, the CEO of Google at the time, and it took less than ten years to become one of the most concerned topics in the computer field. It has become the general consensus of enterprises.

After 2015, the scale of the total amount of social data has broken through a certain "critical point" after years of continuous acceleration and rapid growth, and the digital application scenarios of enterprises have become more extensive and common. A large number of cross-departmental, cross-business lines, and even cross-branch, cross-organization, and cross-regional data sharing and linkage analysis have made the trend of generalization, agility, personalization, and scenarioization of enterprise data analysis more and more obvious.

The path of enterprise digital transformation has also changed from a single-point breakthrough in the past to data sharing and asset management across the group, across organizations, and across regions, as well as full-scene data empowerment.

In this transition, some Internet and technology giants have put forward the concept of "data in the middle", starting from organizational structure changes, promoting business and data sharing, and accelerating business agile innovation. However, the "data center" has not changed the underlying technical architecture of the data infrastructure, and cannot break through the actual problems from the bottom to fully meet the data application needs of enterprises in the era of big data.

In order to break through the shortcomings of the old data infrastructure and traditional technology routes, the "data cloud platform" came into being. The data infrastructure has also entered a new stage.

Based on new technology architectures such as cloud native, separation of computing and storage, and virtual computing clusters, the data cloud platform realizes independent expansion of computing and storage nodes, thereby breaking through the scalability and flexibility limitations of MPP and SQL-on-Hadoop big data platforms .

In addition, the data cloud platform also overcomes the shortcomings of SQL-on-Hadoop databases in SQL standards and ACID features, and can support the smooth migration of digital applications from traditional shared storage data warehouses and MPP data warehouses to data cloud platforms.

In addition, through the underlying innovation, the data cloud platform gets rid of the influence of the open source technology system with Hadoop as the core, and overcomes its shortcomings in performance optimization and concurrency. The parallel executor is deeply optimized, and the elasticity is scalable, thus greatly Improve the performance of complex queries for multiple tables, batch data, and complex table associations.

Finally, the data cloud platform incorporates the concept of "data center" on the basis of the big data platform, thus forming a three-tier structure of "data platform-data assets-data services". To better meet the needs of large and medium-sized enterprises represented by group-type and multi-branch enterprises in the process of digital transformation.

The data cloud platform fully integrates cloud-native features, with more unified and powerful data capabilities, and support for AI applications, providing enterprises with a "more unified and stronger" digital transformation power.

Through its high flexibility, strong scalability, strong sharing, strong compatibility, strong complex query capabilities, automated machine learning support and other technical capabilities, it helps companies to more effectively deal with a series of large-scale, high-efficiency, and intelligent The trend of digitization promotes the comprehensive upgrade of the depth and breadth of digitization in the future.