PingCAP DevCon 2021 million-word long review: foreseeing the future ecology of data technology

PingCAP DevCon is PingCAP's annual technology event, dedicated to discussing the integration of cutting-edge technology and digital trends, and is known as the vane of observing forward-looking trends in open source industries and databases. **On July 24, PingCAP DevCon 2021 with the theme of "openness, connection, foresight" was held in Beijing, bringing a technological feast to tens of thousands of online and offline audiences. Liu Qi, the founder and CEO of PingCAP, introduced PingCAP's thinking on open source, enterprise level, internationalization, and future development direction at the conference. **The following is a transcript of the speech, 10366 words, and it takes 55 minutes to read.

database is undergoing an epoch-making experiential change

In 2021, the database is undergoing an epoch-making experiential change. If you take the automobile industry as an analogy, this change is like a leapfrog change from fuel vehicles to electric vehicles. The new generation of cars has brought a brand new experience in many innovative functions such as large-screen visual interaction, strong power, automatic driving, and self-management.

new generation of cloud databases is undergoing similar experience changes in terms of resource expansion, auto-scaling, manageability, and self-service. of the interactivity, ease of use, and intelligent driving capabilities that everyone feels on the new generation of cars today have begun to be reflected in the new generation of databases.

Let's review the history of database development through two curves: the first curve started in 1970, and that generation of databases was called classic databases. Classical databases support the entire information age, and they play a vital role in various applications of enterprises. Since the beginning of the Internet era, the rapid growth of data, the acceleration of digitization, the innovation of hardware technology, and the innovation of software technology have greatly promoted the transformation of database technology. In the last decade of the second curve, combining the trends of open source and cloud, a new generation of cloud databases is gradually coming to the center of the stage.

Among them, 2020 is the year of the watershed of these two curves. Since this year, "open source, simplify, and merge" has become a new technological trend. We can imagine what kind of car we will drive in 2030? What kind of database will be used? The experience and evolution direction of the two are actually highly consistent, that is, more powerful, smarter, more worry-free, and more fun.

It sounds like the prospects are bright, but the reality is cruel. According to Gartner, 85% of big data projects in enterprises have failed. Why is there such a high failure rate?

There are several reasons:

First, with the rapid growth of the business, the database is usually too late to expand and cannot continue to grow in the footsteps of the business. The three founders of PingCAP are all witnesses of these pains. In the past, we would go online for a new application and prepare for one to two months to spend many nights for an online promotion. At that time, various sub-databases and sub-tables were rewritten. Nevertheless, every time it goes online, it is still fearful;

Second, there are many projects with huge investments that have no way to transform the value of data in a timely manner. In fact, many times today, it is not a lack of data, but there is a huge gap between the data itself and the value of the data, such as the inability to meet the timeliness requirements.

Third, usually the technology stack you choose is often very long. From the database to the queue through Binlog or CDC, some changed and updated data are transferred to the queue, and then the ETL is made from the queue, and after the ETL is completed, it enters the data warehouse. The technology stack will be very long, making it difficult to realize the value of the data. Today's real-time promotion, real-time risk control and other aspects require responsiveness in seconds or even milliseconds.

Digitization has brought about a big change during the time when the epidemic is raging. Everyone has a deeper understanding of digitization, and the timeliness of data has become a rigid need. Real-time transaction processing, real-time feedback, real-time convergence, and real-time insight have become a must for the digitization of the entire scene. Today, second-level data analysis and feedback may bring tens of millions of dollars in income or loss.

These challenges have also caused many companies to deposit complex technology stacks like spider webs, which has further led to the deterioration of data timeliness, and the realization of data value has become more difficult.

Believe in the power of

The Internet era has brought about a surge in data volume, more complex technology stacks, and various divergent needs that continue to emerge in the business. In the next ten years, the growth of data will only be faster and will show exponential changes. If we stay in the status quo and do not make technological changes, the pain we feel today will be exponentially amplified in ten years.

When it comes to change, many people may have a feeling that PingCAP is a relatively special company with some different tunes. But it is not necessarily possible to say what this adjustment is. In fact, PingCAP's idea of starting a business at the time was very simple. It was to build a database to make data support for business easier and more convenient, and operation and maintenance engineers could also reduce the pressure and even make They have a good night's sleep. A good night's sleep is such a luxury today.

Of course, only the original intention is far from enough. A product with such a complex database cannot be completed by a few engineers' whimsical ideas. It needs to be polished by users in real scenes and continuous contributions from developers. **During the creation of PingCAP, we have always believed that ****TiDB is not a product of PingCAP, but the common "work" of all developers, all users, all DBAs, and all operation and maintenance engineers. **This comes from our belief in openness and the belief in the open source community.

Some people have different steps because they heard the sound of drums in the distance

Thoreau has a sentence in "Walden Lake": " Some people have different steps because they heard from a distance."

When we developed TiDB six years ago, we encountered many doubts and many disapproval. From the beginning of the business, I couldn't sleep for upgrading and expansion, and after the later products were finally developed, I couldn't sleep because I couldn't find users. For an enterprise, the replacement of a database is a bit like a stent for a heart, and it is also like a replacement of an engine for an airplane in operation. Almost no one is willing to take such a risk.

At that time, we heard a lot of questions: "TiDB? Haven't heard of it!", "Do you have any industry user cases?" We were able to persevere because the first batch of users were brave enough to try new things. . At that time, TiDB was still very immature, untested, and lacking user cases.

Today, you can see that TiDB is iterating so fast, because we have also heard the drums from afar, and our pace will become faster and firmer. The "we" here refers to a group of people who insist on taking the open source road, including our users and contributors.

Believe in the Power of Openness

Speaking of the distance, we will tell a true story from the distance. There is a professor and his students in a university in Guatemala, both of whom are very active users in the Chaos Mesh®️ community. They have made a very interesting project on GitHub. By building a distributed system, they can display the statistics of new crown vaccination around the world in real time. Chaos Mesh® guarantees the stability of this distributed system in this project.

Before seeing this project, we did not even know that Guatemala is a country in Central America, nor did we know that Chaos Mesh®, the chaos test project created by PingCAP, could participate in the work of the new crown epidemic that all mankind is concerned about in this way. I don’t know that this project can contribute its social value in another way.

All this just shows how much an open community has the ability to extend, and it can spontaneously extend to places in the world that are completely unimaginable. allows us to further see the power of openness. The open source community has crossed national boundaries, across nationalities, innovated continuously, and contributed the results to the entire community again, so that people in a wider range can benefit, and further form a very Good forward cycle.

Today, PingCAP's open source community has spread all over the world. The open source project founded by PingCAP has won more than 41600 stars on GitHub, more than 1400 contributors, from dozens of countries around the world. Open source is the starting point of PingCAP, and it is also PingCAP's long-term strategy.

The development process of PingCAP has gone through the process from our own belief to the industry's recognition. In the CNCF's 2019-2020 contribution ranking, PingCAP ranks first among Chinese companies and sixth in the world.

CNCF CTO Chris Aniszczyk specially recorded a video for PingCAP DevCon 2021, expressing his tribute to PingCAP: "Thank you PingCAP for being one of CNCF's early projects and providing early contributions through TiKV. PingCAP is the sixth largest of all CNCF projects. Contributors, in terms of company size, this is quite a crazy achievement. Because the top companies, such as Google, Red Hat, VMware, and Microsoft, are all super large companies. Therefore, I would like to pay tribute to PingCAP and sincerely thank PingCAP for Contributions and ideas made by CNCF."

Today, PingCAP's open source projects are no longer limited to TiDB. In the past two years, our other two open source projects TiKV key-value database and Chaos Mesh® cloud native chaos test platform have entered the CNCF Foundation for incubation :

TiKV, as the storage engine of TiDB, also serves as an independent distributed key-value database, and officially graduated from CNCF in September 2020. As a graduation project, TiKV has successfully served many well-known companies such as JD.com, Meituan, and Zhihu;

Chaos Mesh® successfully entered the CNCF sandbox project last year, and was actively adopted by users including Tencent, WeBank, Xiaopeng Motors, Qiniu Cloud, ByteDance, etc., and became one of the necessary tools for fault drills in the cloud native environment It is also the most popular chaos test platform of CNCF in this field.

It is precisely because of PingCAP's active contribution to the open source community that PingCAP has gained tremendous influence in the global open source community.

The Power of Connecting Talents

Whether it is an open source community or an open source project, what is their nature? Speaking of the power of openness, what is its root? On the surface, open source is code, but in essence it is about the connection between people and technology. Code is just one of the carriers of this connection and collaboration.

In the past at a software company, we often said that the most important asset of the company leaves this building every night, It is all about talents. In open source software companies, our real strength is on the invisible network, in various application scenarios, in every PR on GitHub, and in every interaction in the community. The convergence of all power comes from the convergence of wisdom, and also from the connection and trust that are also produced by faith.

So far, based on the open source community, we have formed a global database of talent ecosystem. The innermost link in this ecosystem is PingCAP's R&D team of hundreds of people. They focus on the core R&D of TiDB and actively participate in the joint development of many projects in the community.

In the second ring, there are thousands of developers, including more than 1,400 contributors in the community, as well as many well-known universities and scientific research institutions, as well as top technical experts in various fields around the world. For example, TiDB participated in the early core members of the Rust language. Recently, TiDB has also entered the courses of the world's top universities, University of Wisconsin and Carnegie Mellon University.

In Sanhuan, it is our users and partners, covering the entire data technology ecosystem, affecting tens of thousands of DBAs and operation and maintenance engineers. These users and partners brought their respective scenarios back to the open source community. Together with our R&D and community contributors, they realized product development based on real scenarios and formed a very high-speed iterative mechanism. It accelerates the scene expansion and business innovation of enterprise users.

Open technological ecology

talents constitutes a fast iterative technology ecology. TiDB spans OLTP and OLAP, further forming a complete HTAP capability .

In the field of big data, we have achieved extensive integration with big data technology stacks such as Spark, Flink, and Presto. In the open source community, we have also cooperated with many cross-border projects; in the field of infrastructure, we have also actively embraced cloud computing and The cloud-native technology ecology has established extensive partnerships with the world’s leading cloud computing vendors AWS, Google Cloud, and VMware. At the same time, we have also established various levels of cooperation with many domestic manufacturers, including JD Cloud, Qingyun, and Qiyun. Niuyun, mobile cloud, etc.; in addition, we also maintain a long-term and extensive cooperative relationship with open source vendors such as RedHat and SUSE.

At the upper level of the technology ecology, we work with users and partners around real scenarios, especially real-time online transactions, real-time insights, real-time analysis, and real-time decision-making, to provide users in all walks of life with products, solutions, and services. Based on real scenes in multiple industries such as finance, new economy, telecommunications, manufacturing, logistics, retail, public utilities, etc., stable cooperation and long-term linkage have been established.

The open source innovation road of Ping An Technology

Ping An Technology is a very important enterprise customer in the PingCAP ecosystem. In this conference, Wang Yang, chief engineer of the database team of Ping An Technology, brought a theme sharing of "Ping An Technology's Open Source Innovation Road".

Ping An Technology officially started using PingCAP in early 2018. At that time, the Group Property Insurance needed to make a good start. The database team needs to plan and evaluate IT resources according to the business volume and the scale of the activity volume to determine whether it needs to be expanded.

For Ping An, the opening ceremony and the God of Wealth Festival are held every year, but 2018 encountered very big challenges. This year, Ping An Property & Casualty's business and other insurance businesses have been developing rapidly, and the volume in 2018 was much larger than in previous years. This brought several challenges to the database team: First, Ping An was still using a centralized database at the time. Everyone knows that the limitations of centralized databases require highly skilled personnel to evaluate, and the accuracy requirements for evaluation will also be very high. In the preliminary assessment, the database team believes that no single server can bear the amount of activity that year, so how to ensure the smooth progress of the Fortuna Festival activities?

Because I usually pay more attention to the field of distributed databases, I naturally thought of a native distributed database-TiDB. The PingCAP technical team, Ping An’s TiDB project personnel, and the development, testing, and operation personnel of the subsidiary formed a joint research team to conduct intense testing and verification. At that time, the time left for the research team was very limited, but with its rapid deployment and flexible scalability, TiDB built a 20-node production cluster in just over 20 days, which smoothly guaranteed January 2018. On the 8th, the P&C Insurance Fortuna Festival activities went on smoothly.

Wang Yang lamented: " were hard to imagine in the past. After this campaign, we have increased confidence in PingCAP products , and also greatly increased the professional capabilities of PingCAP's R&D, technology, and support team. Confidence. So we began to promote and use TiDB in different subsidiaries and different systems. For example, the life insurance gold housekeeping activities and the group’s customer information management activities."

With the use of TiDB in Ping An, the Ping An database team is unwilling to be just a super user. They want to be more able to use the power of TiDB, establish more connections, hold a more open attitude, and promote the development of overall distributed database technology. Therefore, they participated in the Ti-Operator and Chaos Mesh®️ projects. Some of them even became reviewers of SIG-K8S, submitted and merged 45 PRs, proposed the grayscale upgrade plan of Operator, and carried out plan design and code implementation. At the same time, the Pod update strategy has been optimized, as well as the performance verification of the HostNetwork network solution. While applying the results to business innovation, Ping An also contributes the results to the global open source community and continues to promote the progress of database technology.

A new generation of enterprise-level database in the eyes of users

As a basic software, the database should also be inclusive. PingCAP's product capabilities and PingCAP's vision not only serve Internet users. Since serving Bank of Beijing a few years ago, TiDB has greatly improved its enterprise-level capabilities and has been continuously and rapidly improving.

According to Davis, senior vice president of PingCAP's business in China, according to incomplete statistics, has more than 1500 companies around the world applying TiDB to solve the business challenges they face in production and business environment. Business growth is .

Why do so many users and customers choose TiDB?

This is because TiDB is a new generation of enterprise-level database designed for the digital age from day one. Today, all walks of life are undergoing digital transformation, and the digitally native TiDB has expertise in the two main directions of digital transformation. The first is agile business improvement, and the second is real-time insight into consumption of data.

The improvement of business agility requires IT systems to respond more quickly to business changes and data growth. Real-time consumption of data requires enterprises to be able to realize real-time insight and analysis, real-time analysis, real-time decision-making, and real-time realization. This requires database and traditional database architecture. The design is very different. As a data-native and cloud-native distributed architecture design, TiDB supports business agility improvement with agile and highly scalable OLTP, and supports real-time data consumption and insights with innovative real-time HTAP architecture.

At the same time, TiDB has accumulated a wealth of infrastructure solutions and application architecture solutions through practices in all walks of life. From "data migration, business continuity, cloud access, security" in infrastructure solutions to "real-time data services to real-time risk control, distributed batches, comprehensive data query" and so on in application architecture solutions, These overall solutions are fully helping enterprises to apply TiDB to achieve business success. Especially in the past three years, PingCAP's innovation in real-time HTAP provides a one-stack data solution that can fully empower enterprises in digital transformation to realize real-time data insight, data analysis, and real-time realization.

Today, TiDB products, solutions, and support services have been widely used by corporate customers in various industries such as finance, new economy, logistics, retail, e-commerce, etc., to help them promote technological innovation and empower business:

• Bank of Beijing uses TiDB to realize online expansion of 500 million pieces of data within 5 hours;
• After China Life Property Insurance uses TiDB, the document status statistics have been shortened from 8 hours to 6 minutes;
• TiDB monitors more than 10,000+ objects in various scenarios of Bank of China, and daily collected data items reached 1.45 billion;
• At China Everbright Bank, TiDB supports 50 million transactions per day.

On April 25th, PingCAP released the brand new TiDB 5.0, which is a huge improvement in function and performance compared to version 4.0. In the three months since its release, it has been used by a large number of users and provided a lot of positive feedback:

• After TiDB 5.0 is launched on Autohome, the performance of aggregation scene query is 20-50 times higher than that of MySQL, and it can calmly deal with tens of millions of DAUs and hundreds of millions of visits;
• 58. TiFlash MPP is introduced in the same city to improve the real-time analysis and calculation capabilities of massive data. In the multi-table related query, it has a 90% performance improvement compared with TiDB 4.0;
• TiDB carried the 350,000 QPS of ZTO Express. After the MPP was enabled, the timeliness of reporting was reduced from 5 minutes in the past to 1 minute.

We can find TiDB in our work and life scenes 24 hours a day. Behind this string of numbers, we can truly feel that TiDB plays an indispensable role in each customer's journey to success.

Business agility, has become the lifeline of digital business

Business agility is a hot topic recently. It is no longer an option, but a necessary skill. Business agility can not only respond to market and customer needs faster, but also improve operating efficiency and reduce operating costs. As digitization continues to deepen, business agility is bound to be an irreversible trend.

Among TiDB's many customers, there is a leading catering retailer who has successfully improved business agility through TiDB:

In 2018, this company began to explore the establishment of a middle office that can support multiple formats, multiple channels, and large concurrency to achieve business agility. Due to the particularity of the catering industry, traffic usually has obvious high and low peak characteristics, requiring technical middle stations to have strong dynamic capacity expansion capabilities. This also means that when choosing a database, whether to support cloud native, whether to support agile and flexible expansion and contraction has become a very important consideration.

Of course, there are two other important considerations. The first is stability, whether there is a mature high-stability solution; the second is whether there is a complete support service system.

Through multi-party PK, TiDB stands out by virtue of its agile, highly available, highly reliable and scalable OLTP, as well as a complete supporting technical service system, and has successfully landed in core middle-office businesses such as users, payment, and messaging. after two years of big promotion and online transactions of hundreds of millions of users, TiDB has been fully verified for its agility, high scalability, and high concurrency.

In addition, the customer also found that because TiDB can focus application scenarios on the business, without considering the sharding logic, and without considering the unpredictability of business growth, the customer has greatly accelerated the application, iteration, and launch of new and old businesses to meet continuous changes. Consumer demand.

The customer has also seen TiDB's activity and development in the community in recent years, which means that the talent ecology is enriched, which also strengthens the customer's choice of TiDB and continued use of TiDB. In the future, they also plan to actively participate in TiDB community activities like Ping An Technology and contribute to the open source community.

Real-time insights have become the nirvana of digital business

Another example is the story of real-time consumption and insights from ZTO Express and TiDB data.

As the largest express delivery company in the world, with the rapid development of business and the rapid increase of data volume, the data cycle of ZTO's original storage in Oracle Exadata is getting shorter and shorter, and the business side's demand for data analysis continues to rise, and the real-time performance The requirements for the Oracle are becoming stronger and stronger. It is no longer possible to solve the problem fundamentally by simply upgrading the hardware of the Oracle all-in-one, and the cost is getting higher and higher. Especially during business peaks, stand-alone performance encounters bottlenecks and the risk of stand-alone failure is very high. In addition, the most critical thing is that the analysis timeliness can only achieve T+1, which cannot meet the needs of the business at all. The technical team of Zhongtong also conducted research on solutions such as HBase and Kudu, but found that these data warehouses are incompatible with existing technology stacks, and they cannot support multi-dimensional queries on the business side, and finally decided to choose TiDB.

TiDB has a one-stack HTAP database solution, which not only supports online horizontal expansion, online storage and computing nodes at any time, but also supports the timeliness of data analysis very efficiently, and is compatible with existing big data and technology ecology, reducing The complexity of the technology stack. **The most important thing is that the TiDB HTAP solution separates row storage and column storage, which can completely isolate OLTP and OLAP resources, which means that it not only realizes real-time data analysis and insight, but also does not affect the performance of TiDB. In addition, ZTO can also calculate the data stored in TiDB through TiSpark, and then support the timeliness analysis of the full link of ZTO by storing it in Hive.

Now, Zhongtong can realize quasi real-time understanding of whether each ticket has a problem in each link, and make real-time decisions and calculations. At present, ZTO Express has completed the 5.0 upgrade test. With the help of the HTAP model, it has achieved a 5 times speed increase in the big promotion monitoring scene. ZTO is also planning to use the real-time analysis capabilities of TiDB HTAP in other analysis services in the future. .

Enterprise-level services help users to fully implement the scene effect

Customer success is the core concept of PingCAP. In addition to products and solutions, customers also need very good support services and operation and maintenance guarantees. PingCAP has a complete support service system, so that customers can rest assured, rest assured, and smoothly. Apply TiDB locally.

By purchasing the PingCAP enterprise-level subscription service, within the validity period, you can continuously obtain the innovation and performance and function enhancements brought by product evolution, and enjoy unlimited remote product support services and professional guidance with support response guarantees. The premium subscription service will also include multi-vendor technical consultations, including special guarantee support for important nodes.

If you need more in-depth services, you can also purchase PingCAP expert services to allow PingCAP experts to provide on-site services such as architecture consultation, application adaptation, online expansion, and in-depth inspections.

In addition, PingCAP can also provide professional training and certification services according to customer needs to help companies cultivate TiDB data technology talent echelons with comprehensive architecture and development capabilities.

Ti-Star plan, through the "hybrid ecology" to help the user journey success

Under the open concept, the company and PingCAP are connected to each other, and a chemical reaction is generated, which injects new impetus into the company's own growth and business transformation and innovation.

Digital China CIO Shen Yang said as a witness that open source technology is leading the development of the entire information technology industry. Digital China and PingCAP have established a long-term ecological linkage, in addition to launching TiDB and Digital China distributed database integrated machine, landing end-to-end industry solutions In addition, the technical team of Digital China has deeply participated in the open source community, developed TiDB for PostgreSQL, and achieved new breakthroughs in open source technology and service solutions.

One of the biggest features of TiDB is the hybrid ecology. This ecology has both the unique value brought by the open source community and the value brought by enterprise-level products and services. From the perspective of the complete life cycle of a customer, PingCAP has many customers. The earliest may be to find TiDB from GitHub, or go to the official website to download TiDB, or compile TiDB by yourself. They gradually encountered some technical consultation questions during the use process, and they may ask questions in the community at first. As they use deeper and deeper, they gradually expand the scope, hoping to establish more face-to-face connections with the PingCAP team and obtain more technical support. And professional services.

Ti-Star plan, PingCAP will provide in-depth support and continuous interaction from the original factory in each of the above links. Ping An Technology is a very good example. This plan will help users make better use of TiDB. Whether you are an enterprise-level user or a community user, as long as you meet certain conditions, you can apply to join the Ti-Star plan. TiDB's community support and enterprise services will help users complete the entire user journey together. Through the co-creation of both parties, while achieving customer success, TiDB also becomes closer to the user's business needs and more grounded.

application scenarios connect global users

Internationalization is a long-term philosophy that PingCAP has adhered to from day one. It is accompanied by PingCAP's open source strategy, making it easier for us to hear from afar.

In the past year, PingCAP has added many users in various parts of the world, including customers in the United States and Japan. PingCAP has also substantially increased local teams in North America, Japan, Southeast Asia, and Europe to better support local users.

Many international customers like TiDB not only because of its good product experience, but also because it has experienced more extreme scenarios for users of the same type. These extreme scenarios cover Internet finance, e-commerce, payment, online games, video and other industries. Seamlessly connect to overseas users to help users solve business needs such as large-scale data processing, risk control, real-time decision-making, and data center.

After verification by Chinese Internet companies and corporate users in these extreme scenarios, we quickly discovered that global users also have similar scenarios, and they also have similar needs. These users can quickly connect through an international community to understand and use TiDB. The product.

The global logistics industry has developed very rapidly since the epidemic, and what supports an efficient logistics system is a real-time data insight system. China's logistics industry is a global leader. In the case of Zhongtong, we have seen how they use TiDB to innovate and achieve real-time insights. Ninja Van is a typical case of bringing China's leading advantage scenario to the world.

At the beginning of last year, the logistics unicorn company Ninja Van, headquartered in Singapore, decided to find an alternative solution to the MySQL stack. Due to the high compatibility between TiDB and the MySQL protocol, they came into contact with TiDB 4.0, and they were only right during the migration process. Minor changes to the code completed the entire migration.

In addition, TiDB's out-of-the-box high availability, cloud-native features, and support for online DDL all helped the database team reduce a lot of operational overhead. Ninja Van Co-Founder & CTO Shaun Chong said: “I hope TiDB will stick to open source. We have evaluated many other open source NewSQL projects. Unfortunately, their community version cannot support the full set of features, but TiDB is not. In my opinion, the community The version is as important as the enterprise version, so we want TiDB to maintain this approach. Therefore, the main benefit of open source is that it is very easy for the community to participate in it, and it has a large user base while ensuring cost-effectiveness."

connects to the cloud-TiDB Cloud

Global users have a strong demand for the value of TiDB, so how can these values be delivered to global users efficiently? PingCAP's choice is through cloud services.

With the evolution of IT infrastructure, cloud services have become a new normal. We see that the main technology stacks of many users are built on the cloud and evolve to a cloud-native architecture. Therefore, more and more users are asking us whether we can provide a TiDB hosting service that can enjoy the powerful functions brought by distributed databases while avoiding complicated operation and maintenance work, and further peace of mind in the cloud-native world.

Therefore, in June 2020, PingCAP launched TiDB Cloud, a fully managed TiDB service, in overseas regions. And through limited preview, some users with strong needs can use it first.

In the past year, TiDB Cloud has continued to evolve its functions, especially in the cloud-native way, in terms of data access security and flexible operation and maintenance, which has greatly improved users' security access capabilities and business continuity. In May of this year, TiDB Cloud was officially opened to global users in overseas regions.

At the same time, PingCAP has established a more agile iterative development method for TiDB Cloud. On the one hand, new cloud-native features can be launched faster. For example, in June this year, the compatibility with the data format exported by Amazon Aurora was further enhanced to make data migration easier; on the other hand, TiDB Cloud closely follows the core functions of TiDB. Users can not only enjoy the functions of the latest version of the database kernel, but do not need to upgrade the system by themselves, and get rid of the tedious operation of upgrading the system. Now, users can use the latest TiDB 5.0 version on TiDB Cloud.

Data security of

All this PingCAP does has only one purpose, which is to make more users more worry-free. Users who host their key business and core data in TiDB Cloud will worry about another issue. How to ensure the security and privacy of these data?

To this end, PingCAP has established a scientific and effective information security management system that complies with international standards. For example, the formation of a global professional security team can comprehensively improve the overall security technology architecture on the one hand, and on the other hand can meet the needs of security incidents. TiDB Cloud has passed SOC 2 Type I/II, ISO 27001: 2013 two global authoritative certification standards , PingCAP is also doing more compliance certifications in the future, through these, the entire security system can be further strengthened to ensure product safety and compliance, and fully meet the needs of domestic and foreign customers. Improve information security risk management and control capabilities, so that users can use TiDB Cloud without any worries.

Global multi-regional data synchronization will be online soon in the third quarter

Cloud computing allows countless companies and entrepreneurs to look around the world and use technology to change the lives of people in different regions. Introducing the business to the world, the data service level under the business also needs a way that can be delivered globally. Independent third-party data services can not only match the data needs of different cloud vendors, but also become one of the core components of cloud services in the future. .

TiDB global multi-regional data synchronization service in the third quarter to help users more easily achieve cross-regional data aggregation, accelerate the construction of big data platforms, and accelerate the implementation of HTAP.

Since its release, TiDB Cloud has been iterating rapidly and innovating continuously. Whether you are a thriving startup company or a large enterprise in digital transformation; whether your company has just tested out to sea, or has provided services in multiple regions around the world; whether you are a MySQL DBA for many years, or a cloud Native architects are an indispensable part of the future blueprint of PingCAP and TiDB. TiDB Cloud hopes to deliver the value of TiDB to every developer in the world faster and more efficiently, allowing users to be anywhere and at any scale. , Enjoy the ability of OLTP scale and Real-Time HTAP.

In order to further achieve this goal, in the fourth quarter of this year, TiDB Cloud will be free for individual developers! Developers can use TiDB standard cluster for free to experience and test TiDB's seamless support for cloud native applications. TiDB Cloud will also provide a free TiFlash analysis engine node, allowing users to experience the latest HTAP functions and personally experience the time savings and efficiency improvements that HTAP brings to real-time analysis.

Cloud computing has popularized computing capabilities around the world through a service-based approach, allowing global users to grow faster. On this basis, the servitization of software on the cloud is a new user demand and industry trend. This trend is more obvious in the post-epidemic era. From video conferencing to document collaboration, from data processing to artificial intelligence, we have seen many fellow travelers on the road to software services on the cloud.

Databricks is one of them. Databricks is a leader in the field of big data and AI. It has been widely recognized worldwide and its business is developing very fast. Databricks has increased from 1.5 million virtual machines to 5 million virtual machines in a year. They hope to find a cross-cloud database solution that can be expanded and elastically. At the same time, they hope that in the future, they can do fast real-time on the basis of data. Analyzing, I have high hopes for TiFlash, and it happens that TiDB satisfies all the imagination for the future. In this rapid development process, PingCAP is fortunate to be a fellow traveler of Databricks on the cloud.

Foresight-Cloud-native next-generation experience

In recent years, there has been a popular saying called the last mile of user experience. Similarly, in the cloud-native environment, the user experience of using the database still lacks the last mile. On the cloud, all businesses have become distributed, and there are a large number of calls between microservices in the distributed system. Between all these calls, we can observe the call sequence and call logic between different microservices.

However, when we create an order, query an order, and send an order from generation to delivery, what happens in the database? We have no way of knowing.

Is it possible to clearly see what a business operation looks like in the end from a business perspective?

**TiDB****Open Tracing is a product specifically aimed at the last mile of database experience. It allows users to achieve true observability in all scenarios. **When generating an order or querying an order on the business side, you can clearly see which SQL statements are executed in the microservices of the database, how much data is scanned, and how many queues are queued in the entire query operation. For a long time, even how many IOs have been experienced.

If the observable is like a telescope, it helps us to have a panoramic view of everything. However, accidents always happen in reality, and how to predict and control risks in advance is also very important.

The ancients often said that "feeding the enemy before the first", if we can't achieve "the failure of the first", all the good things now may collapse in an instant under a fault, so when developing TiDB, PingCAP also developed a chaos test Platform-Chaos Mesh®.

Through the continuous iteration of the open source community, Chaos Mesh® officially released version 2.0. This version improves the overall ease of use, allowing more people to easily inject various faults into the system, further improving automation capabilities, while reducing overall testing costs, and providing more fault injection types.

At present, Tencent Games has used Chaos Mesh® to perform fault drills more than 150 times a week, and records have found and solved more than 100 problems, streamlining and optimizing processes, and improving efficiency by more than 10 times.

The belief we have always believed in

In the past six years, **PingCAP has always insisted on doing a "simple" thing-let the database return to its original appearance, leave the complexity to TiDB, leave the simplicity to the user, and make the database worry-free and easy to use . **Real scenes are the best architects. We believe in openness, being friends of time, embracing innovation, and simple delivery. These are the concepts we have always believed in.

In the past six years, TiDB has received countless attention and encouragement. With the help of everyone, we have gradually grown from a baby waiting to be fed to an energetic teenager step by step. Behind these more than 2,100 days and nights is the meticulous care of countless contributors. It is your PR, every reply, and every encouragement that have made TiDB today. Special thanks to every developer for their contributions. I believe Open and transparent collaboration will surely create new and infinite possibilities. Let us work together to build TiDB into a one-stack service ecosystem. We are on the road together!