Female developers interpret the future opportunities and challenges of databases for you

The annual International Women's Day is here. While offering blessings to all women, let us focus on those female leaders who are shining in the professional field, and show the world the wisdom and power of women.

Here, we also solemnly launch a new column "Big Celebrity Lecture Hall" , inviting big names in the industry to share their unique insights into the development of science and technology. In the first issue, we invited an outstanding female representative, Pan Juan, the Data Hero of Amazon Cloud Technology , to talk about the new trends in the database field.

Q: What position does the database currently occupy in the cloud computing field? What are the application scenarios?

A: No matter in the field of public cloud computing or private deployment, the database always occupies a very important position. Today, the operation of enterprises is inseparable from the support of data, and data has become a crucial factor affecting the production of enterprises. In order to protect these important assets, it is necessary for the database to assume such a role. Today, the data stored in the database is not only the business data on the surface. For enterprises, the business value hidden in the data is exactly the meaning of the existence of the database.

In the field of cloud computing, because of the characteristics of the "cloud" itself, such as elasticity, on-demand allocation, and operability, the database plays a very important role in the software infrastructure, and the environment, characteristics, resources and other capabilities on the cloud will support The database will play a greater value.

As for the application scenario, it should be derived from the user's application scenario. Because all databases are born to facilitate users to store and use data, in some scenarios that require large-scale computing and large storage capacity, NewSQL databases are generated, such as Amazon Aurora of Amazon Cloud Technology; in some KV storage, file storage oriented Under such scenarios, database products such as NoSQL and MongoDB have been produced.

In addition to the database itself, there are many application scenarios that can be developed in the ecosystem around the database. Especially in terms of intelligent operation and maintenance, such as the implementation of concepts such as DevOps and so on.

Q: What are the main technologies in the database field that you study? What do you hope these studies will achieve?

A: I am currently mainly engaged in research work in the field of distributed databases. While leading the contribution of ShardingSphere, the top open source project of Apache, I also serve as the CTO of SphereEx, an open source native business company. In my plan, open source projects and commercialization take two different development paths:

For the open source part, Apache ShardingSphere will follow the Database Mesh concept to provide distributed database solutions and data services for the surrounding ecosystem in the cloud; for the commercial part, SphereEx will provide a more complete integrated distributed database or cloud database solution in the future , and has successively launched a series of products and services in the fields of management and control, intelligent operation and maintenance for the surrounding ecology of the SphereEx cloud database.

In the future, I believe that not only the database, but the whole society will be more closely connected with the cloud. SphereEx and Apache ShardingSphere are gradually deeply binding with cloud vendors to continue to exert greater value on the cloud platform. At present, SphereEx is preparing to join the cloud innovation plan of Amazon Cloud Technology, and jointly provide ShardingSphere with a deployment form on the public cloud with Amazon Cloud Technology. In the future, SphereEx will carry out in-depth cooperation with Amazon Cloud Technology in Marketplace in China and overseas, to provide users on Amazon Cloud Technology with more powerful SaaS product deployment capabilities, and jointly create a more mature cloud environment for enterprise applications.

Q: What difficulties did you encounter when building the database, and what solutions did you use to overcome them?

A: The construction of the database itself is very complicated. Fundamentally, the existence of the database is to meet the needs of users to store and query data. In order to achieve such a goal, the research and development of the entire database requires a very strong technical system and rigorous theoretical support.

Therefore, while we are thinking about the technical difficulties of databases, we cannot just look at the problem from a technical point of view. On the one hand, the R&D team needs to continue to build a system about database technology, but on the other hand, it also needs to consider the actual needs of users, so that the database can achieve a balance between ease of use and availability. What we have to do, which is also more difficult to achieve, is to find a balance between the two.

In the Apache ShardingSphere project, the team independently developed a set of SQL language -- DistSQL, which is a set of database dialects specially designed for SphereEx and Apache ShardingSphere. Just as humans communicate in different languages, so do databases, and different databases have their own distinct dialects. Especially with the popularity of distributed databases, traditional SQL cannot support some capabilities in distributed scenarios. Therefore, we hope to provide a convenient means for users to use this database solution, namely DistSQL, while providing the Apache ShardingSphere distributed database solution ecosystem. Through DistSQL, users can directly operate the capabilities of traditional databases in additional distributed scenarios, such as data encryption and decryption, high availability, and elastic expansion and contraction. Through DistSQL, Apache ShardingSphere simplifies the use steps of users in a standardized way, and shields the complex implementation process for end users.

Q: What are the latest and exciting technologies in the database field? What impact will these technological innovations have on the industry and even on everyday life?

A: As the database is an underlying infrastructure platform, it is difficult for users to directly perceive the changes brought about by the underlying database in daily life. Usually, they indirectly experience the technological progress in the database field through some special events. impact on everyday life. The most typical ones are Black Friday, 11.11 and 6.18 such e-commerce promotion scenarios. For consumer users, all they need is hand speed, but in the eyes of back-end operation and maintenance students, this is a big test for their own technology and database performance. As the technology becomes more and more mature, it has been difficult to see the unavailability of services due to excessive traffic during the promotion period in recent years. The database silently supports the upper-layer applications and services behind it, ensures the availability of the business at a certain key node, and brings a smooth experience to all users throughout the process. This is the impact of technological innovation on life.

As the demand for databases in various fields continues to rise, completely different user scenarios have spawned various databases that focus on their respective subdivisions, such as NoSQL, NewSQL, cloud databases, and some databases that focus on industry subdivisions. At present, the database field has shown a very obvious fragmentation trend. Therefore, in different fields, the development direction of the database is naturally different. Including how to realize the intelligent operation and maintenance of the database, how to connect the database with AI and cloud computing, and how to integrate various Internet infrastructures at the database level, these are the directions that everyone is exploring.

As the co-founder of SphereEx, I have a deep sense of industry trends. With the improvement of the ecosystem around the database and the promotion of automated operation and maintenance and intelligent operation and maintenance concepts, some traditional operation and maintenance models are gradually being marginalized. Therefore, database operation and maintenance students can focus on the database scenario, because the database is a very potential field, but they cannot just focus on the current work. For future career planning, what operation and maintenance students need to do is to improve their irreplaceability and keep themselves ahead of the system.

Q: What is the future development trend of databases in the field of cloud computing?

A: There is no doubt that the combination of database and cloud computing is the general trend in the future. Naturally, SphereEx also hopes to step up cooperation with cloud vendors in this regard in the future, and provide users with SaaS and PaaS-level support on the cloud. Because of its own characteristics such as elastic expansion and contraction of cloud computing, operability, ease of operation, and on-demand allocation, it is very friendly to infrastructure services. As a very important part of China's infrastructure in the new era, databases are combined with cloud-related technologies. Whether it is database vendors, cloud vendors or users, the connection between the three will only get closer and closer.

Q: Do you have any experience sharing about skills improvement for builders who want to enter or are already in the database field?

A: There are two main points, which are about career planning and skill improvement.

In terms of career planning, I always adhere to the saying "choice is greater than effort". If you choose the wrong direction, and continue to run desperately in the wrong direction, this is the most inefficient way. Therefore, in order to avoid this situation, everyone must pay attention to the development of the entire industry and the development of cloud and database fields. Picking the right direction is more beneficial than running in the opposite direction. It is recommended that you pay more attention to the information of cloud manufacturers, such as the release of new products, recruitment plans, etc., which can help you quickly understand the current industry trends. Standing in the wind, can get faster growth.

In terms of skills improvement, I highly encourage everyone to participate in the open source community. From the beginning of the open source culture to today, participating in open source has been an excellent way for most people to gain technical improvement. When you learn about open source and deeply participate in it, you must be able to feel the changes in technology from the practical level, and this change from the perception level will help you actively accept new skills from the awareness level.