Cracking the Dilemma of Database Core Talents: PingCAP's Thoughts and Attempts丨Interview with Talent Plan

Databases can be traced back to the 1960s, and are products of the same era as contemporary electronic computers. From the day it came out, the database has assumed the function of supporting application software upward and adjusting system resources downward. However, the shortage of R&D talents in the domestic database field has seriously affected the development of the database industry.

So why is there a shortage of database talent? How to solve the challenges faced by database talents? With these questions in mind, we interviewed Mr. Wang Yanguang, the head of university relations and talent ecology of PingCAP, and asked him to share PingCAP's thoughts and attempts in the field of database talents.

Database Talent Challenge

Taking the domestic demand for talents as an example, there are only 6,000 to 7,000 developers who graduated from colleges and universities and devoted themselves to distributed databases. But corresponding to the database industry, how big is the demand for talents? Teacher Wang Yanguang gave a figure - 10 times, that is, 60,000 - 70,000 people are needed every year. This gap should not be underestimated. If it is not taken seriously, database developers will face a shortage of fresh blood, even lack of continuous motivation, and face the situation of "no successor".
At present, the curriculum setting of databases in domestic colleges and universities is still based on database use and application development based on databases such as SQL Server, MySQL or Oracle. From the perspective of curriculum, taking relational database as an example, it is mainly divided into three categories: one is about database tables and data organization, such as set theory, relational algebra, relational paradigm, and SQL language; the other is about DBMS implementation Courses; another category is the management operation and maintenance courses of industrial databases.
However, in the past ten years, the demand for the complexity of data management brought by China's Internet economy has spawned the need for more complex DBMS kernel implementation talents. However, it will take time for social demands to be transmitted back to higher education.
Some people may ask, in addition to colleges and universities, there are also various database technology training institutions in the society, why can't they fill this talent vacancy? This is mainly because compared with the database core development positions, the total demand for database use or management and operation and maintenance positions is greater, and social training institutions are generally aimed at the database operation and maintenance market with a larger space. The knowledge path is very deep, and it is necessary to master the basic knowledge of compilation principles, operating systems, and distributed systems. At the same time, in terms of overall talent demand, courses that are not as large as the former lack motivation.

The Origin of PingCAP Talent Plan

In 2018, Liu Qi and Cui Qiu of the PingCAP founder team went to the US Bay Area to attend a database industry conference. They noticed an impressive phenomenon that many lecturers at that conference were from education and academia, including some professors, lecturers and even doctoral students. They found that these people's theoretical level, scientific research level, and engineering level were very good. This incident touched them a lot, so after returning to China, they keenly decided to start some scientific research cooperation with universities.
This is the origin of the Talent Plan. In order to solve the database core talent challenge, PingCAP has launched an open source database development course - Talent Plan. By uniting with excellent universities and enterprises, we will build a best practice platform for database development talent training in universities across the country. Students who pass the graduation assessment will also receive an officially certified graduation certificate.
"Distributed databases and distributed systems are relatively new fields. Recalling that the biggest difficulty when I first learned these contents was that there was no platform and curriculum that combined systematic theory + step-by-step practice, and I could only read scattered materials while reading After exploring at work, I came up with the idea of doing a Talent Plan, and I am very happy to see that the original intention at that time has now become a reality.”
—— Huang Dongxu, co-founder and CTO of PingCAP, for scientific research cooperation with universities, students need to be able to understand industrial products, and Talent Plan has become the bridge connecting academia and industry. Students in schools that cooperate with PingCAP can quickly learn about TiDB and other products through the Talent Plan.
In 2018, a mini version of Talent Plan 0.1 opened. From 0.1, 0.2 to 1.0, 2.0, the path modules and materials of Talent Plan have been continuously increased and iteratively upgraded. At present, there have been more than 2,000 students.

Talent plan course system features

During the learning process, many students will compare the Talent Plan with courses such as MIT 6.824 that focus on distributed theory. In fact, the Talent Plan is a much larger curriculum than MIT 6.824. MIT 6.824 is not strictly a database course system. It mainly talks about distributed systems. In Talent Plan, PingCAP re-implemented MIT 6.824 in Rust language as a course component.
According to Wang Yanguang, Talent Plan's curriculum system includes how to use GitHub, how to collaborate with open source, and even basic skills training in Rust and Go languages. To help understand TiDB, the Talent Plan course also provides TinyKV and TinySQL, the teaching versions of TiDB Server.
For students, the courses of Talent Plan are very difficult. For example, if you want to fully understand a module like TinyKV, it will take about two months and four to five hours a day, which is a very challenging course. Therefore, some students will go to MIT 6.824 before learning the Talent Plan. After they have a basic understanding of the Raft protocol, they will start to learn TinyKV. Because TinyKV has Multi Raft, MVCC and distributed transaction modules in addition to Raft, this module alone has a lot more things than MIT 6.824, and the workload of learning is very large. It can be said that it is really a "little flame mountain".

How to ensure the learning experience?

In order to prevent more students from being discouraged by the huge system and difficult content of the Talent Plan course, or give up halfway,
gentle.
First, hold a study camp.

Before the Spring Festival, Mr. Wang Yanguang's team just held a Talent Plan camp. The learning camp is mainly based on online self-study, supplemented by participating in learning and sharing lectures. PingCAP will invite students who graduated from previous years to be mentors to share the pitfalls and necessary knowledge points encountered in their own learning process with other students to help students spend time. Use it for really critical learning. The scale of the learning camp is about 100-200 students per session, and the number of participants in this session before the Spring Festival even reached 400.
Second, the evolution of the Talent Plan curriculum system.

PingCAP is currently working on an iteration of Talentplan 3.0. The goal is also to make students learn the course more smoothly and add some necessary basic knowledge information. At the same time, some short-term training will also be held. For example, in cooperation with the China Computer Federation, a 6-day intensive short-term training has been done at the VLDB summer camp.
In addition, PingCAP will also hold some competitions to attract some students who are excellent in database development and operation and maintenance and have the potential to learn database development. At the same time, course-related documents are also accumulating. But of course, learning is intensive and normalized, and it is also necessary to follow up with "check-in" all the time.
In the future, PingCAP will also turn the Talent Plan into a learning community. This is not just dominated by PingCAP. All participants are stakeholders. Senior members will naturally form a help-and-lead system. Sub-students can also become mentors, and engineers from the front-line industry can also come in to guide development experience. Everyone works together, enjoys together, and builds a new, open, shared, and co-governed learning community led by an open source model.
Third, co-create a Talent Plan book through the Learning Community.

Before, although Talent Plan also had electronic documents, papers, materials, PPT, practical courses, etc., but there was no physical book. The habit of domestic students still hopes to have a more systematic book,
From concept to practice to homework, it is like a textbook. With such a main line, the learning experience will be better.

Embrace open source courses

As a project launched by PingCAP, the most important feature is of course open source. All courses of Talent Plan have been open source since 0.1. After the courses were made, PingCAP immediately put them on GitHub. At present, it has more than 7,000 likes on Github, which may be the highest in the field of database courses.
The subsequent development of the picture completely exceeded PingCAP's expectations. Talent Plan students come from more than 200 schools, including about 50 overseas schools. Since the curriculum is in English, gradually, international schools that have not yet established a partnership with PingCAP, including MIT, CMU, and some schools in Japan and Europe, also come.
picture
The success of Talent Plan is not only because it solves the problems of PingCAP, but also because it solves industry problems. Before the talent situation, all of PingCAP's industry competitors have become friends, and everyone faces the same problem. This is a matter that is beneficial to the industry and the country and society, so PingCAP will continue to invest.

Advice for Talent Plan students

Teacher Wang Yanguang's first suggestion to the students is to "embracing open source". In the field of distributed databases, there are actually a lot of learning materials on the Internet and GitHub. If you don't embrace open source, it may be difficult to enter this field. Furthermore, if the students are engaged in the development of distributed databases in the future, the way of work is likely to revolve around open source methods, and embracing open source is the general trend. He believes that the resume of students in the new era when applying for a job is no longer a page, and it is likely to be his GitHub ID. Which open source projects he has done, which issues he has mentioned, and which projects he is a contributor can be seen at a glance from GitHub, which is more authentic and credible than a traditional one-page resume.
At the same time, open source projects often refer not only to open source products themselves, but to the larger open source ecosystem. For example, the TiDB ecosystem contains many peripheral tools, big data tools, and even distributions made by other companies. These ecosystems all need development talents. If you are familiar with the core products in the ecosystem, you will become a very popular talent in the ecosystem, and the employment space will be very large.
The second piece of learning advice is learning paths. Due to the difficulty of the entire Talent Plan course system, students can start from the stand-alone database courses, such as Andy Pavlo's 15445 course and MIT 6.824 course, which can be used as pre-order basic knowledge.

Meaning of Talent Plan

At present, the challenge of database talent demand is still great, but Talent Plan and the entire PingCAP Education are making unremitting efforts to enable every aspiring student to quickly enter the open source distributed database development ecosystem through an open source learning channel. .
PingCAP uses open source to melt the boundaries between engineering and academia, and change the teaching model. The relationship between teachers and students is no longer a traditional teaching relationship, but an equal relationship between learning and research in the learning community. Welcome more friends to join the Talent Plan, and finally set foot on the platform of "high-level players" of distributed database through learning. You are also welcome to provide suggestions, contribute wisdom, and create a "co-construction and sharing" learning community together!