The judges were amazed and the projects were constantly surprised. What kind of Hackathon was this?丨TiDB Hackathon 2021 Review

Yesterday, the TiDB Hackathon 2021 with the theme "Explore the Sky" ended perfectly! This year is already the fifth TiDB Hackathon event. The scale of participation is the largest ever. A total of 279 people and 64 teams participated in the competition, including office workers from companies such as Tencent, Huawei, NetEase, Meituan, Byte, JD.com, and Didi. There are also students from Peking University, Beijing Post University, Chinese Academy of Sciences, Shanghai Jiaotong University, RMIT and other universities.

In the two-day and one-night Hacking Time, around the four major tracks of core, tools, ecology, and "∞", the contestants came up with many projects that surprised the judges. In the direction of TiDB kernel, many innovative projects with hardcore and greatly improved performance have been made. In the direction of tools, the observability and diagnostic ease of use of TiDB have been greatly optimized. The graph database has opened up a high-speed channel for TiDB to enter the lake. In fact, many projects are already in the Roadmap of TiDB and have the maturity of landing. In the end, a total of 400,000 yuan in cash prizes, and another 10 teams won the Infinite Creativity Award, Campus Team Award, User's Choice Award, Best Market Potential Award, Cloud Application Award, and Points Challenge. Award, Technical Potential Award, Best Popularity Award .

Project Creative Unlimited

This year's participating teams showed their unrestrained imagination and unlimited creativity, and submitted various creative projects to the organizing committee: such as TiDB cold and hot data tiered storage, TiLaker: opening up a high-speed channel for TiDB to enter the lake, TiDB Flashback , TPC TiKV, Sheriff Dark Horse, collie, oom.ai feature platform based on TiDB/TiKV, TiDB Visual Plan... Even our CTO Huang Dongxu has teamed up to participate. His project is pCloud, which is known as iCloud on the database, and even There are also teams with big brains, realizing the dream linkage between TiDB and Minecraft... There are too many projects to list one by one. You can learn about all the projects through the official website of the event.

Slippery egg group chaoscraft Demo
From the quality point of view, the excellent projects were constantly surprised, the contestants called Hackathon more and more volume, and the competition for awards was very fierce. After intense development and high-density lightning defense, a stood out and entered the final defense. There are still 5 teams doing cloud application award defense, and 5 teams doing Infinite Creativity Award defense .

The Demo Show, which is broadcast live by the six cities of Beijing, Shanghai, Guangzhou, Shenzhen, Hangzhou, and Chengdu, lasted from 13:30 in the afternoon to 22:00 in the middle of the night. Although the final defense lasted for 9 hours, the more people watched, the more excited they became. Once the usually reserved and reserved tech giants introduce their products, they turn into eloquent speakers. Put a few scenes, everyone can feel it:

Final defense scene

In the end, after intense selection, the jury finally awarded the first, second and third prizes, as well as the Technical Potential Award, the User's Choice Award, the Cloud Application Award, and the Infinite Creativity Award. Since the awards race is too intense, there are many excellent projects sorry unsuccessful, the following are all winners: slide around to see more

This year's Hackathon judge, PingCAP VP of Engineering Tang Liu, was full of praise for this year's projects, and gave "irresponsible" comments on many projects:

kernel enhancement

: TiDB hot and cold data tiered storage

This project won the first prize of this Hackathon. The integration with another similar project in this Hackathon will lay a good foundation for the integration of TiDB and S3. At least this Hackathon has verified the feasibility. In fact, the principle is very simple. Put the cold data into S3, and then push the operator down to S3 as much as possible, and use the native select function of S3 to speed up the query. Of course, if the data is already in S3, we can also use other services on the cloud, such as Athena, to do more query aggregation operations to speed up the query. This time everyone is working on partitions. After all, partitioning based on time slices is a very common operation. Later, we are also doing some research on integration with S3 through LSM. I am still looking forward to all these. We can see a lot of results this year. For example, our TiDB Cloud Dev Tier cluster can fully use this mechanism to verify first.

Diagnostic Ease of Use Tool

TiVP: TiDB Visual Plan, Slow SQL Diagnosis

When I finally saw the visualized execution plan, I almost burst into tears. After all, it was really hard for us to diagnose slow SQL before. The large-screen execution plan is almost impossible to read, and if we want to compare the similarities and differences between the two execution plans, it will be even more broken. With visualization, at least the efficiency of analyzing where is slow will be greatly improved, and later we can directly integrate the functions of SQL advisor into TiVP, so that everyone can perform SQL bind, add/drop index operations directly online. After reading this project, I immediately asked my classmate wish, and he directly threw me a more beautiful Visual Plan map. It turned out to be in the roadmap, everyone will wait and see.

Ecological extension

TiMatch - A distributed graph database with complete syntax

Last year TiGraph already amazed everyone, and this year TiMatch is even more exciting. This time, the ease of use is better, and it can be directly upgraded and used for old clusters. Because TiMatch only builds a set of graph indexes internally, and then updates it uniformly with the data of the original relational table through the TiDB distributed transaction mechanism. The syntax is based on the syntax of Oracle graph, so the relationship is complete, but I think the challenge lies in the performance. I hope that the next session will show you relevant data.

TiLaker: Opened up a high-speed channel into the lake for

Last year's Hackathon actually had a lot of projects integrated with Flink, but I saw one in the finals this year. To be honest, I'm still a little disappointed. But this year TiLaker is still quite complete. After all, with the participation of Flink committer, everyone has implemented a CDC connector for Flink, which allows Flink to directly read the incremental data of TiDB and synchronize it to the downstream. With the help of Flink's capabilities, TiDB can better connect with the downstream ecosystem, and I hope that there will be many application cases in the future.

pCloud ：pCloud

This is a very interesting project. Your company's CTO, Dongxu, went directly to the stage to bring the goods. Putting aside the great appeal of his personal scene, from a practical point of view, pCloud has done a really good job. Dongxu just showed the product effect and talked about the business model, but I actually know that the underlying implementation of this project is still very challenging. However, this also gives another reference to the students participating in the next Hackathon. For a project, it is sometimes easier for everyone to pay attention to the technology itself, but if we are making a product or a SaaS service, the understanding of users and the business understanding is also crucial. So even if you feel that you don't have much understanding of TiDB and can't write too hardcore programs, you can make breakthroughs from other directions.
...The thousand-word article is omitted here, and the above views only represent the personal views of the chief architect.

Congratulations to the winning teams and players, please scroll down

Six cities are connected, code force gathers

On the early morning of the 8th, teams from all over the world began to arrive at PingCAP's event sites in various places to sign in, take photos and receive commemorative gifts. The contestants included office workers from companies such as Tencent, Huawei, NetEase, Meituan, Byte, JD.com, and Didi, as well as students from universities such as Peking University, Beijing Post, Chinese Academy of Sciences, Shanghai Jiaotong University, and RMIT.
![]()
The contestants raced against time to prepare the project, or engaged in enthusiastic group discussions, or non-stop Coding... In order to deliver the project that satisfied themselves and the judges within the specified time, they did their best!

The youngest player is only 8 years old, and programming starts from a teenager~

Distributed event guarantee, food and drink must not be

A hard-core hackathon requires food and drink. In order to ensure the smooth running of the six-city distributed event, the volunteers prepared various afternoon teas, lunches, and dinners for the contestants, and shuttled through the arena to provide distributed logistics support for everyone.

judges and teachers

This year's Hackathon projects are more exciting than each other. The judges and teachers have also worked hard to watch every detail of the project. After the demonstration, the judges and teachers will also ask questions on the spot, and there is no shortage of suggestions for improving the project.
slide around View more
Special thanks to Chen Su, CTO of DataPipeline, Chen Yu, partner of Yunqi Capital, Dai Wen, head of PingCAP TiDB Cloud, Feng Guangpu, head of the multi-point Dmall database team, PM of PingCAP tool team, Guan Shengliang, co-founder and technical VP of Taosi Data, and Apache Foundation Member Guo Wei, Head of Meituan Database R&D Center Li Kai, SphereEX Co-founder & CTO / Apache ShardingSphere PMC Pan Juan, Xiaomi Big Data Committee Secretary-General Pan Yingchao, Google Cloud Data Management Specialist Qiao Mu, Digital China Group Vice President and CIO Shen Yang, PingCAP VP of Engineering Tang Liu, Apache Software Foundation Director/Tetrate Founding Engineer Wu Sheng, Huachuang Capital Partner Xie Jia, Mingshi Capital Executive Director Xu Zhihao, PingCAP Chaos Engineering Team Leader Yin Chengwen, Shanghai Jiaotong University Distinguished Twenty-two teachers, including professor/doctoral supervisor/ACM class founder Yu Yong, Apache Pulsar PMC member/StreamNative co-founder Zhai Jia, PingCAP DataPlatform PM Zhai Yulong, PingCAP R&D director Zhang Jian, and Datafuse Labs co-founder Zhang Yanfei, etc. review.

competition, we will also interview the outstanding project teams to introduce their project design ideas, implementation process and future work direction in depth, hoping to bring you some inspiration. Stay tuned!
Finally, I would like to thank the sponsors and partners for their strong support for the event, and thank the volunteers for their dedication! See you next year!