Think outside the "box" and talk about the works of the RTE Application Innovation Competition

"RTE 2021 Programming Challenge" co-sponsored by Agora and Huanxin has ended successfully. In this year’s competition, experts from different fields in the industry participated in the final selection, including Bai Huancheng, technical leader of the Linux Chinese technology community, Gong Yuhua, senior iOS development engineer at Agora, and Li Chao, chief architect and lecturer of Beijing Audiovisual Beating. 、SegmentFault CTO Qi Ning, Huanxin CTO Zhao Guibin, Shengwang Agora Web R&D Engineer Zhang Qianze, and Shengwang Agora SDK Senior Architect Zhang Zhen. The entries have been evaluated from different perspectives.

We also found several judges and chatted with them about their impressive innovative applications and their views on the development of real-time interactive scenarios in the future.

Works that impressed the judges

This year, from the 200+ teams, 46 teams participated in the finals. The creativity, technical realization, and potential value of these works, each with their own strengths, left a deep impression on the judges.

such as "Urban Explorer" item . It is a social product built using map LBS + cloud signaling SDK + real-time audio and video. Through geographic location, different people can find like-minded partners during the journey and explore the city together. The multi-person language chat room scene is realized in the product, and friends can communicate through voice. Judge Qi Ning said, “They combine games with real-life adventures and have a relatively good experience. They also make good use of the real-time communication capabilities of the sound network. A little improvement is a promising commercial project.”

Another example is the implementation of virtual anchor project "Vchat" on the Web. It uses the WebGL engine of tensorflow.js as the backend, and uses the existing open source face recognition model to recognize the position of the face and the state of the facial features through the camera. Then update the face data to the virtual 3D model in real time through Three.js and Vrm.js. In fact, Zhang Qianze, one of the judges, has developed a virtual anchor on the Native side, but this is the first time he has seen someone use Web technology and machine learning to implement a virtual anchor, and the skeleton can be replaced, which is very interesting.

There is also the first prize of "Agora Home AI" with the theme of smart home. The judge Li Chao gave an evaluation. "It combines audio and video, AI, and the Internet of Things, and brings us more Imagine the space, I believe that this product can become a phenomenon-level product after more detailed marketing." It uses Cloud Signaling SDK to realize remote control of IoT devices. At the same time, the 1V1 video of human and machine is realized through the Agora RTC SDK of the sound network, and the video collected by the robot is sent to the PC control center to perform AI intelligent detection and trigger event response.

In fact, whether it is impressive or not has nothing to do with whether you get the award or not. Sometimes the original intention of the project can also move people. For example, for the development of "transliteration" for the hearing impaired and visually impaired, the judge Bai Huacheng believes that our Internet has done a lot for "sound people" in recent years, but we have done too little for those people who really need attention and help, so For such a project, we deserve more attention.

RTE scenario potential is much more than this

year's contestants submitted works very enthusiastically. Most of the works are closely integrated with society and life. It can be seen that everyone really wants to solve some practical problems and develop . 〃

Qi Ning's comments after seeing the contest entries

In fact, there is still a lot of room for imagination in the real-time interactive track. For example, the judge Zhang Ganze mentioned a scene that is relatively close to everyone, that is, games and immersive audio and video. Game and immersive audio and video experience interaction should be the evolution direction of real-time audio and video in the future. still stayed in the mode of "prepare a fixed environment, and then everyone joins and interacts with the set gameplay". However, the future model may place more emphasis on direct interaction between users. Users can directly interact and create gameplay, and then obtain an immersive experience, truly making real-time audio and video as ubiquitous as air and water.

has never seen that audio and video will become a basic service. How to better integrate it with other technologies should be a major technical direction in the future . The judges Li Chao and Qi Ning both believe that if you look at real-time audio and video, there are still many scenarios to be explored, such as the use of real-time audio and video technology to replace the cumbersome SMS identity authentication mechanism; in the elderly-oriented scene, there are also many participants developing this time. The work, but the idea is still mainly around audio and video calls, chats, etc., in fact, more innovations can be made from the perspective of humanization.

Of course, the judges also encourage everyone to jump out of the real-time audio and video thinking of "imprisonment". Bai Yucheng made an idea, "You can look at it from the perspective of computer implementation. When we transform all projects that require polling to achieve real-time , What kind of changes will happen to products and businesses? In addition, in addition to the transformation of traditional businesses, it can also be the addition of some digital scenarios. There are many opportunities here, depending on whether the entrepreneur is willing to catch it."

There are many things worth exploring in the real-time interactive scene. Every innovation may make the user's interactive experience more immersive. And real-time interaction is not limited to audio and video interactions, jumping out of the "thinking box" may also open up a new blue ocean. If you have any ideas, please leave a message to communicate with us.

Notice

The winning teams of this year will share their development experience in detail one by one. The follow-up will be posted on the "Agora Developers" public account, you can scan the QR code to follow.