Rongyun CTO Ren Jie: Strong interaction, RTC's next "burst" scene

Foreword: Real-time audio and video RTC (Real-Time Communication) is a technology and industry that has rapidly spread and emerged in the black swan incident of the new crown epidemic. Capital overweights and explosive applications have enlarged the imagination of real-time audio and video. Ren Jie, the CTO of Rongyun, a safe and reliable global Internet communication cloud manufacturer, recently accepted an interview with the media. What are the key points and innovative use scenarios of RTC technology, what industry will RTC detonate in the future, and where are the opportunities for developers, and explained himself the opinion of.

Rongyun CTO Ren Jie

RTC's new challenge

Before the popularization of RTC, video conferences and video teleconferences on tall buildings required dedicated networks and proprietary hardware. Around 2012, with the construction of 3G and 4G networks and the popularization of WiFi, RTC technology began to enter the Internet. However, Ren Jie mentioned that various mobile phones, PCs and other devices have begun to enter the field of video communication on the Internet, and the devices have become uncontrollable. In addition, from a proprietary network to a wide area network and the Internet, as well as a wireless network, and a 4G network, the network has become uncontrollable.

"The PC is not designed for the communication scene, so it has different processing at the level of echo cancellation, noise reduction, and video capture and encoding. There are many device adaptations, algorithm re-adaptation, and a lot of fine functions to be done; mobile phones In a wireless network, packet loss and bandwidth will change instantaneously. These are new problems faced in new scenarios. We have to solve them again, using new algorithms and new models to deal with them." Ren Jie said.

In early 2021, W3C (World Wide Web Consortium) and IETF (Internet Engineering Task Force) announced that WebRTC has become an official standard. Ren Jie believes that WebRTC is accepted and recognized by standardization organizations, but it does not mean that RTC technology has matured. "If an industry continues to erupt and continue to develop at a high level, it is difficult to say that its technology is mature, because there are more and more scenarios, and more and more equipment technologies are used. Audio and video coding is constantly changing, and under new scenarios The network confrontation, machine learning, and noise reduction technology of China are all in the process of continuous blowout." Ren Jie said.

New technologies and scenarios have also brought new challenges. Ren Jie summarized four RTC challenges facing Rongyun at this stage:

1. High-definition video: With the upgrade of infrastructure such as WiFi and 5G, users have higher requirements for audio and video quality. Rongyun will follow the new coding standards to provide high-quality, high-bit-rate, and high-frame-rate video experience.

2. Voice reproduction: Traditional algorithms can no longer meet the existing high-quality and high-reduction user experience and requirements. RTC will be combined with AI to develop technologies to reduce noise, improve intelligent resolution, and improve overall visual clarity. This is A major direction of Rongyun RTC technology research and development.

3. Adaptation of a large number of smart devices: In addition to Android, there will be a large number of real-time operating system RTOS devices entering the RTC field, and some of these devices have lower configurations and uneven standards. RTC manufacturers need to adapt to ensure interactive experience.

4. Global node deployment to serve developers going overseas: The demographic dividend of the Internet market is gradually disappearing. Developers going overseas have a great demand for audio and video services. Rongyun is deploying global nodes and better scheduling algorithms to ensure the world Any place, any way of access, there is a superior audio and video experience.

Three innovation scenarios of RTC

Although the epidemic has greatly popularized RTC, Ren Jie believes that the epidemic itself has not created a new RTC scene, but has promoted the implementation of audio and video in various scenes and industries, and has accelerated the development of RTC.

He gave an example that online education is undoubtedly an RTC scenario that has been promoted and widely implemented during the epidemic. In addition to the online courses of k12 (kindergarten through twelfth grade, abbreviation of preschool education to high school education, referring to basic education), quality education is now also migrating online. However, different from basic education, quality education involves the transfer and guidance of experience and is a more personal process, such as music and painting, which require relatively high process requirements. The improvement of RTC technology and user experience will help the online process of quality education.

Another innovative scenario for education is online examinations. Examinations have always been a serious matter, and the process control has become more stringent. Due to the impact of the epidemic, the examination has also begun to be online. For example, the examination room is equipped with a front camera and a rear camera, and the whole process is recorded to facilitate supervision and spot checks after the exam.

In addition to education, Ren Jie mentioned that the second RTC innovation scenario will be a strong interactive and entertainment application. E-commerce live broadcasts, shared movies, online auctions, 3D virtual image voice social networking, online chorus, etc., these strong interactive applications will become more and more, it is worth developers to use their imagination from the product level.

The third innovative scenario is in industry applications. After the epidemic, various industries, including government services, finance, insurance, transportation, customs, etc., are required to "contactless services". They are all shifting the process of handling affairs and approval services to online. These are also RTC is a very important new scene.

When AI meets RTC

Regarding the integration trend of RTC, AI, 5G and other new technologies, Ren Jie introduced the solutions to four types of problems in the audio and video fields by AI technology.

AI intelligent noise reduction. For example, in the online education scenario, the sound of (teacher/student) hitting the keyboard, pouring water, and coughing, the past algorithms are powerless to these unsteady noises, and AI technology can distinguish more scenes and distinguish noise, which will help RTC Great improvement in noise reduction;

AI super-resolution processing. Video images transmitted under the same bandwidth can be decoded and restored with AI technology to make them clearer than before;

Use AI technology to distinguish and separate different video scenes or audio scenes. For static scene coding, definition and frame rate control, for dynamic scene coding and frame rate control, AI technology can be used to distinguish them, and different processing can be done; and AI technology is used to reconstruct the bandwidth estimation algorithm.

For the more distant future, in the era of the True Internet, Ren Jie said that the entire process of audio and video from acquisition to encoding, transmission, decoding, rendering, and interaction will be a large piece of capabilities that the True Internet must have, and RTC is expected Become the infrastructure of the True Internet.