Rongyun 2021 X-Meetup Technology Salon discusses the new direction of audio and video technology

On May 15, 2021, the third stop of Rongyun X-Meetup Technology Salon will continue to Shanghai. This salon focuses on the "new direction of audio and video technology". It is composed of Jiang Chunyu, senior engineer of Rongyun Audio and Video R&D, Xu Jing, founder and CEO of Time Robot, Liu Jia, senior engineer of Rongyun IM R&D Center, and Li Yalong, architect of Xueersi Online School. With senior audio and video technology expert Li Wei, and five technical experts as speakers, they exchanged and shared new observations about audio and video technology with developers from the perspective of popular application scenarios and proceeded from technical practice.

Audio development on iOS

This year, due to the demonstration effect of Clubhouse and Tiya, the product of Yuchaofang has become popular, and the development technology of audio has attracted the attention of developers. Jiang Chunyu, a senior audio and video R&D engineer from Rongyun, has been focusing on technology R&D in the field of mobile and audio and video for many years. He shared the theme of "iOS Audio Device Development-Core Audio".

Jiang Chunyu, senior engineer of Rongyun Audio and Video R&D, delivered a speech

Jiang Chunyu believes that the difficulty of mobile audio processing lies in sound beautification, voice change, real-time high sound quality and diversified scene play. From the perspective of iOS devices alone, to break through these difficulties, it is inseparable from the Audio Unit provided by iOS. It is a powerful and flexible audio processing technology that supports mixing, equalization, format conversion and real-time input/output for recording, Play, offline rendering and real-time conversation.

Based on Audio Unit, Rongyun SDK builds multiple functional modules such as long sound effects and short sound effects, and finally completes the audio mixing output on the audio device. In scene-based practice, Jiang Chunyu shared two typical scenes of a music chat room and a large conference room with hundreds of people as examples, and shared the technology development and optimization plan of Rongyun SDK. For example, the music chat room pays attention to high sound quality, beautiful voice change, and comfortable noise is better. Developers need to optimize algorithms according to these needs; while the optimization of super large conference rooms requires intelligent streaming on the server side and simultaneous multi-person voice. The voice that can intelligently select the conference speaker appears.

Jiang Chunyu concluded: Audio Unit is a powerful audio processing framework. Audio processing must be based on the Audio Unit framework to build content and continuously polish and optimize the audio processing content. In the future, Rongyun Audio and Video SDK will continue to develop new functions based on the needs of different scenarios, continue to optimize audio products, and provide developers with better solutions.

Build a low-latency and high-reliability signaling system

As a leading manufacturer of the Internet communication cloud track, Rongyun will be the first in the industry to propose the “IM+RTC+PUSH” overall communication solution in 2020. The channel for Rongyun RTC to arouse users is SDK signaling that relies on IM. Therefore, Liu Jia, senior engineer of the IM R&D Center of Rongyun this time, shared the "Exploration and Practice of Building a Low-Latency and Highly Reliable Signaling System" to help development To better understand how Rongyun IM collaborates with RTC to provide highly reliable communication capabilities.

Liu Jia, Senior Engineer, Rongyun IM R&D Center

Liu Jia introduced that when constructing a high-reliability audio and video signaling system in the design of an IM signaling system, the first step is to layer services, including the access layer, internal services, and data storage. The principle of separation should be divided into API and CMP according to business differences and different service objects, so that the whole can be monitored and maintained. Secondly, it is necessary to build a complete monitoring system to monitor the performance of the network through visual charts and deal with system bottlenecks in a timely manner.

Regarding the realization of the low-latency signaling system, Liu Jia shared that Rongyun not only uses the global acceleration network to reduce network latency, but also reduces the amount of data transmission based on Rongyun's own communication protocol, and uses the cache mechanism to increase the service processing speed. . In addition, Liu Jia took cache design as an example to illustrate that improving cache hit rate through consistent hashing, efficiently using CPU processing power, and implementing asynchronous storage are all key to achieving low-latency system design.

Based on these design points, Liu Jia demonstrated the language chat room system architecture in the scenario of massive concurrent users, providing developers with dry goods solutions. At the same time, he also summarized the three advantages of Rongyun's existing audio and video overall service architecture: First, the signaling service and the media service are decoupled, and there is no need for state synchronization between the two services; second, the media service is focused Communication and signaling services are focused on capabilities; thirdly, deployment is simple, which facilitates global deployment of media services.

The architecture design of the live broadcast system meets the needs of users for real-time performance

In this salon, live audio and video scenes are also a key topic. Xu Jing, the founder and CEO of Pickup Robot, who has been deeply involved in Internet audio and video for 12 years and has accumulated rich practical experience in the field of live broadcast, shared the architecture design based on live broadcast answering scenes through his "Internet live broadcast fast actual combat". The key technical points and countermeasures, as well as how to ensure the quality of the video and audio in the live broadcast, are explained in detail.

In the salon, Li Yalong, the architect of Xueersi Online School, who focuses on online education, also brought developers a sharing of "Online Education Live Broadcast System Architecture Upgrade" for the live broadcast scene of low-latency education classes. He focused on the development of online school video technology, online school large class live broadcast system, online school public welfare live broadcast class, and low-latency live broadcast to explore the design points of these four aspects, and analyzed and explained them. For developers who focus on online education, it has universal demonstration significance.

In addition, Li Wei, a senior audio and video technology expert, delivered a speech on "Using WebRTC to Build Real-time Online Classrooms". Li Wei used to work at the Institute of Computing Technology of the Chinese Academy of Sciences and CC Video. During his tenure, he used WebRTC technology to develop commercial products such as live broadcasts, online classrooms, and video conferences. The number of concurrent users reached 5 million. He also wrote "Detailed Explanation of WebRTC Technology: From 0 to 1 to Build a Multi-person Video Conference System". He has many years of practical experience in this field and has very in-depth research on WebRTC. The sharing of his practice has also benefited developers. Very shallow.

Conclusion

In this salon, the five lecturers shared the common feature: they all started topics based on current hot scenes. It can be seen that application scenarios are the basis for "discussing new directions in audio and video technology", and the more popular the scenarios, the more likely it is that the development potential of this field is greater, so the more it needs to be carried by new technologies and new products. .

With the further implementation of 5G and the continuous optimization of network bandwidth and network quality, there will be more possibilities for audio and video communication in terms of usage levels and usage scenarios. As far as developers are concerned, only by storing as early as possible, mastering as many new technologies as possible, and paying attention to new directions can they win in the present and win in the future.