Event Review | Intel-based End-to-End Audio and Video Optimization

Guide: On August 18, 2022, "The horizon is boundless, and the sound is born together", the 2022 NetEase Audio and Video Technology Conference was successfully held. Product and technical experts share special topics. This article is "End-to-end audio and video optimization based on Intel technology" shared by Zhang Liyu, a technical expert from Intel.

Guest introduction: Zhang Liyu, Intel Internet industry solution expert, has long been engaged in the construction of audio and video infrastructure, including video encoding, video analysis, performance optimization and other directions.

---

With the optimization and upgrade of live broadcast technology, video interaction and smooth video have become standard. In this process, the traditional Internet information flow mainly based on text and pictures is gradually replaced by the multimedia flow represented by audio and video. Taking the current hot short video service as an example, by the end of 2021, the stickiness of Internet users to short video services has become the first among all Internet services, with a total usage time of 25.7%. For most ultra-clear audio and video live or recorded broadcasts, how to optimize it to provide users with a better audio-visual experience is a big issue.

Over 30 years of Internet development has brought earth-shaking changes to China and has become an important driving force for China's economic development. The digital economy continues to grow rapidly, and the integration of information technology and the real economy accelerates. The digital economy has become a new engine for China's economic growth.

In this process, the application of audio and video has become more and more extensive, and visual information has replaced traditional words, pictures, symbols, etc., which more intuitively reflects the essence of information and facilitates more direct information exchange. According to specific data, by 2022, China's ultra-high-definition video market will exceed 4 trillion.

In general, with the development of new technologies, audio and video technology needs to be continuously upgraded to support new application scenarios, new models, and new application needs. Therefore, for the entire audio and video field, it is not only necessary to optimize the upper-layer software system, but also to find new solutions in the underlying hardware technology facilities, so as to have a truly subversive innovation in the field of audio and video. Technology upgrade.

The deployment of large-scale artificial intelligence also puts forward requirements for future graphics processing products, requiring a high degree of visual assurance, continuous optimization of artificial intelligence and continuous computing, and also brings a strong demand for data centers in the cloud era. The new architecture, the realization of intelligent computing, the processing of massive data, the integration and acceleration of easy-to-configure computing power, the support of multi-load operation and maintenance, etc. Intel has established a software acceleration system, GPU roadmap, spanning all products from the client to the data center, to help customers meet the different needs of the data age. From Intel's point of view, the first is to build audio and video infrastructure through hardware capabilities. Intel has been working hard to provide developers with the right combination of hardware architecture and software in order to provide them with a specific optimal solution. Intel's XPU strategy is not a new processor or product, but a combination of architectures, including different acceleration and processing chips such as CPU, GPU, FPGA, and ASIC, corresponding to different application loads and power consumption requirements.

Intel's optimization solution for voice and video processing can be divided into two layers. The underlying infrastructure provides support for the entire cloud platform video cloud, and secondly, the specific workload of the upper video cloud is optimized. In terms of hardware infrastructure, Intel through Intel® Xeon® Scalable processors, Intel® FPGA products, Intel® Optane™ persistent memory, and Intel® Ethernet network adapters, Intel® Visual Cloud Media Analytics Accelerator Cards and other products, Provide powerful computing, storage and network processing capabilities for various innovative solutions based on audio and video capabilities; on the other hand, in terms of software optimization and acceleration, Intel's Media SDK, SVT, Intel® oneAPI, etc. A complete software stack to accelerate the performance of audio and video capabilities.

In addition, Intel also has many integrated data function libraries that provide a lot of performance optimizations. For example, Intel's IPP can quickly implement discrete cosine transforms in the second process of a function. At the same time, the amount of code is greatly simplified. By using the IPP function library, optimizing the algorithm, minimizing the memory usage, and adding the Intel AVX512 instruction set, the performance can be improved by 3 times in the discrete cosine transform calculation.

In general, Intel's hardware technology facilities are like performance accelerators. Through the combination of hardware products such as Intel Xeon processors, Intel's FPGA products, and Intel's Optane memory, it provides very powerful technologies for various audio and video-based innovative solutions. capacity, storage capacity, network capacity. The software is more like a system optimizer. It uses a complete software stack to accelerate the performance of audio and video capabilities in different application scenarios. Under the combination of software and hardware, Intel can provide cloud or data center-based audio and video for various platforms. Provide professional and efficient encoding and decoding capabilities, help cloud service provider users to bring more convenient, high-definition, and more affordable audio-visual experience, while enabling cloud manufacturers to more effectively promote system optimization, reduce operating costs, and ultimately A synergistic effect of 1+1 greater than 2 is achieved.

For Intel, the future data center will realize the physical separation of computing and storage, complete the integration of security features in the internal architecture of the chip, realize more flexible hardware, software application services, and software will be presented in the form of microservices. , through ubiquitous connections, from cloud to edge infrastructure and artificial intelligence, to help mankind continue to grow and innovate, and to fully empower the transformation of all things intelligent.