Ali266 was first commercialized, helping Youku to save up to 40% in code rate

Alibaba Cloud's self-developed encoder Ali266 was officially launched on Youku in January 2022. This is the industry's first known commercial implementation of the H.266/VVC standard encoder. After two months of actual operation data, after Ali266 is turned on, the bit rate of the original H.265/HEVC scheme can be saved by up to 40% and the freeze rate by 50% under the same picture clarity, which fully proves the commercial value of Ali266. .

This is known to be the fastest H.266/VVC encoder in the world

Ali266 is a codec implementation created by Alibaba Cloud based on the latest generation of international video coding standard H.266/VVC for different scenarios such as on-demand, live broadcast, and real-time communication. The self-developed H.266/VVC encoder S266, which is evolved from Ali266, has achieved 8 firsts in objective tracks and excellent results in subjective track championships in MSU2021 (World Encoder Competition). From the published information, Ali266 is currently the fastest H.266/VVC real-time HD encoder in the world. Combined with the self-developed H.266/VVC decoder, even low-end mobile phones can watch the latest H.266/VVC clearly and smoothly. H.266/VVC HD video.

Ali266 commercialized for the first time, helping Youku

Alibaba Cloud has been exploring and promoting the commercial use of Ali266. In January 2022, Ali266 was officially launched on Youku, helping Youku to take the lead in using the latest international H.266/VVC video codec standard. The actual operation of nearly 2 months shows that after turning on Ali266, the bit rate of the original H.265/HEVC solution can be saved by up to 40% and the freeze rate by 50% under the same picture resolution. Significant gains, fully proving the commercial capability of Ali266.

With its deep cultivation and accumulation in the field of video encoding and decoding, Alibaba Cloud is committed to promoting the practical application of the H.266/VVC standard and the universalization of the latest scientific and technological achievements. Youku is China's leading online video platform. It now supports PC, TV, and mobile terminals. It has various content forms such as copyright, self-production, co-production, self-channel, and live broadcast. It is committed to using richer word-of-mouth content and more extreme content. audio-visual experience and more innovative interactive gameplay.

Behind the provision of these application experiences is a series of very complex technologies, of which one of the most core and basic technologies is video codec technology. To this end, Alibaba Cloud worked closely with the Youku team to commercialize Ali266 for the first time, and immediately launched the latest international video codec standard H.266/VVC, allowing Youku to fully enjoy the technology brought by the upgrade of the video codec standard. dividend.

The origin of the H.266/VVC standard

Speaking of the origin of the H.266/VVC standard, I have to mention the two giants in the world: the International Organization for Standardization and the Moving Picture Experts Group MPEG (Moving Picture Experts Group) under the International Electrotechnical Commission ISO/IEC and the International Telecommunication Union Telecommunication Standard VCEG (Video Coding Experts Group) under the sub-bureau ITU-T. These two giants have experienced more than 30 years since they released the first-generation video codec standards MPEG-1 and H.261 in the early 1990s. The well-known H.264/AVC and H.265/HEVC are the results of the cooperation between the two giants.

Since the release of H.265/HEVC in 2013, the two major international standardization organizations, ITU-T and ISO/IEC, have continued to work together to release the first official version of H.266/VVC in July 2020. As shown in Figure 1, H.266/VVC provides a number of advanced video coding tools including image blocking, prediction, transformation, quantization, entropy coding, and loop filtering. Compared with the previous generation H.265/HEVC standard , which roughly doubles the compression ratio.

Figure 1: H.266/VVC underlying technology

Under the leadership of Dr. Ye Yan, Alibaba Cloud has been deeply involved in the formulation of the H.266/VVC standard since the beginning of 2019. It took a year and a half to submit nearly 100 technical proposals, and a number of technical proposals were adopted, making important contributions to the formulation of international standards and establishing a solid technical foundation for Alibaba's business development. During the formulation of H.266/VVC, members of the Alibaba Cloud standards team also acted as the heads of several panel discussion groups (AHG) and several core experiments, raising the voice of the Chinese video industry on the international standards stage.

Ali266: Complete H.266/VVC real-time HD encoding and decoding capabilities

While closely following the standardization process of H.266/VVC, Alibaba Cloud also immediately launched the Ali266 codec development project to create a complete self-contained codec capability and better open up the end-to-end ecosystem. Provide the industry with a new generation of video codec solutions.

On the decoding side, the Ali266 decoder improves the decoding speed through a series of engineering and algorithm optimization methods such as multi-core parallelism, assembly optimization, memory usage and memory access efficiency optimization. Mid-to-low-end mobile phones with an integrated Ali266 decoder can also watch the latest H.266/VVC videos or live broadcasts clearly and smoothly. The flagship models released in recent years can even support 4K@30fps real-time decoding.

In addition, the Ali266 decoder fully considers the needs of commercial software in terms of stability, robustness, memory usage, etc. After large-scale tests on Android, iOS and other high-, medium- and low-end devices, the robustness of the Ali266 decoder is verified. performance, stability and commercial availability.

On the encoding side, in order to meet the needs of different business scenarios, the Ali266 encoder currently supports four grades: slow, medium, fast and faster. Among them, the slow grade is mainly suitable for offline applications, and the encoding speed is comparable to the x265 veryslow grade.

In the 2021 World Encoder Competition hosted by MSU, the Ali266 slow grade was named S266 by the Alibaba audio and video team, and won 8 of the 14 evaluation indicators of the FullHD objective performance track. 2 and 2 are third; in the subjective performance track, S266 won the first place by a large score among the 16 participating encoders in the world. Under the same subjective quality as the benchmark encoder x265 officially designated by MSU, the bandwidth is saved. as high as 71%. The results of the MSU competition show that S266 is the only encoder with all indicators in the top three, which fully proves the comprehensive and balanced encoding performance of S266 under all indicators.

In order to realize the practical application of the H.266/VVC standard and the universalization of the latest scientific and technological achievements, through the continuous efforts of the Alibaba Cloud team, Ali266 will become the industry's first public 720p real-time high-definition H.266/VVC encoder in June 2021. , and became the industry's first H.266/VVC encoder to open 1080p real-time HD in September of the same year.

Compared with the x265 medium grade, under the same quality, the Ali266 faster grade achieves a bandwidth saving of more than 40%, bringing huge bandwidth dividends to real-time video applications such as live broadcast and video conferencing. As the first real-time high-definition H.266/VVC software encoder, Ali266 has taken an important step in the industry, proving that the existing computing power platform can allow users to fully enjoy the powerful compression of the latest international standard H.266/VVC ability.

The technical solution behind Ali266's first landing on Youku

Mobile access solution

The Youku playback kernel highly encapsulates the playback process into several modules: Consumer, open Source, open Decoder, and openRender. openSource gets the original data from the network, demultiplexes it into audio and video es data and manages the cache. The Consumer continuously takes the data out of the cache and sends it to openDecoder, and gets the decoded data from openDecoder, and then sends the data to openRender. Rendering and post-processing. Their general relationship structure is shown in Figure 2:

Figure 2: Youku player architecture and Ali266 technical solution

The interaction between openDecoder and decoder is shown in Figure 3. openDecoder will send the data to be decoded to the decoder, and then receive the decoded data from the decoder. Neither Send nor Receive block the playback process. The decoder maintains the input and output buffers internally, Send copies the data to the input packets, the worker decodes the data from the input packets, the decoded data is stored in the output frames, and then the Receive can get the valid data for rendering.

Figure 3: Details of the interaction between Youku player and Ali266

The cross-platform, extensible, and high-reliability features of the Youku playback kernel ensure that Ali266 can land on the Youku platform quickly and smoothly. The playback kernel is developed at the native layer and has natural cross-platform features. Therefore, after accessing Ali266, it can be quickly released on multiple platforms without much additional development. Assure. openDecoder encapsulates all decoding-related logic, and only exposes a few simple interfaces to the core to complete logical interaction. Therefore, to access the Ali266 soft decoder, you only need to implement the key api of openDecoder, and connect the complex logic of the decoder and the player. Stripped apart for quick access to new codecs. The playback core implements an intelligent and smooth frame dropping system, which can drop some non-key frames in extreme cases to ensure smooth video playback. With the complete quality monitoring system of the playback core, the online playback quality of Ali266 is fully monitored and effectively guaranteed, making the upgrade of the latest international codec standard on Youku as smooth as silk.

Server access plan

Youku transcoding server currently supports different encoders (H.264/AVC, H.265/HEVC, etc.), different resolutions (270p to 8K), different frame rates (25fps/60fps/120fps, etc.), different package formats ( mp4/hls/cmaf, etc.) and other hundreds of combinations, the server can deliver different streaming data based on the end-side playback capability to ensure the best user experience.

In the Youku transcoding system, Ali266 access mainly includes encoder access, decoder access, and support for H.266/VVC in the encapsulation format. The modules involved and the transcoding process are shown in the following figure:

Figure 4: Ali266 access to Youku server module diagram

Encapsulation layer: Youku's commonly used encapsulation formats are mainly MPEG-TS and MP4. The server implements support for H.266/VVC code stream encapsulation and decapsulation in the Demux and Mux modules.

Encoding and decoding: The access to encoding and decoding is completed by registering and calling the API of Ali266Encoder and Ali266Decoder in the transcoder. In the production link, in order to correctly detect the parameter information of H.266/VVC code stream, the Probe, Parser capability and Ali266Decoder for H.266/VVC code stream are integrated in the Demux module.

In order to make the Ali266 encoder better adapt to Youku application scenarios, Youku server and Alibaba Cloud team cooperated with each other to make targeted adaptations:

(1) In order to improve the playback effect of H.266/VVC video, the encoder supports the ability to repeatedly deliver coding parameter information, which ensures the user experience when fast forwarding, rewinding and dragging playback.
(2) For the quality and smoothness of the video, Ali266Encoder has been optimized for bit rate control. While ensuring the video quality, the fluctuation of the bit rate is reduced, and the freeze rate at the playback end is compared with H.265/HEVC. Reduced by 50%.
(3) In order to adapt to Youku's smart file multi-definition smooth switching with bandwidth, the time stamp processing is optimized in Ali266Encoder, so that audio and video time stamps are compatible with other format streams (H.264/AVC, H.265/HEVC). ) are consistent to ensure seamless adaptive switching of Youku smart files.
(4) Realize that Youku's existing MP4/MPEG-TS streaming media encapsulation format supports H.266/VVC standard. Since the H.266/VVC standard is released soon, various media processing tools have not yet supported the encapsulation of H.266/VVC. To this end, Youku developers use the officially defined CodecId in the PMT of MP4 box and TS according to the official MPEG documents, and cooperate with the player to open up the entire production and playback link.

In addition, Youku also defines the resolution-definition level for H.266/VVC in the production chain, and the playback server can adaptively select H.266/VVC based on the end-side support capability. The video stream of the appropriate grade is distributed, so that the H.266/VVC encoding is implemented on the Youku production and playback link as a whole, and the expected effect is achieved.

The industry's first H.266/VVC commercial landing results

Ali266 was officially launched on Youku mobile terminal in January 2022, becoming the industry's first known commercial implementation of H.266. As of March 23, it has been played more than 600,000 times. The indicators are as follows:

• Code rate: Under the same definition, the code rate is reduced by up to 40% compared to H.265;
• Experience: Under the same conditions, the freezing rate is only 50% of that of H.265, the average decoding time is about 8ms, and the stability exceeds 99.95%;
• Power consumption: H.266 soft-decoding playback can be played for 10 hours at full power, saving traffic up to 1G;
• Content: Covers popular variety shows, movies, TV dramas and other scenes, such as movies such as "Antique Bureau Middle Game" and "Changjin Lake", variety shows such as "Happy Twist", and TV series such as "The Truth".

In the future, continue to evolve the commercial road of Ali266

It is understood that in the second half of 2021, MediaTek released the world's first TV SoC chip Pentonic 2000 built on a 7nm process. It is also the world's first chip to support H.266/VVC video decoding, with 8K/120Hz decoding capabilities. This is a huge help for improving the ecology of H.266. It can be said that H.266/VVC is not in the future tense, but has actually entered the present tense.

Ali266 closely follows the latest technology development trend and is closely integrated with the audio and video industry, which not only provides Youku with industry-leading encoding and decoding technology, but also gives you more business possibilities. In order to further support Youku's rich content forms and business scenarios, tap the dividends brought by new technologies, and promote the continuous development of the video industry, Alibaba Cloud will work closely with Youku next to focus on 10bit HDR and Youku Free View. , ultra-high-definition 4K~8K, high frame rate 60fps~120fps and other video content, and further expand from the on-demand scene to the live broadcast scene, continue to promote the technological innovation of H.266/VVC codec, continue to explore innovative audio and video technology, strengthen its Universality in more video applications and more terminal devices, more technologies will be beneficial to consumers, enabling the video demand of thousands of industries, spawning new product forms and business models, bringing audiences More extreme audio-visual experience and more innovative interactive gameplay.

---

"Video Cloud Technology", your most noteworthy public account of audio and video technology, pushes practical technical articles from the frontline of Alibaba Cloud every week, where you can communicate with first-class engineers in the audio and video field. Reply to [Technology] in the background of the official account, you can join the Alibaba Cloud video cloud product technology exchange group, discuss audio and video technology with industry leaders, and obtain more latest industry information.