Youku play black technology | Free perspective technology experience optimization practice

Author: Deng Xiaolong (White Exhibition)

This article is the first article in the series of "Youku Broadcasting Black Technology", "Free View Technology Experience Optimization Practice", after which we will successively launch "WebRTC-based Live Broadcast "Cloud Multi-View" Technology Analysis" "Free View Technology's Full Link Strategy And Landing Practice", welcome to click on the upper left corner [Alibaba Mobile Technology] to follow us, click to follow not to get lost~

"this! Did you watch the fourth season of "It's Hip-hop"? I don’t know if any of my friends have the same "DNA" as the author. In addition to the bursting stage, it is comparable to a cross-dimensional real movie viewing experience, allowing users to be immersive under the effect of a free-view video experience.

Free-view video, as a novel viewing mode in Youku, brings users a brand-new viewing experience. It has also attracted a high degree of attention as a highlight of Youku in many external cooperation. However, with the continuous expansion of product volume, there are still many problems in the overall playback experience and delivery link of the current free view, such as unsmooth playback, unclear content, and low equipment coverage that need to be optimized and resolved.

Based on this, Youku's technical team carried out a comprehensive optimization and upgrade of the free perspective in the first half of the year. In the following content, we will start from the overall goal of comprehensive optimization and upgrade, focusing on the playback experience and user scale, and explain in detail the overall optimization strategy and plan of the Youku player team.

What is free perspective

figure 1

Figure 1 above is the style of each frame of the free-view video, and the following is called the depth map.

Principle of Free View: Based on the original playback link, the free view algorithm SDK is added to process the depth map of each frame to generate a screen with a specified angle and finally display it to the user.

Client architecture design

This part mainly introduces the core logic of the realization of free perspective. The two modules in the middle separated by two dashed boxes are on the left side of the logic implemented in the player SDK for free perspective, and on the right side is the strategy for optimizing the free perspective playback experience. The optimization strategy will be described in detail later.

Play business layer :

1. The core user interaction includes angle rotation gear (mainly generating angle information for use by the calculation party), free-view video user prompts, and transition animations;
2. Online functions can be turned on or off at any time through switch control.

player middle layer: mainly includes two parts, one is the link transformation of the middle layer to support free view, and the other is to download the algorithm files required for free view videos. After the download is completed, the file path is passed to the algorithm layer for use.

player kernel layer : Process the data interaction between the kernel and the algorithm layer, and then synthesize the texture data processed by the algorithm SDK and display it directly on the screen.

: Responsible for the download of Youku on-demand and live video files, without special modification for free perspective, mainly using the downloader's multi-slice download function to improve download efficiency.

algorithm: layer is to reconstruct the depth map based on the algorithm to generate a picture with a specified angle.

Free viewing angle performance optimization program

• optimization direction: First of all, we have to figure out why the carton can know how to optimize. After investigation, the cause of the freeze is caused by the lack of current data of the player. The player needs to wait for enough data before it can continue to play. Therefore, we have concluded that downloading data in advance, multi-channel downloading, and reducing the bit rate of the video are from here. Starting from three aspects can reduce the lag rate;
• solution: early stage of 161a9ce92cba4c, we tried pre-caching, video streaming smart files, kernel dynamic Buffer, multi-channel download, free viewing angle dynamic drop, player dual-instance switching down rate, continuous pre-loading, over-speed mode, video AV1 There are 9 schemes of code reduction rate. After passing the actual feasibility investigation, the four schemes of pre-caching, video streaming smart file, kernel dynamic Buffer, and multi-channel downloading were finally determined.

Optimization practice

Stall rate optimization

Video streaming smart file

The above figure is a schematic diagram of the smart effect. The smart file dynamically determines the bit rate of the next TS segment according to the smart file algorithm, so as to achieve the effect of dynamically reducing the bit rate.

smart file algorithm architecture diagram

There are a few points that need to be highlighted:

• smart file controller with the data source and other modules: collects video metadata and playback status information (such as buffer duration), network information, bit rate/resolution selection at the slice level, and definition switching Control, as well as event response and timeout control on other data source links;
• strategy engine framework: supports an interface/environment/container that supports multiple strategies to run. Each algorithm strategy is implemented according to the input from the player kernel and network environment information, and a defined output is given;
• data link closed loop: client decision-making information buried point report, cloud data analysis and processing, optimized configuration update or model delivery. Among them, the strategy framework and the algorithm strategy implementation of various definition options are the core soul of the entire smart file. The strategy framework provides a platform. At present, Youku's smart file uses ABTest to support everything from discrete rules to enhanced based Learn the realization of multiple algorithm strategies of neural network models. These algorithms can dynamically adjust algorithm parameters according to configuration or model delivery, compare and optimize each other, and complement each other.

Kernel Dynamic Buffer

Through the strategy configuration platform, the specified strategy is uniformly issued to dynamically set the kernel buffer size, so as to achieve the purpose of maximizing the use of download resources.

Multi-channel download

As shown in the figure above, the multi-channel download technology divides each independent file into N small pieces for download, and each small piece corresponds to a download channel on the right, so that multiple channels can be downloaded in parallel to improve download efficiency. In order to achieve the goal of reducing lag.

Pre-cache

1. Realize cacheable broadcast control information and video stream files; (as shown in Figure 2 below)
2. Support strategy for dynamically issuing video pre-buffer size; (as shown in Figure 2 below)
3. Unification of the free viewing angle capability: The configuration of the free viewing angle capability is unified to the broadcast control background, and the client's free view capability configuration items are removed, so as to avoid unpredictable problems caused by the inconsistency of the broadcast control background and the client configuration in the future. (As shown in Figure 3 below)

Figure 2

Figure 3

Scene coverage

Since the free-view algorithm SDK supports two rendering modes, the normal mode based on DIBR, and the degraded mode of turning off DIBR. On the basis of this condition, for devices with insufficient performance that can support DIBR well, it is possible to enter a free view through the degraded mode, which is feasible from a technical and product perspective.

before transformation

after transformation

Data comparison & effect

business & technical optimization effect:

Hip-hop 4 free-view video (one month) has nearly doubled the total broadcast volume of Hip-hop 3 on-demand in the same period last year. Fluency increased by nearly 70%.

scenario coverage income:

1. Not all low-end models can support the degraded mode of free viewing angle, because the scene is special, the resolution of the video input source required by the algorithm needs to be no less than 4K, so the device in the [4k decoding, support DIBR] interval belongs to The scope of expected improvement in this optimization;
2. Through this new degraded rendering technology transformation, nearly 30% of the low-end models were newly added, and the final total coverage increased from 50% (only support mid-to-high-end models) to nearly 80%;

Summarize

Providing users with a better and richer viewing experience has always been our goal of Youku, and it is also our motivation for continuous exploration and experimentation. How to make users feel the temperature of technology, instead of just seeing the icy literal and numerical improvement, to make a better experience that users can feel personally, this is a direction of our future efforts. At the same time, we are also creating a free perspective on live broadcasting, and do our best to explore more and more updated ways of watching movies.

Next week, we will release this series of "Analysis of Live Broadcasting "Cloud Multi-View" Technology Based on WebRTC", thank you for your attention to [Alibaba Mobile Technology], we will continue to talk in the next article.

, 3 mobile technology practices & dry goods for you to think about every week!