Unveiling the Secret Hema Xiansheng Android Short Video Second Broadcast Optimization Solution

As an important means of carrying content, short videos are the focus of attracting users. The content and experience of short videos are directly related to whether users are willing to stay for a long time. Therefore, the optimization of the experience is particularly important. In the previous article, we shared the optimization of iOS short video second-broadcasting. In this article, let’s talk about the optimization of the Android side.

Author｜Shaoyang
Review｜Taiyi

The Hema immersive short video playback page before optimization has a significant gap with mainstream short video apps in terms of body feel and fluency. The main problems are the splash screen when playing the cover and the slow streaming speed. Therefore, the goal of optimization is to solve the existing shortcomings of Hema immersive short video and align with the immersive short video experience of mainstream apps, such as Douyin and Handtao. The specific indicators are:

• Meet hard indicators: playback success rate, first frame duration, second opening rate.
• Satisfy the user's body fluency. (In order to reflect the real experience of users watching short videos, Hema has added a second-broadcast somatosensory indicator: the time from the user to the video to the first frame of the video.)

Optimization effect comparison

First of all, let's take a look at the effect comparison with other apps before and after optimization:
https://www.youku.com/video/XNTgwMzAwNDQ0OA==

environment

• Phone: Pixel 4
• OS：Android 10

• Player: Taobao player

  problem analysis

  Home splash screen

  Hema initially added a cover image display to ensure that it is not a blank page when entering the screen, which is hidden during playback. It can be seen from the somatosensory index that even before optimization, the somatosensory playback time is very short, only about 200ms (not including the sliding process). During the sliding process, it takes about 600ms to display the cover page until the video is officially played, and then quickly display the playback screen. At this time, the user still has a strong screen flicker and frustration, and the experience is extremely poor.
  Solution: The first frame of the video is displayed during the sliding process, and the cover is no longer displayed, so there will be no more frustration during playback. The optimization here needs to be optimized together with the slow flow problem.
  

  Slow streaming speed (slow playback experience)

  : The slow rate of outflow caused by the server is usually caused by large files and poor network links. Available H.265 and CDN accelerated optimization
  client : client playback needs to go through download -> load + decode -> render three steps, and the three steps are executed linearly. Therefore, it is necessary to go through about 1s of preparation before playing the screen in the window. Here you can consider executing download -> load + decode .
  

  Optimization plan selection

  In the early stage of optimization, we considered three optimization schemes.

Solution 1: Dual players + pre-download
Advantages: It occupies a small amount of memory, and the idea is simple.
Disadvantages: The optimization efforts are limited, and it is impossible to take into account both upslip and downslide at the same time.

Solution 2: Customize three-player management + pre-download
Advantages: At the same time, you can turn pages up and down, and experience close to Douyin.
Disadvantages: Player management and recycling are complicated to implement and easily confused; high memory usage.

Solution 3: Three players (implemented based on RecyclerView's caching mechanism) + pre-download
Advantages: At the same time, both up and down pages are turned, the experience is close to Douyin, and the caching mechanism is hosted by RecyclerView.
Disadvantages: high memory usage, frequent creation and destruction of players.

Finally, because of cost-effective factors, the third option was chosen.

Principle of Scheme 3: Before turning the page

• The current player starts to play video 1.
• The pre and next players load video data 0 and 2 respectively.
• At the same time, video data 3-7 is added to the pre-download queue.

Principle of Scheme 3: After turning the page

• The holder recycled by RecyclerView destroys the player.
• The holder in RecyclerView onBind creates a new next player.
• The current player starts to play video 2.
• The pre player seeks 0 and pauses, and the next player creates and loads the video 3 file.
• At the same time, the pre-downloading clears the unconsumed queue, and the video data 4-8 is added to the pre-downloading queue to start downloading (videos that have been cached here will not be downloaded repeatedly).

Specific optimization plan

Multi-player transformation

In order to solve the problems of somatosensory frustration and slow flow, the multi-player combined with the RecyclerView solution is used for transformation, and the steps are as follows:

1. Set the number of caches: Use the RecyclerView feature to configure setItemViewCacheSize to ensure that there are 3 holders in the memory (1 holder in the cache, 1 holder pre-created, and the current holder).
   mRecyclerView.setItemViewCacheSize(1); // Set the number of caches
2. Override the onBindViewHolder of the Adapter to create a player and pre-load the decoded video content, and the player controls to pause when the first frame is parsed. At this time, onSurfaceCreated has not yet been called back, and the screen has not been rendered to the screen.
3. Monitor onPageRelase to control the pause of the player whose screen is about to be removed, and seekTo (0), so that it can be played immediately when sliding back to the screen.
4. Monitor onPageSelected to control the player that is about to enter the screen to start playing. Note : Since the decoding has been completed during onBindViewHolder, you only need to enter the screen 1px, and will immediately trigger the drawing of the Surface (only executed once), that is, the contents of the window will display the first frame of the video.
5. Override the onViewRecycled of the Adapter. Since the current holder is about to move out of the screen, the holder outside the screen in the direction of the move out will be recycled. At this time, the player is recycled and destroyed.


Multi-player + RecyclerView schematic

Three players greatly improve the experience of immersive short videos, and mainly solve the following problems:

• In the process of sliding up and down, the picture that enters the screen is the first frame of the video, and there will be no visual frustration.
• Pre-create the player before the official play, load and decode it, saving the preparation work before playing the video. (Ps: The download process is also included here).

• Due to pre-loading and decoding, when entering the screen, the Surface is triggered to render instantly, which is visually imperceptible. Therefore, the cover image is no longer needed before the video is played, avoiding the splash screen problem caused by the inconsistency of the cover image and the first frame.

  Pre-download optimization

  I mentioned earlier that multiple players can realize the ability to turn pages in seconds, and the experience has been greatly improved. However, because the pre-created player needs to download the video file at the same time when it is loaded, the next player here is ready for the video. The time will increase to about 1s. If the user slides to the video before the player is loaded and decoded, a noticeable black screen will appear, resulting in a very poor experience.

Because the pre-loading time is too long, and it is impossible to predict whether the user will swipe quickly. Here need to be downloaded and fast swipe detection in advance.

Regarding pre-downloading, we must first know the internal playback process of the player. The local proxy here is implemented by the video caching mechanism. For details, refer to the next chapter.

Player internal process

Pre-download strategy

Here, in order to save the process of requesting network data, we download the first frame of the video before playing, and adopt the following strategy:

• File size: Download the first frame in advance by downloading a 1MB video file. (Ps: 1MB has been tested to include the first frame, and the file is relatively small).
• Advance amount: 5 downloads in advance (pageSize is 10).
• Concurrency: The download adopts synchronous queue download (to avoid bandwidth occupation caused by asynchronous download, and video freezes during normal playback).
• Fast sliding optimization: Fast sliding clears the download queue to avoid frequent triggering of downloads during fast sliding.

• Download timing: when loadMore, push the first 5 videos into the queue; when onPageSelected, skip the next 5 videos and push them into the queue (the next video is automatically downloaded by the preloaded player, and repeated downloads here will cause video screens).

  Fast skating definition

  When the user quickly turns the page (slides again before onPageSelected is called), onPageSelected will not trigger, and onPageRelease will trigger multiple times. If the difference between the release position and the current postion is judged in onPageRelease> 1, it means that the user quickly turns the page at least once. At this time, it is defined as the fast sliding state, and the pre-downloading and player pre-loading should be stopped.
When onPageSelected is called back, it means that the user does not continue to turn the page, and the fast sliding state is cancelled at this time. Start to perform pre-download and resume player pre-loading.

Pre-download flowchart

  Cache optimization

  The player currently used by Hema is Taobao's internal player. The player itself does not have file caching and pre-downloading functions. After the player is re-created, even the same video will not be cached and needs to be downloaded again. An open source caching framework "com.danikula:videocache" is introduced here.

  Scheme principle

  The address played by the player is delegated to the local cache service. The cache service is responsible for forwarding the data stream while saving the file, such as:
The video address is: https://****.mp4 .
The local agent address is: http://127.0.0.1:8888 (assuming the port number is assigned as 8888).
The address after the proxy is: local proxy address + video address (URL encoding).
That is: http://127.0.0.1:8888/https%3A%2F%2F****.mp4


  Follow-up optimization outlook

  There is still much to be done about the optimization of multimedia. In addition to the immersive second broadcast scene, we can also:

• the player’s general scenes in seconds, such as the card video on the homepage list. At present, the average first frame of the homepage takes 550ms, and the longer one is more than 1000s, which has obvious frustration. On the basis of the immersive solution, we can provide general pre-download capabilities, thereby reducing the player’s download rendering time.
• Resume and memory optimization. replay is another aspect to enhance the experience, and users can intuitively feel whether it is coherent or not.
• page single player hosting. most scenarios, only one player is playing on a page. This enables page player reuse by managing a unique player, thereby optimizing memory and experience.

In the next issue, we will continue to share the experience optimization practice of Hema iOS/Android short video replay.

"Video Cloud Technology" Your most noteworthy audio and video technology public account, pushes practical technical articles from the front line of Alibaba Cloud every week, and exchanges and exchanges with first-class engineers in the audio and video field. The official account backstage reply [Technology] You can join the Alibaba Cloud Video Cloud Product Technology Exchange Group, discuss audio and video technologies with industry leaders, and get more industry latest information.