Introduction to short video is an important means of carrying content and is the focus of attracting users. The content and experience of short video are directly related to whether users are willing to stay for a long time. Therefore, the optimization of the experience is particularly important. In the previous article, we shared the optimization of iOS short video seconds, and in this article, let’s talk about the optimization of Android.
Author｜Shaoyang
Review｜Taiyi
The Hema immersive short video playback page before optimization has a significant gap with mainstream short video apps in terms of body feel and fluency. The main problems are the splash screen when playing the cover and the slow streaming speed. Therefore, the goal of optimization is to solve the existing shortcomings of Hema immersive short video and align with the immersive short video experience of mainstream apps, such as Douyin and Handtao. The specific indicators are:
- Meet hard indicators: playback success rate, first frame duration, second opening rate.
- Satisfy the user's body fluency.
(In order to reflect the real experience of users watching short videos, Hema has added a second-broadcast somatosensory indicator: the time from the user to the video to the first frame of the video.)
Optimization effect comparison
First of all, let's take a look at the effect comparison with other apps before and after optimization:
Environment
- Phone: Pixel 4
- OS：Android 10
- Player: Taobao player
problem analysis
◆ homepage splash screen initially added a cover image display to ensure that it is not a blank page when entering the screen, which is hidden during playback. It can be seen from the somatosensory index that even before optimization, the somatosensory playback time is very short, only about 200ms (not including the sliding process). During the sliding process, it takes about 600ms to display the cover page until the video is officially played, and then quickly display the playback screen. At this time, the user still has a strong screen flicker and frustration, and the experience is extremely poor.
solution: displays the first frame of the video during the sliding process, and no longer displays the cover, so there will be no frustration during playback . The optimization here needs to be optimized together with the slow flow problem.
◆ slow streaming speed (slow playback feel) server: The slow file streaming speed is generally caused by poor network streaming on the Available H.265 and CDN accelerated optimization
Client: client player needs to undergo download -> Load decoding + -> Rendering three steps, three steps and a linear actuator. Therefore, it is necessary to go through about 1s of preparation before playing the screen in the window. Here you can consider executing download -> load + decode .
Optimization plan selection
In the early stage of optimization, we considered three optimization schemes.
Solution 1: Dual players + pre-downloading Advantages: small memory footprint, simple idea. Disadvantages: The optimization efforts are limited, and it is impossible to take into account both upslip and downslide at the same time.
Solution 2: Customize the three-player management + pre-download Advantages: At the same time, it takes into account the up and down pages, and the experience is close to Douyin. Disadvantages: Player management and recycling are complicated to implement and easily confused; high memory usage.
Solution 3: three players (implemented based on RecyclerView's caching mechanism) + pre-download advantages: At the same time, it takes into account the up and down pages, the experience is close to Douyin, and the caching mechanism is hosted by RecyclerView. Disadvantages: high memory usage, frequent creation and destruction of players.
Finally, because of cost-effective factors, chose the third option .
Principle three of the scheme:
- The current player starts to play video 1.
- The pre and next players load video data 0 and 2 respectively.
- At the same time, video data 3-7 is added to the pre-download queue.
plan three principles:
- The holder recycled by RecyclerView destroys the player.
- The holder in RecyclerView onBind creates a new next player.
- The current player starts to play video 2.
- The pre player seeks 0 and pauses, and the next player creates and loads the video 3 file.
- At the same time, the pre-downloading clears the unconsumed queue, and the video data 4-8 is added to the pre-downloading queue to start downloading (videos that have been cached here will not be downloaded repeatedly).
Specific optimization plan
Multi-player transformation
In order to solve the problems of somatosensory frustration and slow flow, the multi-player combined with the RecyclerView solution is used for transformation, and the steps are as follows:
1. Set the number of caches: Use the RecyclerView feature to configure setItemViewCacheSize to ensure that there are 3 holders in the memory (1 holder for cache, 1 holder pre-created, current holder).
mRecyclerView.setItemViewCacheSize(1); // 设置缓存数量2. Rewrite the onBindViewHolder of the Adapter to create a player and preload the decoded video content, and the player controls to pause when the first frame is parsed. At this time, onSurfaceCreated has not yet been called back, and the screen has not been rendered to the screen.
3. Monitor onPageRelase to control the pause of the player whose screen is about to be removed, and seekTo (0), so that it can be played immediately when sliding back to the screen.
4. Monitor onPageSelected to control the player that is about to enter the screen to start playing. Note : Since the decoding has been completed during onBindViewHolder, here you only need to enter the screen 1px, will immediately trigger the drawing of the Surface (only executed once), that is, the content entering the window will display the first frame of the video.
5. Override the onViewRecycled of the Adapter. Since the current holder is about to move out of the screen, the holder outside the screen in the moving direction will be recycled. At this time, the player is recycled and destroyed.
Multi-player + RecyclerView schematic
Three players greatly improve the experience of immersive short videos, and mainly solve the following problems:
- In the process of sliding up and down, the picture that enters the screen is the first frame of the video, and there will be no visual frustration.
- Pre-create the player before the official play, load and decode it, saving the preparation work before playing the video. (Ps: The download process is also included here).
- Due to pre-loading and decoding, when entering the screen, the Surface is triggered to render instantly, which is visually imperceptible. Therefore, the cover image is no longer needed before the video is played, avoiding the splash screen problem caused by the inconsistency of the cover image and the first frame.
pre-download optimization
I mentioned earlier that multiple players can realize the ability to turn pages in seconds, and the experience has been greatly improved. However, because the pre-created player needs to download the video file at the same time when it is loaded, the next player here is ready for the video. The time will increase to about 1s. If the user slides to the video before the player is loaded and decoded, a noticeable black screen will appear, resulting in a very poor experience.
Because the pre-loaded takes too long to , and it is impossible to predict will quickly slide . Here you need to download and fast slide detection in advance.
Regarding pre-downloading, we must first know the internal playback process of the player. The local agent here is realized by the video caching mechanism, please refer to the next chapter for details.
player internal process
◆ Pre-download strategy
Here, in order to save requesting network data , we download the first frame of the video before playing, and adopt the following strategy:
- File size: Download the first frame in advance by downloading a 1MB video file. (Ps: 1MB has been tested to include the first frame, and the file is relatively small).
- Advance amount: 5 downloads in advance (pageSize is 10).
- Concurrency: The download adopts synchronous queue download (to avoid bandwidth occupation caused by asynchronous download, and video freezes during normal playback).
- Fast sliding optimization: Fast sliding clears the download queue to avoid frequent triggering of downloads during fast sliding.
- Download timing: when loadMore, push the first 5 videos into the queue; when onPageSelected, skip the next 5 videos and push them into the queue (the next video is automatically downloaded by the preloaded player, and repeated downloads here will cause video screens).
◆ Fast skating definition
When the user quickly turns the page (slides again before onPageSelected is called), onPageSelected will not trigger, and onPageRelease will trigger multiple times. If the difference between the release position and the current postion is judged in onPageRelease> 1, it means that the user quickly turns the page at least once. At this time, it is defined as the fast sliding state, and the pre-download and player pre-load should be stopped.
When onPageSelected is called back, it means that the user does not continue to turn the page, and the fast sliding state is cancelled at this time. Start to perform pre-download and resume player pre-loading.
pre-download flowchart
Cache optimization
The player currently used by Hema is Taobao's internal player. The player itself does not have file caching and pre-download functions. After the player is re-created, even the same video will not be cached and needs to be downloaded again. An open source caching framework "com.danikula:videocache" is introduced here.
◆ Principle of the scheme
The address played by the player is delegated to the local cache service. The cache service is responsible for forwarding the data stream while saving the file, such as:
The video address is: https://****.mp4 .
The local agent address is: http://127.0.0.1:8888 (assuming the port number is assigned as 8888).
The address after the proxy is: local proxy address + video address (URL encoding).
That is: http://127.0.0.1:8888/https%3A%2F%2F****.mp4
Follow-up optimization outlook
There is still much to be done about the optimization of multimedia. In addition to the immersive second broadcast scene, we can also:
1. Optimize the player’s general scenes in seconds, such as the card video on the homepage list. At present, the average first frame of the homepage takes 550ms, and the longer one is more than 1000s, which has obvious frustration. On the basis of the immersive solution, we can provide general pre-download capabilities, thereby reducing the player’s download and rendering time.
2. Resume and memory optimization. replay is another aspect to enhance the experience, users can intuitively feel whether it is coherent or not.
3. Hosting of page single player. most scenarios, only one player is playing on a page. This enables page player reuse by managing a unique player, thereby optimizing memory and experience.
In the next issue, we will continue to share the experience optimization practice of Hema iOS/Android short video replay.
Welcome to the "Video Cloud Technology" public account.
"Video Cloud Technology" Your most noteworthy audio and video technology public account, pushes practical technical articles from the front line of Alibaba Cloud every week, and exchanges and exchanges with first-class engineers in the audio and video field. The official account backstage reply [Technology] You can join the Alibaba Cloud Video Cloud Product Technology Exchange Group, discuss audio and video technologies with industry leaders, and get more industry latest information.
Copyright Notice: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users. The copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用。你还可以使用@来通知其他用户。