Tencent developed a new trick, and PAG has been used for animation production since then

Beginning in 2020, the sudden outbreak of the epidemic has made online life develop rapidly, and short video platforms have become an important way for the majority of netizens to enjoy entertainment, learn skills, and share their lives.

According to the 48th "Statistical Report on Internet Development in China" released by CNNIC, as of June 2021, the number of online video users in my country reached 944 million, of which the number of short video users was 888 million, accounting for 87.8% of the total netizens. The daily usage time is 125 minutes.

Under the growing short video user base, in order to attract more traffic, animation designers and developers need to continuously and efficiently create various creative special effects to meet the diverse content needs of users.

Animation is a very important part of affecting user interaction. Now, animations on various platforms are rarely implemented using native code, so the development cost is too high and the cycle is too long. At present, the commonly used animation workflow solutions in the industry are Lottie and SVGA, both of which can export the animation produced by Adobe After Effects (AE for short) into a file, which can be loaded and rendered in the terminal APP, which to a certain extent improves the animation development online. efficient.

Based on the above background, I would like to recommend an excellent animation production component developed by Tencent: the open source animation rendering library PAG.

1. What is PAG

Official definition: PAG (Portable Animated Graphics) is a complete set of animation workflow. Provides export plug-ins from AE, to desktop preview tool PAGViewer, to cross-platform rendering SDKs at each end.

The following figure shows the PAG workflow. The process is similar to Lottie. After the designer uses AE to design the animation, he reads the AE project file through the PAGExporter plug-in, and selects one of vector export, BMP pre-synthesis, and hybrid export methods to export one according to specific needs. PAG binary file, the client decodes and renders the PAG binary file, each end shares a set of C++ implementation, and the platform end only does interface encapsulation.

2. Advantages of PAG

Compared with the animation components SVGA and Lottie on the market, PAG has the following characteristics:

• The animation file is small and the decoding speed is fast
• All AE effects can be achieved
• Complete supporting tools, support real-time preview effect
• Runtime preserve animation effects and edit text or content in real time

1. The animation file is small and the decoding speed is fast

The PAG scheme has placed the file format in the most important position from the beginning of the design, and the goal is to create a standard bearing format for AE animation content.

Compared with the JSON data structure adopted by the Lottie solution, PAG draws on the SWF open source file format that has been tested in the industry for decades, and adopts a more rigorous binary data structure. Naturally, it has the advantages of higher compression rate, faster decoding speed, and single-file delivery (no external pictures).

Also in on file size , PAG by utilizing the characteristics of the animation file itself, gained high compression ratio . Default by skipping a large number of stored bits used to store compact, the same type of animation content can be compared with the embodiment average reduction% 50 about the file size. In terms of performance, the average real-time rendering performance of PAG can reach Lottie's 1.5 to 2.5 times .

2. Full AE feature support

Lottie supports only vector export, but vector mode actively implements only a small subset of AE features. PAG
Not only supports more AE features in the vector export method, but also introduces the hybrid export function of video sequence frames combined with vectors.
Powerful, to achieve support for all AE features while maintaining the editability of the animation runtime.

3. Complete supporting tools, support real-time preview effect

Different from Lottie, SVGA, PAG draw on animation rendering is implemented in C ++ layer 2D graphics rendering library through self-development, platform-independent interface to end rendering, can achieve consistency across platforms rendering .

The desktop preview tool PAGViewer ensures that the rendering results are completely consistent with the mobile terminal, so that the designer can intuitively see the display effect on the mobile terminal without the need to go back and forth to confirm. At the same time, a performance detection panel is provided to help development engineers optimize according to the quantified performance indicators of the material.

3. Detailed explanation of PAG's technical capabilities

Then introduce the technical solutions corresponding to the advantages of PAG:

1. BMP pre-synthesis

In order to achieve full support for AE characteristics, PAG uses AE SDK screenshot of the exported file form, so AE can export any effect, but there are two significant disadvantages: 1, the export file is too large; 2, the picture can not be edited

file problem solution

In response to the problem that the file is too large after the screenshot, the PAG component expands the video format, compresses the original picture sequence frame to nearly 1% of the size, and then supports the transparent channel, as shown in the following figure, the left side is the RGAB video content, the right side is the video content. It is the grayscale image of the Alpha channel, and it is merged back into the RGBA image when it is finally rendered, so as to realize the support for the transparent channel. During the rendering process, since hardware-accelerated decoding is enabled, a YUV texture can be directly obtained. And in order to avoid copying textures back and forth between the CPU and GPU, a custom Shader script is used to use hardware acceleration to complete YUV conversion and alpha channel merging in one drawing process. Improved rendering performance by an average of 10%.

Image Editing Problem Solution

In view of the fact that BMP pre-synthesis cannot be edited, PAG extends the granularity of BMP pre-synthesis support from file to synthesis, and supports the mixed export of vector and BMP pre-synthesis, so as to support all AE features and maintain runtime editability.

2. Layer editing

In the era of smart templates, such as one-click movie production and King's Battle Report, the template is no longer a single PAG file, but a random combination of multiple PAG files, and the rules of the combination are controlled according to business needs. As a result, PAG introduces the editing architecture of layer rendering numbers, which not only supports text and placeholder image editing, but also supports layer-level editing.

As shown in the figure below, a file is a rendering tree, which supports arbitrary modification positions at the layer level and even adds and deletes layers. It also supports adding other PAG files to this rendering tree as subtrees. In terms of the combination of the time axis, PAG has the ability of time stretching, including multiple adaptive modes such as loop, speed change, and freeze frame. Each layer also provides the ability to adjust the starting time, and can freely set the relative position on the timeline.

3. Overall video rendering

The reason why Lottie's animation scheme cannot be applied to video synthesis is mainly because it relies on platform-related UI frameworks and has low development costs, but it also causes it to only be rendered to UI views and cannot be used in child threads.

In order to support off-screen rendering and sub-thread rendering, PAG is directly developed based on the C++ cross-platform architecture. It has been restored from the bottom-level animation interpolator to the upper-level timeline and layer rendering tree system. Although the development cost is high, all the Ends share the same set of code, which naturally guarantees cross-end rendering consistency. The most important thing is that it can directly render to off-screen textures, and perfectly supports sub-threaded animation rendering.

4. Server-side rendering

As mentioned earlier, the rendering of PAG is implemented based on the C++ layer, and the platform side only provides the encapsulation of the rendering environment and interfaces. In actual use, for cost reasons, most servers are still CPU servers, and GPU servers are mostly used in scenarios such as AI computing.

Some special effects in AE, such as Gaussian blur, corner positioning, etc., are implemented through OpenGL, and cannot be rendered using the CPU rendering mode of skia; except for the Linux side, other platforms can use GPU rendering for acceleration. CPU rendering mode requires a series of compatible processing at the code level. In order to take into account the rendering performance and usage cost, PAG provides the OpenGL rendering environment by simulating the GPU by the CPU, and adopts the Swiftshader rendering scheme through the performance comparison of the mainstream Mesa and Swiftshader schemes.

In terms of GPU rendering, the external only needs to provide the EGL environment to complete GPU rendering.

4. Summary

PAG provides a set of simplified and improved animation workflow, in the case of reducing file size, still supports all AE features, and retains the flexibility of animation running editable. With only one connection, designers can quickly get started with all efficient components, and no longer have to reduce the presentation effect due to R&D costs.

At present, the PAG solution has been widely used in dozens of products inside and outside Tencent, covering many national-level applications, such as WeChat, QQ, Tencent Video, QQ Music, Honor of Kings, QQ Zone, etc.

And Tencent PAG was officially open sourced on January 14th, and now it can be used in SDK, designers and developers can use it properly!
You can download the experience on the official website, with the link to the official website: https://pag.io

Addendum - Brief Introduction to Use

How to install PAG:

At present, PAG supports mac (macOS 10.9 and above) and windows operating systems. It is very simple to install and use. Taking the mac system as an example, you need to install PAGViewer first, and select graphical installation; then open PAGViewer, and PAGViewer will automatically detect whether it needs to be installed. /Update the AE export plugin, follow the prompts to install it. Also see usage link: https://pag.io/docs/install.html

Then you can install AE export plug :

How to export PAG files?

a. Export full vector pre-synthesized PAG file

Click to select the composition to be exported, then click the menu "File" -> "Export" -> "PAG File...", select the path to save to export. After the export is successful, double-click the exported PAG file to preview the animation directly.

b. Export the full BMP pre-synthesized PAG file

Modify the composition to be exported to the "_bmp" or "_BMP" , mark the total composition as "BMP pre-composition", and other operations are the same as the vector export mode. (Note: BMP pre-synthesis suffix can be changed, see "Plugin Options Configuration Panel)"

c. Export vector and BMP pre-synth mixed PAG file

The composition can be named without the "_bmp" or "_BMP" , and the partial composition it refers to can be named with the suffix of "_bmp" or "_BMP", and then export the PAG file according to the normal process.

In general, the installation and operation of PAG are very simple, and both designers and development engineers can quickly get started.