Fish-Lottie: How does pure Dart implement a high-performance animation framework?

Author: Cen Yu

background

​Lottie is an animation program open sourced by Airbnb across Android, iOS, Web, etc. It uses JSON to solve the development cost of complex animation implementation by developers.

As we all know, the Xianyu team is the technical team that chose the Flutter solution on the client side earlier, and the current Xianyu project also contains many Flutter interfaces. However, the official has not provided Lottie-Flutter solutions, and some third-party developers currently provide related implementation solutions, which are basically divided into two types:

• Perform data analysis and rendering on the Native side, and then use the bridge method to transmit the rendered data to the Flutter side for display;
• Direct data analysis in Flutter and use Flutter's drawing capabilities for rendering and display.

However, the current open source solutions have some problems. The former will have some problems in performance and display, such as displaying a flickering white screen. The latter has some functional defects in some ability support, for example, it does not support text animation. So this has always been a pain point for the Xianyu team and the entire Flutter developer community.

Project structure

After investigating the official open source lottie-android library, the Xianyu team found whether it is data analysis capabilities or graphics rendering capabilities. Flutter provides implementations comparable to Android. Therefore, referring to the lottie-android library, a pure Dart Package with complete functions and excellent performance is implemented to provide Lottie animation support on Flutter.

As shown in the figure above, the entire project is composed of basic modules, interface layer and control layer, and then supports vector graphics, fill and stroke capabilities, and the details can be seen in Lottie's support capabilities, which are also roughly the same as lottie-android.

Basic module

The basic module is the place to directly interact with the various capabilities provided by FlutterSDK. It is mainly divided into a data model module, an animation drawing module, a data analysis module and a tool module. First of all, for the entire framework, we can first get the JSON file containing the entire animation information, so we need to pass through our data analysis module to parse the data and information contained in the JSON file and pass it to the data model module and the animation drawing module After getting the objects in the data model module, call the drawing capabilities provided by Flutter to draw graphics, and the tool module is mainly responsible for obtaining screen information, string processing, log printing and other tool capabilities.

Interface layer

The interface layer is mainly responsible for JSON data input and animation drawing control and calling. JSON information will eventually generate a LottieComposition object after the data analysis module, which carries the entire JSON animation information. Then pass this object to LottieDrawable, and then LottieDrawable will pass the object to the animation drawing module, so that the animation drawing module can get the animation information, and then LottieDrawable then calls the animation drawing module to draw and refresh the animation.

Component layer

The component layer, here is mainly the custom component that we inherited from Flutter's Widget implementation, and it is also the interface that the framework exposes to developers. Developers only need to create a new LottieAnimationView and pass the path of the JSON file to it. It supports three forms of Asset, Url and File, and then put the LottieAnimationView like a normal Widget in FlutterUI to complete a simple Lottie animation playback. Of course, it will also expose the animation control interface and the layout interface of the control. You only need to pass in the AnimationController, width, height, alignment and other properties when creating a new LottieAnimationView to complete the further customization of the animation.

work process

the whole idea

When a designer uses AE to make an animation, this animation is actually composed of different layers. AE provides multiple layers for designers to choose, such as a solid color layer (usually used as a background), a shape layer (drawing various vectors) Graphics), text layer, picture layer, etc., each layer can be set to translate, rotate, zoom and other transformations. Each layer may contain multiple elements. For example, a shape layer may be composed of multiple basic vector graphics and pen path graphics to form a design pattern, and each element may also contain its own transformation, in addition to the basic transformation. , You can also set the color, shape and other transformations. The animation of the above layers and elements constitutes a complete animation.

As shown in the figure above, we created a new solid color layer in AE and filled it with blue, then created a new shape layer, and added a displacement animation to this shape layer (that is, the transformation of shape layer 1 Set two key frames for the position, and set the initial value and final value on the key frames), then add a rectangular path and a yellow fill to the shape layer, and then animate the size and roundness of the rectangle in the same way. But the size of the key frame is 0 seconds to 3 seconds, and the roundness key frame is 3 seconds to 5 seconds. So it completes the animation of a rectangle from left to right at the same time, first becoming larger and then turning into a circle. Then we exported the above animation to a JSON format file through the BodyMovin plug-in provided by Lottie. This JSON file contains all our drawing and keyframe information just now.

As shown in the figure above, after getting this JSON file, we first parsed and passed the various layer information and animation information produced by the designer in AE to a LottieComposition object through data analysis, and then LottieDrawable obtained the LottieComposition object and Call the underlying Canvas to draw graphics, and use AnimationBuilder to control the progress. When the progress changes, Drawable is notified to redraw, and the drawing module will get the property values ​​at the progress, and then complete the animation playback. .

Data loading and display

Our component layer provides three ways to obtain JSON files, namely asset (program built-in resources), url (network resources), and file (file resources). The flowchart of the entire data loading and display is roughly as follows, omitting the details of the underlying drawing:

Here we take the fromAsset method as an example. The other two loading methods are the same, and they are all handled by LottieCompositionFactory. Here we divide the loading methods into three types according to the different constructors, namely asset, file and url. Then call different loading methods in LottieCompositionFactory according to different types to load the corresponding built-in resources, network resources and file resources and parse the JSON file, and then the final product is a LottieComposition object, which is asynchronously loaded and parsed. After completion, LottieAnimationView will be notified to call. We pass the loaded LottieComposition object to our drawing class. LottieDrawable will create a layer group based on the content in the composition. The layer group contains layers such as shapes, text layers, etc., and those created by the designer when making animations in AE The layers have a one-to-one correspondence. Each layer has different drawing rules and methods, and then the Canvas obtained in LottieAnimationView is passed to LottieDrawable and the draw method is called. In this way, we can use the system canvas to draw our own animation content.

Animation drawing and playback

After the animation is loaded and displayed, we also need to make the screen move. We set the value of AnimationController to the progress of LottieDrawable through AnimationBuilder, and then trigger redraw so that our bottom layer can obtain the animation properties of the current progress through progress, so that the effect of animation can be realized. The timing diagram is roughly as follows:

We use Flutter's built-in AnimationController to control the animation in LottieAnimationView. The forward method allows the progress of the Animation to increase from scratch, which is also the beginning of our animation playback. We keep calling the setProgress function to set the progress of the animation to each layer, and finally reach the KeyframeAnimation layer to update the current progress. After the progress changes, we need to notify the upper layer to redraw the interface, and finally set an isDirty variable in LottieDrawable to true. In the setProgress function, after completing the progress setting, we obtain the isDirty variable of lottieDrawable. If this variable is true, it proves that the progress has been updated. At this time, we call the overridden method markNeedPaint(). At this time, the system will mark the current component as needed For updated components, Flutter will call our rewritten paint function to redraw the entire screen. Like the displayed process, we draw layer by layer, and at the bottom layer we will get the corresponding attribute value in KeyframeAnimation according to the current progress, and then the drawn picture will change. By continuously updating the progress in this way, and then re-obtaining the attributes corresponding to the current progress for redrawing, the playback effect of the animation can be realized.

Realize the difference

Component layer

Android side

For lottie-android, AnimationView and Drawable form the entire component layer. AnimationView is inherited from ImageView, LottieDrawable is inherited from Drawable. The whole work process is basically the same as that mentioned above. The developer writes LottieAnimationView in the xml file and sets the resource path of the JSON file. Then AnimationView will initiate data acquisition and analysis. After the analysis is complete, the Composition object is passed to LottieDrawable, and then the overridden draw method is called for animation display.

Then the entire animation playback, pause, progress and other controls are all done by the developer obtaining a reference to the AnimationView in the code and then calling various methods, but in fact the real animation control is controlled by the ValueAnimator in LottieDrawable. At the same time that LottieDrawable is initialized, ValueAnimator is also created, which will generate an interpolation of 0 to 1, and set the current animation progress according to different interpolations. The animation control methods such as pause and play in LottieAnimationView actually call the corresponding method of this ValueAnimator itself to control the animation.

Flutter side

For Flutter, it does not provide components like ImageView and Drawable for us to inherit and rewrite. We need to customize a Widget. There are generally three ways to customize components:

• native components

We obviously cannot use this method here, because we need to get the canvas provided by the system to draw.

• implement CustomPainter

In Flutter, a self-painting UI interface CustomPainter is provided. This interface provides a 2D canvas Canvas. Some basic drawing APIs are encapsulated inside Canvas. Developers can draw various custom graphics through Canvas. We can get the system canvas in the rewritten paint method, and then pass this canvas to our LottieDrawable to complete the animation drawing, and then when the property changes cause the screen to be refreshed, the shouldRepaint returns true. But this solution will have some problems that cannot be solved. We all know that the entire LottieAnimationView is embedded in FlutterUI as a Widget. We often need to customize the size of the animation playback area (ie LottieAnimationView), but when the developer does not set this width and height When the value is set or the size is set larger than the size of the parent layout, we also need to adapt and transform the size according to the constraints of the parent layout on the child layout. However, in the CustomPainter provided by Flutter, the corresponding interface is not exposed so that we can obtain the constraint property of the RenderObject corresponding to this Widget, and it is impossible to size according to the constraints of the parent layout when the developer does not set the width and height of the LottieAnimationView itself. Adaptation, so this implementation scheme was abandoned.

• custom RenderObject

We all know that the Widget in Flutter is just some lightweight style configuration information, and the class that really performs graphics rendering is RenderObject. So we can also override the paint method in the RenderObject class to get the system canvas for drawing. This solution is more complicated than the previous one. We need to define a RenderLottie class that inherits from RenderBox, and then rewrite the paint method to pass the system's canvas to LottieDrawable. Call the markNeedPaint method where refresh is needed to complete the interface. Repaint. Then for RenderObject, we can get the constraint property of the current component, that is, when the developer does not set the size of the LottieAnimationView or the set size exceeds the complex layout, we can also adapt to the size of the parent layout. Next, you need to define a component LeafRenderLottie that inherits from LeafRenderObjectWidget and override the createRenderObject method and return the RenderLottie object, and override the updateRenderObject method to update the progress of RenderLottie and other properties. This completes the implementation of a LottieWidget. So how do we control the playback of the animation? Our LottieAnimationView is embedded in FlutterUI as a Widget, and generally does not get its reference to call the method, then we pass in an AnimationController provided by Flutter, and then in LottieAnimationView The build method returns an AnimationBuilder and passes the progress value of the AnimationController to LeafRenderLottie. If the developer does not pass in the AnimationController, we provide a default controller for simple animation playback. The key code is as follows:

@override
  void paint(PaintingContext context, Offset offset) {
    if (_drawable == null) return;
    _drawable.draw(context.canvas, offset & size,
        fit: _fit, alignment: _alignment);
  }

//RenderLottie的paint方法

Text drawing

Android side

Canvas in the Android SDK provides the drawText method, which allows you to draw text directly on the canvas. The Android implementation scheme is as follows:

private void drawCharacter(String character, Paint paint, Canvas canvas) {
    if (paint.getColor() == Color.TRANSPARENT) {
      return;
    }
    if (paint.getStyle() == Paint.Style.STROKE && paint.getStrokeWidth() == 0) {
      return;
    }
    canvas.drawText(character, 0, character.length(), 0, 0, paint);
}

Flutter side

But there is no such method in Flutter's Canvas. After investigation, we found that Flutter provides a special TextPainter to draw text. The Flutter implementation scheme is as follows:

void _drawCharacter(
      String character, TextStyle textStyle, Paint paint, Canvas canvas) {
    if (paint.color.alpha == 0) {
      return;
    }
    if (paint.style == PaintingStyle.stroke && paint.strokeWidth == 0) {
      return;
    }

    if (paint.style == PaintingStyle.fill) {
      textStyle = textStyle.copyWith(foreground: paint);
    } else if (paint.style == PaintingStyle.stroke) {
      textStyle = textStyle.copyWith(background: paint);
    }
    var painter = TextPainter(
      text: TextSpan(text: character, style: textStyle),
      textDirection: _textDirection,
    );
    painter.layout();
    painter.paint(canvas, Offset(0, -textStyle.fontSize));
}

Bezier curve

Android side

As we mentioned in the background, the Bezier curve is one of the three elements that make up the animation. Our animations are often not played linearly, if you need to achieve the effect of fast and slow. We need to use the Bezier curve to map from the progress to the attribute value when obtaining the attribute value through the progress. PathInterpolator is provided in the Android SDK to achieve this. Our JSON file uses two control points to describe the Bezier curve. We pass the coordinates of these two control points to PathInterpolator, and then call the interpolator when the property value is obtained. The getInterpolation can get the mapped value. The following are the key methods to achieve:

interpolator = PathInterpolatorCompat.create(cp1.x, cp1.y, cp2.x, cp2.y);

public static Interpolator create(float controlX1, float controlY1,
            float controlX2, float controlY2) {
        if (Build.VERSION.SDK_INT >= 21) {
            return new PathInterpolator(controlX1, controlY1, controlX2, controlY2);
        }
        return new PathInterpolatorApi14(controlX1, controlY1, controlX2, controlY2);
}

public PathInterpolator(float controlX1, float controlY1, float controlX2, float 
                        controlY2) {
        initCubic(controlX1, controlY1, controlX2, controlY2);
}

private void initCubic(float x1, float y1, float x2, float y2) {
        Path path = new Path();
        path.moveTo(0, 0);
        path.cubicTo(x1, y1, x2, y2, 1f, 1f);
        initPath(path);
}

//Andorid内置贝塞尔曲线生成关键方法

Flutter side

And there is no such ready-made path interpolator in Flutter, we can only implement it according to the source code. After viewing the relevant Android source code, I found that we only need to pass the coordinates of the two control points in the JSON to the cubicTo method in the Flutter path to generate the Bezier curve, and then implement an input parameter as time t, and the result is The method of progress p after mapping is fine, and the specific implementation can be completed by referring to getInterpolation in PathInterpolator. The following are the key methods to achieve:

interpolator = PathInterpolator.cubic(cp1.dx, cp1.dy, cp2.dx, cp2.dy);

factory PathInterpolator.cubic(
      double controlX1, double controlY1, double controlX2, double controlY2) {
    return PathInterpolator(
        _initCubic(controlX1, controlY1, controlX2, controlY2));
}

static Path _initCubic(
      double controlX1, double controlY1, double controlX2, double controlY2) {
    final path = Path();
    path.moveTo(0.0, 0.0);
    path.cubicTo(controlX1, controlY1, controlX2, controlY2, 1.0, 1.0);
    return path;
}

自定义Flutter贝塞尔曲线生成关键方法

Effect comparison

We have currently implemented a closed-loop Demo project using fish-lottie. In it, we also selected the lottie json file in the lottie-android project for testing. We found that the release package is both in terms of fluency and animation restoration. Reached the level of the official sample App, below I will use some animations to compare and illustrate:

Among the above, the former is the animation played by fish-lottie on the flutter page, and the latter is the animation played by lottie-android on the native page. It is not difficult to see that fish-lottie can be comparable to lottie-android whether it is rendered or played. Degree.

Among the above, the former is a dynamic text animation using fish-lottie, and the latter is a dynamic text animation of lottie-android. It can be seen that fish-lottie can also provide not lost to lottie-android in terms of dynamic attributes and real-time text rendering. Effect. And because our text drawing implementation scheme is different from the original one, we can better expose the font style interface, so that developers can not only customize the text, but also can dynamically customize the style in real time. This is the current situation. Functions not provided by lottie-android.

Future prospects-from static to interactive

The current usage scenarios of Lottie are just a static playback of an animation. For example, an animation of a thumb will appear after a thumbs up, and a heart-shaped animation will appear after the collection. At most, the playback of some of the entire animations can be controlled by the progress. But in the process of implementing the entire framework, I found that lottie-android already has some interactive capabilities. The usage is as follows:

 val shirt = KeyPath("Shirt", "Group 5", "Fill 1")
 animationView.addValueCallback(shirt, LottieProperty.COLOR) { Colors.XXX } //需定制的颜色

The effect of the above code is shown in the following figure:

lottie-android implementation scheme

From the above code, we can see that in order to achieve dynamic attribute control, we need to pass in three parameters. The first parameter is similar to a locator, and we need to locate the vector graphics we want to control the attributes in the form of a path. Content, the second parameter is an attribute enumeration variable, which indicates the type of attribute we control, and the last parameter is a callback function that needs to return our dynamically changed target value.

Because the upper component layer is quite different from lottie-android, fish-lottie currently only completes the ability to support animation playback, and the interactive capabilities are under development.

fish-lottie realization ideas

Because of the differences in the dual-end implementation of the upper-level components and the UI construction characteristics, we generally do not get a reference to the Widget to call its methods in Flutter. So you can't directly use lottieAnimationView.addValueCallback() to control dynamic properties like lottie-android. We actually encountered the same problem when we realized the progress control of the animation. So our realization idea is actually the same as AnimationCtroller. We also implement a PropertiesController (property controller), passing a series of target graphics, target properties and callback functions that we need to modify to this controller, and then use this controller as A parameter of the LottieAnimationView constructor is passed to LottieDrawable, and then this attribute controller initiates the matching and callback function settings of the target graphics drawing class. The methods in the underlying drawing class and frame animation class are consistent with lottie-android. The basic idea is consistent with lottie-android, but LottieAnimationView no longer assumes the responsibility of property control, but is assumed by PropertiesController.

Landing direction

With the ability to interact, we no longer can only control the playback of animations. We can get the user's click and touch event to give feedback on the animation, in order to achieve some more complex interactive animation.

As shown in the figure above, the animation effect of this search box background is difficult to achieve if the developer directly develops it. And through lottie, we have a clearer idea, to make a flowing jelly background animation, two content animations, a dark night star and moon animation, and a day cloud animation. We can control the jelly background animation background in black and blue by clicking events. Switch between purple gradient colors, change its partial shape, and show and hide two content animations. When you click the first Pillow button, switch the jelly background animation color to a blue-purple gradient, and then display the cloud animation. When you click the second Baby button, switch the background color of the jelly background animation to black, and then display the star-moon animation. Then for the 3D effect of cloud animation, we can obtain the side offset angle of the mobile phone through the gyroscope sensor of the mobile phone device, and then change the position of each element of the cloud animation according to the angle. In this way, the complex interactive animation effects that were too expensive or even impossible to achieve before can be easily realized through lottie.

• "[AliFlutter] Flutter Fish Redux 2.0 Architecture Evolution Practice"

, 3 mobile dry goods & practice for you to think about every week!