2D animation spine rendering principle analysis and source code interpretation

Preface: What is spine?

In order to give beginners a more intuitive and preliminary understanding, the author provides a simple schematic diagram as follows:

In actual development, the designer provides the animation material exported by the corresponding spine editor, and the developer chooses the corresponding spine runtime library to consume the material and render it on the screen. This is what spine does. Compared with gif and css Frame animation, apng have more powerful flexibility.

A simple comparison is as follows:

Type\Item	colour	size	compatibility	flexibility	cost
GIF	256 colors	small	excellent	difference	low
CSS frame animation	True color	Larger	excellent	general	low
APNG	True color	small	Compatible with some browsers	difference	low
spine2D animation	True color	Large (animation material + runtime library introduction)	excellent	Better	high

GIF generally cannot meet the design requirements due to its own color limitations. Therefore, most of the current page animations are processed by CSS frame animation and APNG, small area animations use apng, and large area animations can consider CSS frame animation or JS animation processing. However, because the business itself is complex and the appearance is changeable, the cm show has to adopt the spine2D animation solution.

Notes:

1) JSON file/binary file: store skeleton information, see below.

2) Material picture: similar to Sprite, it can also be a single material, and one or more pictures can be exported.

3) Atlas file corresponding to the material picture: record the location information characteristics of the material picture on the Sprite map, etc. One atlas file can correspond to multiple material pictures.

1. Basic concepts

1. Skeleton: refers to a collection of data, including all bones, slots, accessories and other information that make up the skeleton.

2. Bones: Taking the official diagram as an example, a character itself is composed of bones with multiple joints. Except for the root bone, each bone has a corresponding parent bone, and the relationship between bones and bones is finally constructed into a tree-like structure.

3. Slot slot: There may be multiple slot slots under a bone bone, and an accessory instance can be placed under each slot slot. The existence of the slot itself has two important meanings, one is to control the rendering order flexibly, and the other is to group similar accessories. A slot can have multiple accessories, but only one can be seen at a time. For a simple chestnut, the slot where the pistol is located in the picture is the "weapon" slot, and this slot can place different weapon accessories, such as "pistol" accessories or "chopper" accessories.

4. Attachment: The currently rendered attachment instance in the slot, that is, the actual material that is rendered on the screen. (It may be necessary to rotate, offset, scale or even mesh the material).

5. Skin: Skin can be regarded as a collection of attachments, or it can be regarded as a mapping query table of attachments. A character can have multiple sets of skins. By switching skins to query different attachment mapping tables, it can be disguised. Realize the character's full body dressing.

Other related concepts:

key frame:

In the editor, the animation is done with the help of key frames, and the transition animation from start to end is processed by spine tweening.

weight and grid:

Weights are used to bind mesh vertices to one or more bones. When the bones are transformed, the vertices will also be transformed. The weight enables the mesh to automatically deform as the bones are manipulated, so that the original complex mesh deformation animation becomes as simple as the bone animation.

Attachment type:
1. Regional attachments: ordinary picture display attachments.
2. Point attachment: A point and rotation in space. Compared with bones, the position and rotation can be changed for different skin settings. For example, different guns shoot from different positions.
3. Mesh attachment: Supports setting polygons in the picture, and then you can manipulate the vertices of the polygon to bend and deform the picture in an effective way.
4. Bounding box attachment: the polygon attached to the bone will be deformed when the bone changes. It can be used for collision detection, creation of a physical body, etc.
5. Trimming attachment: The trimming function allows you to define a polygonal area. Similar to the bounding box attachment, it will shield other slots in the drawing sequence.
6. Path attachment: used to set the path.

Among them, the area attachments and grid attachments are commonly used in business.

three constraints:
1. IK constraint: the scene where the end point of the sub-bones is fixed by inverse dynamics.
2. Transformation constraint: Transformation constraint refers to copying the world rotation, movement and scaling of bones to multiple bones.
3. Path constraint: Use the path to adjust the bone transformation. The bone can follow the path or adjust the rotation to point to the path.

Two, spine architecture and core class interpretation

The overall spine architecture is layered as follows:

The core spine classes are as follows:

Reading the official class diagram provided above will give you a clearer understanding and understanding of the overall architecture design of spine.

1. Loading module : It is for the processing of resource loading. After the skeleton information of a spine image is exported, it will generally be exported in the form of json or binary file. Because the json form of plain text file is too large, the official provides binary file export In the form of the binary file, supplemented by the code of the runtime library. Secondly, the atlasAttachmentLoader in the Loading module will be responsible for parsing the atlas file. Since the atlas file itself is in the form of a string and contains the location information of the material in the Sprite image, it needs to establish an "association relationship" with the material after the analysis. For example: Eyes-close material is rotated at the x and y position of picture1.png at the angle of z, and the constructed mapping relationship will be used for consumption when the attachment is instantiated.

2. Spine Texture Atlas module : A material image maps an atlasPage, and a certain region in a material image maps an atlasRegion, and the detailed drawing information of the region has been essentially completed in the previous module.

3. Rending module: is rendered by the rendering layer traversing the slot. It is not detailed here. The rendering layer is not part of the spine core library. On-screen rendering can be rendered by canvas, webGL or other third-party rendering libraries, such as pixijs.

4. SetupPoseData module: or called the SkeletonData module. The data source is input from here for processing, but it is not the final data. It can be understood that a layer of preprocessing is performed on the data, and the bone data will be processed as boneData. Slot data is processed as slotData. Of course, some data does not need to be processed again. Here, the corresponding attachment instance will be constructed based on the atlasRegion generated earlier and stored in the skin. The skin is essentially an attachment mapping table.

Secondly, the data object itself is different from the instance object.

Data objects are stateless and can be shared among any number of skeleton instances. The name of the data object class with corresponding instance data ends with "Data", and the data object without corresponding instance data has no suffix, such as attachments, skins, and animations.

Instance objects have many attributes that are the same as data objects. The attributes in the data object represent the assembly posture and usually do not change. The same property in the instance object represents the current pose of the instance when the animation is played. Each instance object maintains a reference to its data object, which is used to reset the instance object back to the assembly pose.

For example, SkeletonData is a data object, and Skeleton is an instance object.

5. Instance Data module : Or called the Skeleton module, the Skeleton instance itself is the real and direct data source of the on-screen rendering of the rendering layer. The rendering layer will read the slot information on the Skeleton instance and render the corresponding accessories. Here Many data objects have been processed into corresponding instance objects. For example, boneData has been processed as a Bone instance, and slotData has been processed as a Slot instance. Secondly, as shown in the figure, there are two key methods in the Skeleton instance. updateWorldTransform and setToSetUpPose.

updateWorldTransform is to update the world transformation, which essentially triggers the calculation of the bone position. Since the position of the bone may be rotated and the corresponding child bones will also be affected, it is necessary to update the world transformation to recalculate the latest coordinate positions of all bones.

setToSetUpPose is to update the instance to the current initial state. It is usually called when it is initialized or when the character state is reset, and it will switch the character's bone dress up to the initial initial state.

6. Animation module : The animation module is separated separately, which not only makes it easier to maintain and update the state information of the instance, but the overall architecture logic is also simple and clear. The animation state instance triggers the update of the skeleton instance, and then the skeleton instance calls updateWorldTransform to update The world changed, and then re-rendered on the screen.

An animation instance is composed of multiple timelines. These timeline instances come from different variants of the Timeline class, and are fundamentally inherited from the underlying TimeLine class. Since an animation process may involve multiple changes, different animations need to be divided The difference is a single timeline that handles rotation, a single timeline that handles scaling, and so on. Although different types of animations will be separated into different timelines, eventually a certain time node will be activated and triggered, and all the timelines "effects" are all at the same time.

Three, spine source code interpretation

1. Slot: Store the current posture of the slot. The slot is {@link Skeleton#drawOrder} to organize attachments and provide a location for storing attachment status. The state cannot be stored in the attachment itself, because the attachment is stateless and can be shared across multiple skeletons.
(The deform attribute is processing information for mesh attachments.)
(Set the initial action through setToSetUpPose)
(You can getAttachment and setAttachment directly in the slot instance)

2. SlotData: The data format of the data used in the slot instance, including index, slot name, attachment name, boneData, etc.

3. BoneData: The data format used in the bone instance, including index (the bone also has index), bone name, parent bone data, bone local conversion data, and world conversion mode.

4. Bone: The key method updateWorldTransformWith updates the world coordinates of the bone. Contains some other methods, conversion of world coordinates and local coordinates, rotation conversion, etc.

5. SkeletonData: The data format corresponding to the Skeleton instance. Contains bones, slots, skins, events, animations, and various constraints. provided
Because it is a data object, only some methods of findbone and findslot are provided.

6. Skeleton: Create new Bone and Slot based on data. The bone has an index to establish an association relationship in order. Call setToSetupPose to set bone and slot to the initial position. It will traverse and call the corresponding methods of bone and slot. updateWorldTransform calls bone's updateWorldTransform to update the bone position. Provides some get and set methods for Bone, slot, and attachment.
There is an update method to update the time.

7. SkeletonBinary: used to read binary skeleton files.

8. SkeletonBounds: Collect each visible BoundingBoxAttachment and calculate the world vertices of its polygon. Mainly used for collision or hit detection. Called by the rendering layer.

9, SkeletonCilpping: mainly for the processing of ClippingAttachment, called by the rendering layer.

10. SkeletonJson: used to parse and process the skeleton json exported by spine. You need to pass in an attachmentLoader so that it can construct a corresponding attachment instance to process bone, slots, ik, skins, animation and other data.
Process bone and slot to construct corresponding instance data.
Processing skins generates corresponding attachment instances with the help of loader.
Process animations to generate different timeline instance objects.
Finally, the corresponding SkeletonData instance is constructed.

11. Skin: All attachments under a set of skins are under the skin instance, providing methods for operating skin and attachments. The method of operation here is equivalent to a dictionary, not to change the character's appearance.

12. Attachment directory: The processing methods of various attachments are inherited from the Attachment base class. The corresponding attachment class method is actually called by the corresponding AttachmentLoader.
AtlasAttachmentLoader implements the corresponding method.

13. Texture: Defines the Texture abstract class, and defines some abstract methods that need to be implemented.

14. TextureAtlas: parse and process atlas text, realize TextureAtlasReader to read line by line, texture
Get the corresponding texture with the help of the textureLoader callback passed in from outside.
Each small piece of material corresponds to a TextureAtlasPage, and the corresponding TextureAtlasRegion is constructed after the material information is read and parsed.

15. AnimationStateData: Stores the duration of the blending (cross-fade) to be applied when the AnimationState animation changes.

16. AnimationState: Call the animation over time, the animation is queued to be played, and multiple animations are allowed to be superimposed.
Store animations in multiple tracks, distinguish the timelines of different animations, and process logic for event events.

17. Animation: Various timeline classes are implemented. Animation is responsible for calling the apply method to trigger the update, and its apply method will call the apply method of each timeline to update. The realization of the timeline class finds the corresponding key frame and decides how to render.

18. AssetManager: Static resource management, including pulling text resources, pulling binary resources, and loading textures. Call TextureAtlas to process atlas text, etc.

Four, rendering library code interpretation

canvas：
1. AssetManager: Nothing to do, just use AssetManager in core
2. canvasTexture: inherit Texture.
3. SkeletonRender: Pass in the skeleton data, render the canvas,
drawImage will traverse the slots in drawOrder, render the region attachments one by one, and use the ctx.drawImage API to crop and render the image.
drawTriangles will calculate the vertices and render the green lines in debug mode.

threejs：
1. ThreeJsTexture: A layer of wrapper is made for the texture of threesjs itself, and the filter is processed.
2. MeshBatcher: MeshBatcher inherits from the Mesh of Threejs itself. Call SkeletonMeshMaterial to get the material.
3. SkeletonMesh: SkeletonMeshMaterial inherits from ShaderMaterial, which contains shader code, vertex shader and fragment shader.
SkeletonMesh inherits from the Object3D class.
The core rendering function updateGeometry. After the skeleton updates the world, the rendering function is called to traverse drawOrder.
RegionAttachment and MeshAttachment will be rendered. The rendering uses MeshBatcher, and the texture is passed into batchMaterial as the material.

webgl：
1. GLTexture: Get the canvas and render the corresponding image onto the canvas.
2. Camera: set the camera position
3. WebGL: Defines ManagedWebGLRenderingContext, which is actually to get the context of webgl.
4. Input: Monitor events for elements, mouse and touch events.
5. Shader: self-implemented shaders, fragments and vertices.
6. SkeletonRenderer: Responsible for the on-screen rendering of the skeleton. The rendering function needs to use PolygonBatcher for on-screen rendering. Similarly, only RegionAttachment and MeshAttachment will be rendered.
7. PolygonBatcher: Bind the shader here, set the blending mode, bind a Mesh instance object, Mesh is a separately encapsulated mesh class, and finally call the rendering method exposed by Mesh.
8. Mesh: A separately encapsulated Mesh class that allows you to set indices and vertices, and render on the screen with the help of context's drawElements and drawArrays methods.
9. SceneRenderer: The top call class, instantiates batcher, webgl context, shader, instantiates SkeletonRenderer, and exposes different rendering methods, including drawSkeleton, drawSkeletonDebug, drawTexture, drawRegion, etc.

Five, the overall flow chart of spine rendering

According to the previous introduction, we have an understanding of the basic concepts, and understand the overall architecture design of spine, introduce the core modules, and interpret the spine core library source code and the key logic of the rendering layer source code to sort out the spine rendering The overall flow chart is as follows:

Seeing this, I believe you have a general understanding of the overall architecture design of spine and the rendering process. Next, after clarifying the underlying internal processing flow, our next step is to implement the changeover and changeover API. Realization, please listen to the next breakdown!

Thanks for watching~~