Three.js Sprite source code analysis

This article will explain the Sprite of Three.js (r105), which mainly includes the following aspects:

1. Simple introduction and use
2. Geometry of Sprites
3. Always towards the camera principle analysis
4. Analysis of the principle of constant size

Simple introduction and use

In the project, I mainly use Sprite to create some labels in the 3D scene. Here is a simple example to understand the basic usage of Sprite:

 const map = new THREE.TextureLoader().load("sprite.png")
const material = new THREE.SpriteMaterial({ map })
const sprite = new THREE.Sprite(material)
scene.add(sprite)

The effect is as follows (the canvas size is 300px * 400px, the gray background area is the canvas area, the same below):

Sprite has several features:

is a plane

Sprite is a plane, that is, Sprite's Geometry describes a plane rectangle. I will talk about it when I explain the source code below.

always facing the camera

We know that objects in a 3D scene are composed of triangles, and each triangle has a normal. The normal direction and the camera line of sight direction can have any relationship. The characteristic of Sprite is that the normal direction of the plane rectangle and the direction of the camera line of sight are always parallel and opposite.

The final rendering effect is that the drawn Sprite is always a rectangle without deformation .

For example, to rotate a normal plane 45 degrees along the X axis:

 const geometry = new THREE.PlaneGeometry(1, 1)
const planeMaterial = new THREE.MeshBasicMaterial({ color: 0x00ff00 })
const plane = new THREE.Mesh(geometry, planeMaterial)
plane.rotation.x = Math.PI / 4
scene.add(plane)

Results as shown below:

And do the same for Sprite (for comparison, change the texture to a solid color):

 const material = new THREE.SpriteMaterial({ color: 0x00ff00 })
const sprite = new THREE.Sprite(material)
sprite.rotation.x = Math.PI / 4
scene.add(sprite)

Results as shown below:

Can be set to cancel the effect of perspective camera near big and far small

Perspective Camera (PerspectiveCamera) simulates the effect of human eyes seeing the world: near big and far small.

By default, Sprite is also close to big and far small, but you can cancel this effect through the sizeAttenuation property of SpriteMaterial . The implementation principle of sizeAttenuation will be explained in detail later.

Geometry of Sprites

First look at the source code of the Sprite constructor ( Sprite.js ):

 var geometry; // 注释1：全局geometry

function Sprite( material ) {

    Object3D.call( this );

    this.type = 'Sprite';

    if ( geometry === undefined ) { // 注释1：全局geometry

        geometry = new BufferGeometry(); // 注释1：全局geometry

        var float32Array = new Float32Array( [ // 注释2：顶点信息和贴图信息，一共四个顶点
            - 0.5, - 0.5, 0, 0, 0,
            0.5, - 0.5, 0, 1, 0,
            0.5, 0.5, 0, 1, 1,
            - 0.5, 0.5, 0, 0, 1
        ] );

        var interleavedBuffer = new InterleavedBuffer( float32Array, 5 ); // 注释2：每个顶点信息包括5个数据

        geometry.setIndex( [ 0, 1, 2,    0, 2, 3 ] ); // 注释2：两个三角形
        geometry.addAttribute( 'position', new InterleavedBufferAttribute( interleavedBuffer, 3, 0, false ) ); // 注释2：顶点信息，取前三项
        geometry.addAttribute( 'uv', new InterleavedBufferAttribute( interleavedBuffer, 2, 3, false ) ); // 注释2：贴图信息，取后两项

    }

    this.geometry = geometry; // 注释1：全局geometry
    this.material = ( material !== undefined ) ? material : new SpriteMaterial();

    this.center = new Vector2( 0.5, 0.5 ); // 注释3：center默认是(0.5, 0.5)

}

From the above code we see two pieces of information:

1. Note 1: All Sprites share a Geometry;

2. Note 2:

   1. The length of each vertex information is 5, the first three items are the x, y, and z values of the vertex information, and the last two items are the texture information, which are stored in an array;
   2. A total of four vertices and two triangles are defined. The coordinates of the four vertices are A(-0.5, -0.5, 0) , B(0.5, -0.5, 0) , C(0.5, 0.5, 0) f4db8385278260c403fe70654c5317e7--- , ---6961a89e6d7d6e026e150be127c4bc40---f , ---c074165---eb8e18b074165 D(-0.5, 0.5, 0) The two triangles are T1(0, 1, 2) and T2(0, 2, 3) , which is T1(A, B, C) and T2(A, C, D) . The coordinates of the center point of the rectangle formed by these two triangles O are (0, 0, 0) . These two triangles form a square 1 X 1 . As shown in the following figure (the Z axis is all 0, which is not shown here):
      

Always towards the camera principle analysis

For a point in a 3D scene, the final position is generally calculated as follows:

 gl_Position = projectionMatrix * viewMatrix * modelMatrix * vec4( position, 1.0 );

Among them, position is the coordinate in the 3D scene, and this coordinate needs to go through

1. Matrix transformation of the object's own coordinate system (displacement, rotation, scaling, etc.) (modelMatrix)
2. Matrix transformation of camera coordinate system (viewMatrix)
3. Projection Matrix Transformation (projectionMatrix)

That is, the last used coordinates are obtained by a series of inherent transformations of the coordinates in the 3D scene. Among them, the matrix transformation of the camera coordinate system is related to the camera, that is, the information of the camera will affect the final coordinates.

However, the sprite is always facing the camera. We can speculate that the calculation of the Sprite position is definitely not the inherent transformation above. Let's take a look at how Sprite is implemented.

The logic of this block is implemented in the shader ( sprite_vert.glsl.js ):

 void main() {
    // ...
    vec4 mvPosition = modelViewMatrix * vec4( 0.0, 0.0, 0.0, 1.0 ); // 注释1：应用模型和相机矩阵在点O上

    // 注释6：缩放相关

    vec2 alignedPosition = ( position.xy - ( center - vec2( 0.5 ) ) ) * scale;
    // 注释2：center默认值是vec2(0.5)，scale是模型的缩放，简单情况下是1，所以，此处可以简化为:
    // vec2 alignedPosition = position.xy;

    vec2 rotatedPosition;
    rotatedPosition.x = cos( rotation ) * alignedPosition.x - sin( rotation ) * alignedPosition.y;
    rotatedPosition.y = sin( rotation ) * alignedPosition.x + cos( rotation ) * alignedPosition.y;
    // 注释3：应用旋转，没有旋转的情况下，rotatedPosition = alignedPosition
    // 其实就是把rotation等于0带入上述计算过程

    mvPosition.xy += rotatedPosition; // 注释4：在点O的基础上，重新计算每个顶点的坐标，Z分量不变，保证相机视线和Sprite是垂直的

    gl_Position = projectionMatrix * mvPosition; // 注释5：应用投影矩阵
    // ...
}

The calculation process of vertex coordinates is as follows:

1. Note 1: The coordinates of the calculation point O in the camera coordinate system;
2. Note 2-4: Taking O as the center, on the plane Plane1 perpendicular to the camera line of sight, directly obtain the coordinates of each vertex in the camera coordinate system;
3. Note 5: The coordinates obtained above are directly in the camera coordinate system, so there is no need to apply modelMatrix and viewMatrix, just apply projectionMatrix directly;

The actual position of ABCD in space and the position A'B'C'D' of the actual drawn ABCD are shown in the following figure:

Analysis of the principle of constant size

As we mentioned earlier, you can cancel the effect of perspective camera by setting the SpriteMaterial sizeAttenuation property of ---234bc65faaaf3702400b0bb3e43db115---. The implementation logic of this block is still implemented in the shader ( sprite_vert.glsl.js ):

 void main() {

    // ...

    vec4 mvPosition = modelViewMatrix * vec4( 0.0, 0.0, 0.0, 1.0 );

    vec2 scale; // 注释1：根据模型矩阵计算缩放
    scale.x = length( vec3( modelMatrix[ 0 ].x, modelMatrix[ 0 ].y, modelMatrix[ 0 ].z ) );
    scale.y = length( vec3( modelMatrix[ 1 ].x, modelMatrix[ 1 ].y, modelMatrix[ 1 ].z ) );

    #ifndef USE_SIZEATTENUATION

        bool isPerspective = ( projectionMatrix[ 2 ][ 3 ] == - 1.0 ); // 注释2：判断是否是透视相机

        if ( isPerspective ) scale *= - mvPosition.z; // 注释2：根据相机距离应用不同的缩放值

    #endif

    vec2 alignedPosition = ( position.xy - ( center - vec2( 0.5 ) ) ) * scale;
    // 注释2：顶点信息计算考虑缩放因子，此处，同样不考虑center的影响，简化后如下：
    // vec2 alignedPosition = position.xy * scale;

    // ... 注释3：同上，计算顶点位置过程

    #include <logdepthbuf_vertex>
    #include <clipping_planes_vertex>
    #include <fog_vertex>

}

The perspective camera has the effect of near big and far small. If you want to eliminate this effect, you can set different zoom ratios for objects at different camera depths. Obviously, this scaling is related to the depth of the camera. Sprite is also implemented like this:

1. Note 1: Calculate the scaling applied by the model itself, both horizontally and vertically. When not set, the scaling in both directions is 1;
2. Note 2: Associate the zoom ratio with the camera distance;
3. Note 3: When calculating the position of A'B'C'D', add the effect of scaling.

Next, let's look at the key code scale *= - mvPosition.z; why is it reasonable?

First, the relationship between the actual rendering size of the object and the camera is introduced. Here, we only consider the simplest case: what is the actual rendering size of a vertical line segment on a plane perpendicular to the camera's line of sight L ?

The calculation process is shown in the following figure:

In the vertical direction, the actual rendered size is:

 PX = L / (2 * Z * tan(fov / 2)) * canvasHeight

Wherein, L the actual size of the object, Z is the object distances from the camera, fov is in radians, canvasHeight is canvas high.

Obviously, the actual displayed size is related to Z. The larger the Z, the smaller the value of PX, and the smaller the Z, the larger the value of PX. Then, to eliminate the effect of Z, we can multiply L by a Z, which is L' = L * Z :

 PX = L' / (2 * Z * tan(fov / 2)) * canvasHeight
PX = (L * Z) / (2 * Z * tan(fov / 2)) * canvasHeight
PX = L / (2 * tan(fov / 2)) * canvasHeight

When the size of the object is fixed, the viewing angle of the camera is fixed, and the canvas is fixed, the actual displayed size PX is a fixed value, which achieves the effect that the size of the Sprite remains unchanged.

This is also the role of the above scale *= - mvPosition.z; . mvPosition.z is the Z b5edf96d7db6c11bc9945e0f8baed048--- in our formula above. The reason why there is a negative sign is because in the camera coordinate system, the direction the camera looks at is the negative direction of the Z axis, so the Z value of the object that appears in the camera's line of sight is negative, so adding a negative sign becomes Positive number.

So, how to set the display size of Sprite, for example, let the display height of Sprite be 100px ?

In fact, from the above formula we can get:

 PX = L / (2 * tan(fov / 2)) * canvasHeight
L = PX * (2 * tan(fov / 2)) / canvasHeight

Let's take fov is 90度 as an example, because at this time tan(PI / 2 / 2) is exactly 1 is more intuitive to calculate and look like. ,at this time:

 L = PX * (2 * tan(fov / 2)) / canvasHeight
L = PX * 2 / canvasHeight

In the Geometry part of the Sprite, we know that the Geometry is a rectangle of 1 X 1 . So L is how much, we can add L times the zoom to the object.

For example, when the camera angle is 90 degrees, the canvas size is 300px * 400px, and if you want the Sprite to display a height of 100px, set the scale to 100 * 2 / 400 = 0.5 :

 const material = new THREE.SpriteMaterial({
  color: 0xff0000, // 使用纯色的材质举例，纯色的容易判断边界，可以通过截图的方式验证实际渲染的像素大小是否正确
  sizeAttenuation: false // 关闭大小跟随相机距离变化的特性
})
const sprite = new THREE.Sprite(material)
sprite.scale.x = 0.5 // 注释1：X轴方向也设置为0.5
sprite.scale.y = 0.5
scene.add(sprite)

The screenshot of the effect is as follows:

In the above code comment 1 part, we also use the same scaling ratio as the Y-axis scaling, and the actual pixel size of the X-axis displayed at the end is also 100px . What if we want to display different pixel sizes in the X-axis direction? In fact, it is the same as calculating the vertical direction.

From the above figure, it can be found that the calculation method of the pixel size of the X axis is the same as that of the Y axis. The main reason is that the annotations ① and ② in the above picture all apply a camera's aspect ratio, so they are offset. That's why sprite.scale.x = 0.5 renders the X-axis pixel size is also 100px .

For example, in the above example, if you want the X-axis to display 75px , you can set scale.x = 75 * 2 / 400 = 0.375 , and the effect is as shown below:

Summarize

This article introduces the simple use of Sprite and the implementation principles of some features. I hope everyone can gain something!

If there are any mistakes, please leave a message for discussion.