THREE.js rendering order

This article will describe the rendering sequence of THREE.js, where the rendering sequence refers to the objects in the front and back positions, and how to render the occlusion relationship between the objects.

The main content includes:

1. What is the default rendering order of opaque objects;
2. What is the default rendering order of transparent objects;
3. When opaque objects and transparent objects are rendered together, what is the default rendering order;
4. How to change the default rendering order of objects.

What is the default rendering order of opaque objects

First, we introduce the effect we want to achieve through a simple example:

A basic scene, two 4X4 squares are placed in the scene, the squares are parallel to the XOY plane, the red square is placed at (0, 0, 0), and the green square is placed at (2, 0, -2).

The camera uses THREE.PerspectiveCamera, this kind of camera will have the effect of near big and far small, the camera is placed at the position (0, 0, 6), at this time, the camera looks in the negative direction of the Z axis.

The spatial positions of the camera and the two planes are shown in Figure 1 and Figure 2 below:

figure 1

figure 2

Through the spatial relationship between the camera and the two planes, we know

1. The red plane is displayed in front of the green plane;
2. The red plane will block part of the green plane;

The final rendering effect is shown in Figure 3 below:

image 3

At this point, you can think for yourself first, if you let you realize it, how would you realize it?

I thought about it myself. First of all, we know the distance between these two objects and the camera. We can sort the objects according to how far they are from the camera, and then render the objects in order from far to near. That is, the green plane is rendered first, and then the red plane is rendered. There is no problem with the simple example above. Does it apply to other complex scenarios?

For example, are the two planes at the same distance from the camera? As shown in Figure 4 below:

Figure 4

According to our previous assumption, for the above scenario, what we rendered is either green completely on top, or red completely on top, as shown in Figure 5 or Figure 6 below:

Figure 5

Image 6

But the actual rendering result should be as shown in Figure 7 below:

Figure 7

Therefore, the above-mentioned scheme of rendering in order from far to near can only satisfy some usage scenarios. So, how does THREE.js implement it?

THREE.js is mainly an open source library that uses WebGL to build 3D scenes, and is an easy-to-use package of the capabilities provided by WebGL. So before discussing the rendering order of objects in THREE.js, we must first look at how WebGL renders objects in different positions. WebGL is based on OpenGL, so the question becomes how OpenGL renders objects in different positions.

OpenGL uses depth testing (depth testing) to ensure that the object is rendered with the correct occlusion relationship. The depth test uses a depth buffer. Similar to the color buffer, except that the color buffer stores the color value of each pixel, and the depth buffer stores the depth of the current pixel color value of the color buffer. The depth buffer and the color buffer have the same width and height.

Then, in the rendering process, for each pixel, the data we store includes:

1. The color value of the current pixel (obtained through the color buffer);
2. The depth of the object fragment corresponding to the current pixel in space (obtained through the depth buffer).

Below, we use the example in Figure 4 above to illustrate how OpenGL achieves the correct occlusion relationship of objects through depth testing.

Suppose the object is drawn on a green plane first, and the object is drawn pixel by pixel when it is drawn. At this time, the information we know includes the color value and depth information of the pixel. For each pixel projected on the green plane, we write the color value at the position of the pixel in the color buffer, and write the depth information at the position of the pixel in the depth buffer.

Then, we start to draw the red object. When drawing each pixel of the red object, we know the color value and depth information D2 of that pixel. Then, we obtain the depth value D1 corresponding to the drawn pixel in the depth buffer according to the coordinates of the pixel, and then compare the values of D1 and D2. There are three situations:

1. If D2 is less than D1, then the current pixel of the object is in front, that is, the pixel should take the color value of the current pixel of the object. At this time, update the color value of the current pixel in the color buffer to red, and update the depth of the current pixel in the depth buffer to D2;
2. If D2 is greater than D1, then the current pixel of the object is behind, that is, the pixel will not be displayed, so the color buffer and depth buffer values of the current pixel do not need to be changed;
3. If D2 is equal to D1, the behavior is consistent with that D2 is less than D1.

In summary, we have a function to judge whether the pixel is rendered. The input of this function is

1. The depth value of the pixel currently waiting to be rendered;
2. The depth value of the current pixel in the depth buffer.

The output is a boolean indicating whether the current pixel is rendered with the new color value.

The default value of this function in THREE.js is LessEqualDepth, which is the three cases of comparing D2 and D1 above. All values of this function can refer to THREE.js official website Depth Mode .

So, still the above example:

1. For each pixel on the left side of the red object, the depth value D2 is less than the depth value D1 in the depth buffer, so the color buffer is updated to red.
2. For each pixel on the right of the red object, the depth value D2 is greater than the depth value D1 in the depth buffer, so the original color is maintained.

The end result is that the left half is red and the right half is green.

We analyzed the situation of drawing the green plane first, and then the red plane. You can try to analyze the situation where the red plane is drawn first, and then the green plane is rendered.

The final result is that the occlusion relationship of the rendering result basically has nothing to do with the order of drawing.

What is the default rendering order of transparent objects

The rendering sequence of opaque objects was described above. So, if the objects in the scene are all transparent objects, how are they rendered?

Take the previous example as an example. At this time, we set both planes to be semi-transparent. As shown in Figure 8 below:

Figure 8

If the previous logic is still used, the color of each pixel is either unchanged or the color of the new object is used. After adding the logic of the depth test, the rendered effect is shown in Figure 9:

Picture 9

In real life, we should be able to see the objects behind the transparent object through a transparent object. Obviously, Figure 9 does not achieve such an effect. So, what is the problem?

When we were in the depth test earlier, when writing the color value into the color buffer, either write the color value of the current object or discard the color value of the current object. For transparent objects, the final displayed color value is not the color of a single object, but a blend of the colors of multiple visible objects.

Then, in the previous steps, when we judge that the current object is in front, we can change from simple and rude use of the color value directly to a blend according to the transparency of the current object's color value and the color value in the color buffer. Then update the color buffer with the mixed color value.

THREE.js provides a variety of blend methods, the default is NormalBlending. The calculation formula of NormalBlending is as follows:
color(RGB) = (sourceColor * sourceAlpha) + (destinationColor * (1 - sourceAlpha))
color(A) = (sourceAlpha * 1) + (destinationAlpha * (1 - sourceAlpha))

Add the upper mixing logic, and the final effect is shown in Figure 10 below. This effect is also an effect that meets our psychological expectations:

Picture 10

When rendering opaque objects, we found that the final effect has nothing to do with the order in which the objects are drawn. So, what about transparent objects? Let's experiment:

Render the red plane first, then the green plane

The rendering effect is shown in Figure 11 below:

Picture 11

Render the green plane first, then the red plane

The rendering effect is shown in Figure 12 below:

Picture 12

It can be seen that for the rendering of transparent objects, the drawing sequence will affect the occlusion relationship of the rendering result. So what causes this?

Let's analyze the situation where the red plane is rendered first, and then the green plane is rendered. First, after the red plane is drawn, the data related to the red plane is stored in the color buffer and the depth buffer. At this time, for each green pixel that is blocked by the red plane, the depth test is performed first, and the depth test fails, so the pixel is directly discarded.

So the problem here is that when the depth test is successful, we can choose whether to blend and the function of the blend; but when the depth test fails, the pixel is directly discarded, instead of providing you with a function that allows you to customize The color value of this pixel.

In summary, the final rendering result of transparent objects is related to the drawing order of the objects. When the transparent object is drawn in the order from far to near, the result will greater extent on ; when the transparent object is drawn in the order from near to far, the result will basically not meet our expectations. Unless you do it intentionally.

The reason why it is said that greater than instead of must be because there are some special circumstances. From the above conclusions, we can also know that the rendering result is related to the drawing order. We can sort the objects before drawing them. But it should be noted that we sort using a coordinate information that represents the overall position of the object, rather than sorting according to each pixel of the object. So for two intersecting objects, no matter what the drawing order is, the final rendering result is incorrect. As shown in Figures 13 and 14 below:

Figure 13

Picture 14

In the above situation, I have not found a solution yet.

When opaque objects and transparent objects are rendered together, what is the default rendering order?

If there are both opaque objects and transparent objects in our scene, then, imagine on the basis of the previous, how should we achieve it?

Firstly,

1. For opaque objects, the drawing order is not required;
2. For transparent objects, they need to be drawn in order from far to near;

In summary, is it possible to sort all objects in the order from far to near, and then draw them in this order?

I thought about it myself and found no problem, but found that THREE.js is not implemented according to this logic. Let's first talk about the default rendering order of THREE.js:

1. First, divide the objects in the scene into two arrays according to whether they are transparent or not;
2. For the array where the opaque objects are located, sort them in the order of near and far;
3. For the array where the transparent objects are located, sort them in the order of farthest to nearest;
4. Draw the array where the opaque object is located;
5. Draw the array where the transparent object is located.

I thought about it, the reason why THREE.js is implemented in this way should be considered from the performance side.

First of all, for opaque objects, although the drawing order has no effect on the rendering result, it still affects the rendering performance. For example, for two parallel planes AB, plane A is closer than plane B. At this time:

1. Draw A first, then B:

   1. Perform depth test on all pixels of plane A, the test is successful, and the color and depth buffer are rewritten;
   2. Perform depth test on all pixels of plane B. For the part that is not blocked by plane A, rewrite the color and depth buffer; for the blocked part, the depth test fails and returns directly;

2. Draw B first, then A:

   1. Perform a depth test on all pixels of plane B, and the test is successful, and the color and depth buffer are rewritten;
   2. Perform a depth test on all pixels of plane A, the test is successful, and the color and depth buffers are rewritten.

Through the above comparison, it can be found that when the opaque objects are drawn in the order from near to far, the operation of rewriting the color and depth buffer of the occluded part can be omitted, so the performance is improved to a certain extent.

Secondly, when we finish drawing opaque objects in order from near to far and start to draw transparent objects, the depth test of transparent objects behind the opaque objects fails, so the following color and depth buffer update operations will not be performed, so It can also improve the performance of rendering transparent objects to a certain extent.

Therefore, if there is no distinction, when opaque objects and transparent objects are drawn together, then all objects must be drawn in the order from far to near. Then, the depth test will be successful in most cases, that is, there will be more color and depth buffer update operations, which affects the performance to a certain extent.

How to change the default rendering effect of an object

Most of the effects we mentioned above are the effects of the default rendering order, but if you want to change the default rendering effects, is there any way?

The answer is yes.

Control depth test

Earlier we talked about depth testing. The three steps of depth testing are all controllable:

1. Whether to conduct an in-depth test;
2. Depth test function behavior;
3. Whether to update the depth buffer.

These three steps are controlled by the following three properties of Material:

1. depthTest : whether to perform a depth test;
2. depthFunc : the behavior of the depth test function;
3. depthWrite : Whether to update the depth buffer.

In addition, when you need to turn on the depth test, you need to turn on the depth parameter when initializing WebGLRenderer, which will create a depth buffer. The default value of this parameter is true, which means that you don't need to pay attention to this attribute under normal circumstances. Of course, if your needs clearly do not require in-depth testing, and the performance requirements are relatively high, you can manually turn off this value to reduce certain storage costs.

Control the drawing order

We mentioned earlier that THREE.js draws opaque objects and transparent objects in the order of near-to-far and from far-to-near respectively. So is this sorting implemented for us by THREE.js? Or do we need to control the drawing order ourselves?

THREE.js sorting is enabled by default, this is by WebGLRenderer of sortObjects implemented properties. If automatic sorting is not turned on, the drawing order is the order in which the objects are added (note that transparent objects and non-transparent objects are still rendered separately at this time).

The default sort order of opaque objects and transparent objects can refer to the painterSortStable and methods of the source code.

So, how do we intervene in the above sorting process? There are mainly two ways as follows:

1. For the above two methods, we can notice that there is a renderOrder attribute, which is what we need. For specific instructions, please refer to the renderOrder document ;
2. Completely customize the sorting logic through setOpaqueSort and setTransparentSort

Custom blend function

When we talked about rendering transparent objects earlier, we mentioned that the default blending function of transparent objects is NormalBlending. The behavior of this blending function is also optional. For specific supported behaviors, please refer to Material.blending .

Summarize

This article mainly describes the rendering sequence of opaque objects and transparent objects in THREE.js, mainly involving the following contents of THREE.js:

1. Material

   1. depthWrite(default is true)
   2. depthTest(default is true)
   3. depthFunc(default is LessEqualDepth)
   4. A series of attributes related to blending and blending

2. Object3D

   1. renderOrder(default is 0)

3. WebGLRenderer

   1. depth
   2. sortObjects(default is true)
   3. setOpaqueSort
   4. setTransparentSort

The above point of view is based on the results of current research on THREE.js, and there may be cognitive errors. If so, please leave a comment.