Introduction to Computer Graphics·Rasterization

Author: front-end engineer

Hawthorn recent computer graphics into the pit, inside the heart to do a share, sharing is mainly based on Yan Qi makes the course . The following are the notes shared internally this time, , please use this slide to eat .

Computer graphics is widely used in games, movies, design, VR & AR, visualization, simulation, GUI, fonts and other fields, including rasterization, curves & meshes, ray tracing, animation simulation (Animation/simulation) and other content. Among them, rasterization can be said to be the most basic concept in computer graphics. In simple terms, rasterization is a process of mapping a three-dimensional scene into a plane figure, then decomposing the figure into polygons, breaking the polygons into pixels, and displaying them on the screen.

The mathematics involved in rasterization is mainly linear algebra, including vectors (sum, dot product, cross product) and matrices (product, transpose, identity matrix), etc. A large number of transformations involved in rasterization can be expressed as operations on vectors and matrices.
Linear transformations such as scaling, shearing, flipping, and rotation can be represented by matrix multiplication. After introducing homogeneous coordinates, translation can also be processed by matrix multiplication just like linear transformation. Three-dimensional complex rotation including pitch, yaw, and roll can be described by Rodriguez's rotation equation.

Basic transformation operations such as zooming, rotation, and translation occur in the modeling tranformation stage, which is similar to the scene finding and model placement in the process of taking pictures. The model transformation is followed by the view tranformation stage. In this stage, we determine the relative position of the camera and the object, similar to finding the angle and placing the camera in the process of taking pictures. Specifically, we use the position of the camera, the direction to look at, and the upward direction to represent the camera. Then, in order to simplify the calculation, move the camera and the object together, and move the camera to The origin of the coordinate system. The next stage is projection transformation (projection tranformation), which is similar to pressing the shutter in the photographing process. Projection is mainly divided into two kinds of orthogonal projection and perspective projection. Orthogonal projection is relatively simple, the light is parallel, and the relative position of each point remains unchanged. The scene is translated and zoomed to [-1, 1]^3 , and then the Z value is removed and pressed onto the plane. Here, scaling to a standard cube is to simplify the calculation. Perspective projection, near large and far small, is similar to the way the human eye sees objects. Using the three-point properties that the near plane will not change through extrusion, the far plane will not change through extrusion, and the center point will not change through extrusion, perspective projection can be achieved. These three stages constitute the view transformation. Finally, there is a viewport transformation, which transforms the scene that was previously scaled to a standard size according to the length and width of the screen and projected it on the screen.

Rasterized decomposition graphics are polygons. Among them, triangles are the most widely used, because the inside of the triangle is a plane, the inside and outside are clearly defined, and any point inside can be interpolated and gradual. When the triangle is output to the screen, each pixel on the screen is traversed, and the cross product of the vector is used to determine whether the pixel is within the triangle. Of course, the actual calculation will use the bounding box to simplify the calculation, and the pixels outside the bounding box do not need to perform the vector cross product operation. In order to deal with sampling defects such as sawtooth and moiré, you can first blur and then sample. Although the convolution operation on each pixel can achieve blur, the calculation is relatively large, so supersampling is generally used, and the sampling points are added inside each pixel and then averaged.

The Z value can be used to deal with the problem of occlusion and maintain a Z-Buffer to record the depth of the pixel. If the Z value of the pixel on the triangle surface is less than the value in the Z-Buffer, then update the value in the Z-Buffer to the current Z value , And update the color of the pixel to the color of the pixel on the current triangle.

The next thing to deal with is the shading problem, because the color of the object will be affected by the light. What's more complicated is that different materials and light interact differently. The most basic shading model is the Blinn-Phong reflection model, which comprehensively considers shading from the three dimensions of highlight, diffuse reflection, and ambient light. Diffuse reflection mainly considers the diffuse reflection coefficient of the material, the light intensity, the distance of the light source, the incident direction and the angle between the surface of the material. Different shading frequencies will produce different display effects, including flat shading, gouraud shading, and phong shading. The plane-by-plane normal uses the triangular face as the coloring unit. The calculation is fast, but the effect is poor. The normal vectors of all triangle faces sharing a vertex are added together to obtain the average value, and the normal vector of the vertex can be obtained. Based on this, each vertex can be colored, and then the color of each point in the triangle can be directly interpolated to calculate the color. This is gouraud shading. If you do not directly interpolate and calculate the color of each point in the triangle, but directly rasterize it to form a pixel, and then color each point accordingly, it is phong shading.

The rendering pipeline often mentioned in computer graphics can usually be divided into vertex processing, triangle processing, rasterization, fragment processing, and frame buffer processing. In fact, it is roughly the content mentioned above. Vertex processing is to perform view transformation on all vertex data. Triangle processing is to combine these vertices into triangle faces (triangulation), and then rasterization to determine whether the pixel is in the triangle, and then fragment processing. Each piece is colored, and finally all the information is integrated and output to the screen. Among them, in addition to the aforementioned coloring, the fragment processing can also be used for texture mapping, and texture maps can be used instead of colors.

Finally, I recommend two great video tutorials:

• 3Blue1Brown's linear algebra is the essence of , to understand linear algebra in a visual way (total duration: 2h40m).
• Yan Modern Computer Graphics , no nonsense, easy to understand, humorous (total duration: 28h+).