A brief introduction to the texture of graphics / WebGL coding

background

This article is included in "Data Visualization and Graphics" column

First of all, I made a simple comparison between Canvas 2D and WebGL in the previous article, and we can understand that the API of WebGL is relatively difficult to understand. So I think the follow-up coding is more to use WebGL to achieve our effects (hey, just play~) Of course, the knowledge points are still from simple to complex. . . If I first introduce visual physics, lighting/global illumination, anti-aliasing, delayed rendering, real-time rendering... Maybe I lost the original intention of creating the column; of course, if you want to discuss in depth, you can communicate privately.

The above has implemented a simple 2d graphic if necessary, please refer to the above

Outline of this article

1. What is a texture? The difference between 2d and 3d texture mapping?
2. The Texturing Pipeline
3. coding (Simple use of textures in WebGL)

1. What is a texture? The difference between 2d and 3d texture mapping?

In computer graphics, texturing is a technique that uses images, functions, or other data sources to change the appearance of an object's surface.

2d texture

2D texture is a simple bitmap image used to provide the color value of the surface points for the 3D model

3d texture

A 3D texture can be considered to be composed of many 2D textures, which are used to describe pictures of three-dimensional spatial data.

2. The Texturing Pipeline

The rendering pipeline may be a black box for daily use, but understanding this will greatly improve your programming... (It doesn't matter if you don't understand it, it doesn't matter that the introduction in this section is very shallow, and most of them are copied from other places. Haha)
Simply put, texturing is an efficient technique for "modeling" the surface properties of objects. The pixels in the image texture are usually called texels, which are different from the pixels on the screen.

Please see the picture below---the detailed process of applying texture mapping to a single texture

1. The projection function (projector function) is to convert the three-dimensional points in the space into texture coordinates, that is, to obtain the position of the surface and project it into the parameter space. For example, there are functions related to spherical, cylindrical, and planar projection
2. The role of the mapping function (The Corresponder Function) is to convert parameter-space coordinates into texture space locations.

• first step. By projection equations (Projector function) applied to a point in space, to thereby obtain a group called parameter space values (Parameter Space-values) value on the texture.
• The second step. Before using these new values to access the texture, one or more mapping functions (corresponder functions) can be used to convert the parameter-space values to texture space.
• third step. Use these texture-space locations corresponding value (obtain value) from the texture. For example, you can use the array index of the image texture to retrieve the pixel value.
• the fourth step. Then use the value transform function (value transform function) to transform the value of the search result, and finally use the new obtained to change the surface properties, such as material or shading normal, etc.

Normal/normal vector related knowledge needs to be supplemented by

coding (Simple use of texture map 2d in WebGL)

Simply use WebGL to implement a texture (map), and animate (rotate) through a matrix. The inserted gif is a bit stuck.

1. Shader writing adds texture coordinates and modifies vertex coordinates (using matrix/rotation)

//vertex shader
attribute vec4 a_position;
attribute vec2 a_texcoord;  

uniform mat4 u_matrix;

varying vec2 v_texcoord;

void main() {
   gl_Position = u_matrix * a_position;
   // 纹理坐标传递
   v_texcoord = a_texcoord;
}
// fragment shader
precision mediump float;

varying vec2 v_texcoord;

uniform sampler2D u_texture;

void main() {
   gl_FragColor = texture2D(u_texture, v_texcoord);
}

2. Initialize the context (rendering context) and shader program

var canvas = document.querySelector("#canvas");
var gl = canvas.getContext("webgl");
if (!gl) {
    return;
}

3. Initially set up the GLSL program and add a new buffer (texture)

// 设置GLSL程序
  var program = webglUtils.createProgramFromScripts(gl, ["vertex-shader", "fragment-shader"]);
  // 获取顶点坐标需要绑定的地方
  var positionLocation = gl.getAttribLocation(program, "a_position");
  var texcoordLocation = gl.getAttribLocation(program, "a_texcoord");
  // 获取 uniforms
  var matrixLocation = gl.getUniformLocation(program, "u_matrix");
  var textureLocation = gl.getUniformLocation(program, "u_texture");
 
 
  // 创建缓冲区
  var positionBuffer = gl.createBuffer();
  gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
  var positions = [
    -1, -1,
    -1,  1,
     1, -1,
     1, -1,
    -1,  1,
     1,  1,
  ];
  gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(positions), gl.STATIC_DRAW);
  // 为纹理坐标创建缓冲区
  var texcoordBuffer = gl.createBuffer();
  gl.bindBuffer(gl.ARRAY_BUFFER, texcoordBuffer);
  var texcoords = [
    0, 0,
    0, 1,
    1, 0,
    1, 0,
    0, 1,
    1, 1,
  ];
  gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(texcoords), gl.STATIC_DRAW);

4. Create and load texture

 //解决图片跨域问题
 function requestCORSIfNotSameOrigin(img, url) {
    if ((new URL(url, window.location.href)).origin !== window.location.origin) {
      img.crossOrigin = "";
    }
  }
 // 创建一个纹理 { width: w, height: h, texture: tex }
 // 初始化1x1 px像素 图片加载完更新
 function loadImageAndCreateTextureInfo(url) {
    var tex = gl.createTexture();
    gl.bindTexture(gl.TEXTURE_2D, tex);
    // Fill the texture with a 1x1 blue pixel.
    gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 1, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE,
                  new Uint8Array([0, 0, 255, 255]));

    // let's assume all images are not a power of 2
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);

    var textureInfo = {
      width: 1,   
      height: 1,
      texture: tex,
    };
    var img = new Image();
    img.addEventListener('load', function() {
      textureInfo.width = img.width;
      textureInfo.height = img.height;

      gl.bindTexture(gl.TEXTURE_2D, textureInfo.texture);
      // 调用 texImage2D() 把已经加载的图片图形数据写到纹理
      gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, img);
      render()
    });
    requestCORSIfNotSameOrigin(img, url);
    img.src = url;

    return textureInfo;
}
// 加载纹理
var texInfo = loadImageAndCreateTextureInfo('https://webglfundamentals.org/webgl/resources/leaves.jpg');

5. Draw related settings texture coordinates and assign matrix coordinates. Use a coloring program to draw.

function render(time) {
    time *= 0.001; // time 加速度 旋转图片
    // 重新设置canvas大小
    webglUtils.resizeCanvasToDisplaySize(gl.canvas);

    // 裁剪像素区
    gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);
    gl.clear(gl.COLOR_BUFFER_BIT);
    gl.bindTexture(gl.TEXTURE_2D, texInfo.texture);

    // 使用着色器程序
    gl.useProgram(program);

    // 设置参数，让我们可以绘制任何尺寸的图像
    gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
    gl.enableVertexAttribArray(positionLocation);
    gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);
    gl.bindBuffer(gl.ARRAY_BUFFER, texcoordBuffer);
    gl.enableVertexAttribArray(texcoordLocation);
    gl.vertexAttribPointer(texcoordLocation, 2, gl.FLOAT, false, 0, 0);

    var aspect = gl.canvas.clientWidth / gl.canvas.clientHeight;
    var matrix = m4.scaling(1, aspect, 1);  //可是试试将aspect 改为0.1看看效果 - -
    matrix = m4.zRotate(matrix, time);
    matrix = m4.scale(matrix, 0.5, 0.5, 1);
    // 设置矩阵
    gl.uniformMatrix4fv(matrixLocation, false, matrix);

    // 从第0个unit开始获取纹理
    gl.uniform1i(textureLocation, 0);

    // 绘制 (2 triangles, 6 vertices)
    gl.drawArrays(gl.TRIANGLES, 0, 6);

    requestAnimationFrame(render);
}

1. webgl-utils.js webgl related function packaging tool library
2. m4.js Matrix Related Mathematical Function Library
   For the complete code example , please click the git repository to view the code example

What you may need to know about 2D rendering are

1. Texture cache
2. Texture compression
3. 2D/2D texture optimization
4. Rendering optimization...

At last

Finally, I strongly hope that everyone learns relevant theoretical knowledge; theory may be used in very few places in daily life, but it can determine how far you go. (Some people ask what to do, practice hard), the writing speed of this column I speed up (1-2 articles a week) other columns (1 article) basic knowledge related to computer graphics will be brought again. Then, I mainly write the data visualization direction in the follow-up.