3D performance optimization | talk about glTF file compression

introduction

Recently, I need to use a 3D model for T-level interaction. I believe that everyone, like me, will have some questions when starting to work:

• 1. How to choose the export format of the 3D model
• 2. How to optimize the model file
• 3. How about compatibility in large-traffic projects?

Let us carry out detailed exploration, research and precipitation through this article.

1. What is a glTF file

glTF full name of Graphics Language Transmission Format , which is a standard file format for 3D scenes and models.

The core of glTF is a JSON file that describes the entire content of the 3D scene. It consists of the description of the scene structure itself, which is provided by the hierarchy of nodes that define the scene graph.

The 3D objects appearing in the scene are defined using meshes connected to nodes. Materials define the appearance of the object. Animations describe how 3D objects transform into 3D objects over time, and Skins define how the geometry of the object is deformed based on the skeleton pose. Cameras describes the view configuration of the renderer.

In addition, it also includes links with binary data and image files, as shown in the figure below.

2. .gltf and .glb

It can be seen from the blender file export:

There are two extended forms of glTF files, .gltf (JSON / ASCII) or .glb (binary). The .gltf file may be self-contained, or it may reference external binary and texture resources, while the .glb file is completely self-contained (but using external tools you can save its buffer/texture as an embedded or separate file, which will be Mentioned).

2.1 Reasons for the .glb file

glTF provides two delivery options that can also be used together:

• glTF JSON points to external binary data (geometry, keyframes, skins) and images.
• glTF JSON embeds base64-encoded binary data and uses the data URI to inline the image.

For these resources, glTF requires a separate request or additional space due to base64 encoding. Base64 encoding requires additional processing to decode and increase file size (encoding resources increase by about 33%). Although gzip reduces the increase in file size, decompression and decoding still increase a lot of load time.

To solve this problem, a container format Binary glTF was introduced. In binary glTF, glTF assets (JSON, .bin and images) can be stored in a binary blob, which is file .

2.2 File comparison

2.2.1 The same glTF file, .glb format is smaller than .gltf

• Self-contained:

• Referencing external binary and texture resources:

2.2.2 Preview of .gltf file:

• Self-contained:

• Refer to external binary and texture resources:

2.2.3 glb file preview:

• Self-contained:

• Refer to external binary and texture resources:

As can be seen from the figure, when the glTF file is not self-contained, the image file will be requested together with the glTF file.

Then, we can use this feature to achieve some performance optimizations, let us continue.

Three, glTF file split

Mentioned above, glTF file can be split into .gltf / .glb binary file + + texture image, then we can split it up, and a separate texture image compression , to performance optimization.

You can use gltf pipeLine , which has the following functions:

• Conversion between glTF and glb
• Save buffer/texture as embedded or separate file
• Convert glTF 1.0 model to glTF 2.0 (using KHR_techniques_webgl and KHR_blend )
• Use Draco for mesh compression

Here, we are going to use the function "Save buffer/texture as embedded or separate file".

Let's take a look at the split files

Let’s recap, this is how the .glb file introduces external separate textures and binary files

So, just put the split files, into the same path , and then import them as before.

• Compression method

gltf-pipeline -i male.glb -o male-processed.glb -s

• Usage (in Three.js)
  Just use it in general, it's no different from the one that doesn't split

import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const loader = new GLTFLoader()
loader.load(MODEL_FILE_PATH, (gltf) => {
 // ....
})

• Performance comparison

Four, glTF file compression

As described above, glTF files include .gltf/.glb files, .bin files and texture resources. The main plugins related to glTF2.0 are as follows:

So let's take some of them to analyze.

4.1 Mesh compression

4.1.1 KHR_draco_mesh_compression

The most common of grid compressed mode, using open source Draco algorithms for compressing and decompressing 3D grid and a point cloud, and may change the order and number of mesh vertices. Compression makes the file much smaller, but requires additional decoding time on the client device.

• Compression method

You can use gltf-pipeline gltf file optimization tool for compression

gltf-pipeline -i male.glb -o male-processed.glb -d

• Usage (in Three.js)

import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'
import { DRACOLoader } from 'three/examples/jsm/loaders/DRACOLoader'

const loader = new GLTFLoader()

// 创建解码器实例
const dracoLoader = new DRACOLoader()
// 设置解压库文件路径
dracoLoader.setDecoderPath(DECODER_PATH)
// 加载解码器实例
loader.setDRACOLoader(dracoLoader)

loader.load(MODEL_FILE_PATH, (gltf) => {
 // ....
})

• Performance analysis and comparison

The size of the original file glb 3.2M, after draco compressed 1.8M, about the original document 56 is% .

As can be seen from the above code, creating a decoder instance requires the introduction of additional libraries for decoding. setDecoderPath will automatically request the wasm file for decryption. And these two wasm files also increase the request time and the number of requests, so adding these two files, the real compression ratio is about 62.5% .

So, if a project needs to load multiple glTF files, you can create a DRACOLoader instance and reuse it. But if the project only needs to load a glTF file, it is worth considering whether the draco algorithm is "cost-effective".

Use demo to compare performance:

It can be seen that the first loading and decryption time of the draco algorithm is longer than the original file. In actual project, the gap is more obvious, and occasionally decryption blockage occurs , need to re-enter the page to restore functionality.

In addition, there is a very intuitive problem, the loss of model image quality is considerable to the naked eye.

As shown in the figure, they look like in iPhone 12 and Xiaomi MIX2:

All in all, if you want to apply the draco compression algorithm to a large-scale project, you need to compare the following with the actual project:

• (1) It takes time to request two files + decryption. Compared with the compressed volume of the glb file itself, the actual performance is compared;
• (2) Whether there will be a loss of image quality that the designer cannot accept.

4.1.2 KHR_mesh_quantization

Vertex attributes are usually FLOAT , which converts the original floating-point value to 16-bit or 8-bit storage to adapt to a unified 3D or 2D grid, which is what we call quantization. The plug-in is mainly to vectorize it. .

For example, a static PBR-ready grid usually requires POSITION (12 bytes), TEXCOORD (8 bytes), NORMAL (12 bytes), and TANGENT (16 bytes) for each vertex, for a total of 48 bytes. Through this extension, it can be used for SHORT storage location and texture coordinate data (8 and 4 bytes respectively) and BYTE store normal and tangent data (4 bytes each), each vertex has a total of 20 bytes.

• Compression method

You can use gltfpack tool to compress

gltfpack -i male.glb -o male-processed.glb

• Usage (in Three.js)

Just use it in general, it's no different from uncompressed

import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const loader = new GLTFLoader()
loader.load(MODEL_FILE_PATH, (gltf) => {
 // ....
})

• Performance comparison

The original file is 3.2M and compressed 1.9M, which is 59.3% of the original file, which is much faster than the original model.
In the actual project, there is no problem of image quality loss and excessive loading time.

4.1.3 EXT_meshopt_compression

This plugin assumes that the buffer view data is optimized for GPU efficiency-using quantization and using the best data sequence for GPU rendering-and provides a compression layer on top of the bufferView data. Each bufferView is independently compressed, which allows the loader to decompress the data directly to the GPU storage to the greatest extent.

In addition to optimizing the compression rate, the compression format also has two features-very fast decoding (using WebAssembly SIMD, the decoder runs at a speed of about 1 GB/sec on modern desktop hardware), and characters compatible with general compression Section storage. In other words, instead of reducing the code size as much as possible, the bitstream is constructed in such a way that a general compressor can further compress it.

• Compression method

You can use the gltfpack tool to compress

gltfpack -i male.glb -o male-processed.glb -cc

• Usage (in Three.js)

import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'
import { MeshoptDecoder } from 'three/examples/jsm/libs/meshopt_decoder.module.js'

const loader = new GLTFLoader()
loader.setMeshoptDecoder(MeshoptDecoder)
loader.load(MODEL_FILE_PATH, (gltf) => {
 // ....
})

• Performance analysis and comparison

Original file 3.2M, compressed 1.1M, the original file 65.6% , first load times than the original model a lot faster.
In the actual project, there is no problem of image quality loss and excessive loading time.

Five, multiple models of equipment and optimization comparison results

In order to avoid the damage to the model caused by the "draco" compression mentioned above, I found several iPhone and Android phones to test the performance and compatibility. Let's take a look at the results.
PS: The company network has different network speeds in different time periods (such as morning and afternoon), which may have a small impact on the number, but it does not affect the horizontal comparison of file optimization.

iPhone 12 (iOS 14.4, personal use)

Huawei Mate 40 pro (HarmonyOS, personal use)

Xiaomi Mix2 (Android 8.0, test machine)

iPhone 6sp (iOS 13.7, personal computer)

5.1 Summary

It can be seen that for a small number of services that need to use models and only need to load one model, using KHR_mesh_quantization or EXT_meshopt_compression for grid compression, and then using gltf-pipeline for module differentiation and texture image compression is a better optimization solution found so far.

Six, other

In fact, there are still many performance-optimized plug-ins, which are currently undergoing debugging and investigation, and will continue to be updated when there are any new developments in subsequent iterations:

Grid optimized:

• EXT_mesh_gpu_instancing

  Currently 161151b5f6d57b 161151b5f6d57d of not yet supported, and 161151b5f6d57e 161151b5f6d57f of supports

There are also some texture-optimized plugins:

• KHR_texture_basisu
• EXT_texture_webp

Seven, reference materials

1. The Basic Structure of glTF
2. GLB File Format Specification
3. Extensions for glTF 2.0
4. KHR_draco_mesh_compression
5. DRACOLoader – three.js docs
6. CesiumGS/gltf-pipeline: Content pipeline tools for optimizing glTF assets.
7. KHR_mesh_quantization
8. 📦 gltfpack | meshoptimizer
9. GLTFLoader
10. EXT_meshopt_compression
11. [Mesh compression evaluation] MeshQuan, MeshOpt, Draco