Threejs 360 panoramic tool development experience

foreword

Recently I made a 720 panorama tool. Many developers think that the panorama is very simple. There are also many source code of the panorama on github. Below I will sort out some of the pits and solutions we encountered in developing this project, which have been published online https://720.h5ds.com/

Tool introduction:

H5-720 Panorama (H5-720 for short) is a panorama production tool based on threejs + h5. Users can upload panoramic pictures to make panoramas. H5-720 can preset buttons, add elements, add hot spots, add special effects, add A variety of interactive methods, one-click to turn on gravity sensing, VR glasses support, and multi-terminal viewing can be adapted after PC-side production.

What is a panorama?

Before that, I will briefly introduce the panorama, the pictures of the surrounding pictures that can be viewed in 360 degrees without dead ends. The professional explanation of Baidu Encyclopedia is as follows: Panorama is expressed through wide-angle means and in the form of paintings, photos, videos, 3D models, etc. , showing as much of the surroundings as possible. 360 panorama, that is, by capturing the image information of the entire scene with a professional camera or using the modelling software to render the picture, using the software to stitch the picture, and playing it with a special player, that is, the plane photo or computer modeling picture becomes 360 degrees The full view is used for virtual reality browsing, which simulates the two-dimensional floor plan into a real three-dimensional space and presents it to the viewer.

Panorama Asteroid Perspective:

Panorama VR perspective:

When we want to view the panorama, we need to use special software.

Some are called 720 panorama and some are called 360 panorama. In fact, they are all the same, because some people add up the left and right 360 degrees and the up and down 360 degrees, which means that you can view up, down, left and right.

Panorama web solution

To display the panorama on the web side, we use threejs to implement it here. There are three common solutions at present.

1. Map the panorama to the sphere

Similar to the globe, this method is the simplest, but this method has a big flaw. If the texture is too large, the mobile browser will crash directly when it is opened, and cannot support high-definition pictures, and the performance of the PC will be very poor. If it is too large, the GPU performance requirements will be very high.

2. Slice the picture into 6 pieces and map them to a cube

The optimized version for the first case, and the officially provided panorama solution is a hexahedron panorama, which slices the panorama into 6 pieces and maps them to a cube. We can slice the panorama into 6 using a tool: https://jaxry.github.io/panorama-to-cubemap/

Then map to the 6 faces of the cube

This also has some defects: it does not support ultra-high-definition pictures, because the ultra-high-definition pictures will be very large if sliced into 6 pieces, and if it is a cube texture, the asteroid view cannot be used. So the third ultimate solution emerges

3. Slice the panorama into N tile images, and then map them to a sphere

In our project, the picture was cut into 16 parts

Then use the threejs custom shader to paste the texture on the sphere, and each tile image will be loaded and updated independently, so that it can be loaded on demand. To make the loading effect from blur to clear, you need to make 2 layers of balls, the red ball wraps the blue ball, the blue ball uses a tile image, and the red image uses a low-definition panorama, so that the initial blur can be achieved, After each tile image is loaded successfully, it gradually becomes clear. Of course, if you want a higher-definition effect, you can monitor the zoom ratio, and replace the higher-definition tile image in the viewing angle after the zoom becomes larger.

We use a third party to solve the performance problem and preloading problem of panorama