Front-end multi-data rendering optimization

foreword

When I made a request some time ago, I came across the function of a custom list. All his data display is stored through jSON strings, and the use is also through JSON parsing. At first, he has a data upper limit, but later the upper limit is increased. Then there were problems such as Caton,
So this article is to introduce some solutions to solve the rendering problem of a large amount of front-end data

Program

innerHTML

The first is the rendering scheme from a long time ago innerHTML insert, it is the official API, and the performance is better

This is a simple HTML rendering example (the data is at the 10w level during the test, and the difference is enlarged, which will be less than this level in practice)

 const items = new Array(100000).fill(0).map((it, index) => {
    return `<div>item ${index}</div>`
}).join('')
content.innerHTML = items

Performance analysis from Google:

The page was refreshed and scrolled within 10 seconds, and it can be seen that the rendering of the dom blocked the page for 1300 ms

In the performance test, the total blocking time is controlled within 300 milliseconds to be a qualified state, and this time will also be affected by the computer hardware

Summarize the advantages and disadvantages of this method:

• Advantages: The performance is relatively acceptable, but there is also blocking when there is a lot of data

• shortcoming:

  • There is a danger of injection, and the matching with the framework is poor
  • Doesn't solve scrolling performance issues when there's too much dom

Bulk insert

Insert by sharding, if there are 10W pieces of data, we will divide it into 10 times, each time 1w pieces are inserted in a loop

 [...new Array(10)].forEach((_, i) => {
    requestAnimationFrame(() => {
        [...new Array(10000)].forEach((_, index) => {
            const item = document.createElement("div")
            item.textContent = `item ${i}${index}`
            content.append(item)
        })
    })
})

Analyzed by Google:

The performance analysis of page refresh and scrolling is also included here. It can be seen that the blocking time is 1800 milliseconds, which is slightly worse than innerHTML. This is in the order of 10w. The smaller the number, the smaller the time gap will be.

About requestAnimationFrame

The function of requestAnimationFrame : This method will tell the browser that it wants to perform the animation and request the browser to call the callback function to update the animation before the next redraw.

Execution method: When executing requestAnimationFrame(callback), the callback callback function will not be called immediately, it will be put into the callback function queue,
When the page is visible and the animation frame request callback callback function list is not empty, the browser will periodically add these callback functions to the queue of the browser UI thread (the system determines the execution timing of the callback function)

In general, it will not block the execution of other code, but the total execution time is not much different from the innerHTML scheme

Summarize the advantages and disadvantages:

• Advantages: Does not block the execution of the code

• shortcoming:

  • The total time spent inserting is still about the same as innerHTML
  • Likewise, scrolling performance issues are not resolved when there are too many DOMs

other native methods

canvas

Canvas is a tool specially used for drawing, which can be used for animation, game screen, data visualization, image editing and real-time video processing.

Recently, in the Web of the famous framework Flutter, canvas is used to render pages.

Similarly, we can also use canvas to render large amounts of data

 <div style="max-height: 256px;max-width:256px;overflow: scroll;">
    <canvas id="canvas"></canvas>
</div>

 let ctx = canvas.getContext('2d');
[...new Array(100000)].map((it, index) => {
    ctx.fillText(`item ${index}`, 0, index * 30)
})

After actual attempts, canvas is limited, and the maximum height of about 6w can no longer be enlarged, that is to say, under a large amount of data, canvas is still limited.

further optimization

Here is an optimization idea to monitor the scrolling of the outer DOM and dynamically render the canvas display according to the height, which can achieve the final effect, but the cost is still too high.

• Pros: Good performance on render counts

• shortcoming:

  • If you want to achieve the same rendering as a virtual list, it is uncontrollable (it is a better solution in other scenarios, such as animation, maps, etc.)
  • The styles in canvas are difficult to control

IntersectionObserver

IntersectionObserver provides an asynchronous observation of the intersection state of the target element and the viewport. Simply put, it can monitor whether an element will be seen by us. When we see this element, we can execute some callback functions to process certain elements. affairs.

Notice:
The implementation of IntersectionObserver should use requestIdleCallback(), that is, the observer will be executed only when the thread is idle. This means that the priority of this observer is very low, and it will only be executed when other tasks are finished and the browser is idle.

Through this api, we can make some attempts to implement a scheme similar to a virtual list

Here I implemented a virtual list demo that slides down. The main idea is to monitor all the DOMs in the list. When it disappears, remove and remove the monitor, and then add a new DOM and monitor.

Core code:

 const intersectionObserver = new IntersectionObserver(function (entries) {
        entries.forEach(item => {
            // 0 表示消失
            if (item.intersectionRatio === 0) {
                // 最后末尾添加
                intersectionObserver.unobserve(item.target)
                item.target.remove()
                addDom()
            }
        })
    });

Google's performance analysis (first entry to the page and continuous scrolling of 1000 items):

It can be seen that there is basically no blocking, this scheme is feasible, and there is no problem between initial rendering and scrolling

Click to view the details. The demo only implements the scroll down solution:
https://codesandbox.io/s/snowy-glade-w3i9fh?file=/index.html

further optimization

Now IntersectionObserver has implemented a function similar to a virtual list, but frequent addition of monitoring and cancellation seems to have hidden dangers, so I plan to take an expanded solution:

General idea:

The current list is a team of 10, and the current list renders a total of 30. When scrolling to the 20th, an event is triggered, the 30th-40th is loaded, and 0-10 are deleted at the same time, and then triggered in turn

In this case, the number of triggers and the number of monitoring will decrease in multiples. Of course, the cost is that the number of DOMs rendered by colleagues will increase. Later, we will increase the number of each team again, which can maintain a
The number of dom and monitoring are more balanced

compatible

About the compatibility of IntersectionObserver, through polyfill, you can get compatibility with most browsers, and at least support IE7, you can check: https://github.com/w3c/IntersectionObserver/tree/main/polyfill

Summarize the advantages and disadvantages:

• Advantages: A virtual list scheme implemented using native API, no data bottleneck

• shortcoming:

  • The adaptability of the framework in production is not high enough, and the implementation is more complicated
  • There may be some problems with frequent triggering of monitoring and dismissal during infinite scrolling

frame

So many methods mentioned above are all implemented in non-framework. Here we take a look at the performance of lists in react

react

Here is a render of a list of length 10,000

 function App() {
  const [list, setList] = useState([]);

  useEffect(() => {
    setList([...new Array(50000)]);
  }, []);

  return (
    <div className="App">
      {list.map((item, index) => {
        return <div key={index}>item {index}</div>;
      })}
    </div>
  );
}

When the demo is running, it is obvious that the page is stuck

According to Google analysis, in the order of magnitude of 50,000, after the refresh, the rendering is still not completed in 10 seconds. Of course, the performance in the framework is definitely not as strong as the native one. This conclusion is expected

Online demo address: https://codesandbox.io/s/angry-roentgen-25vipz

Another point to note is the transmission of large amounts of data in templates:

 // 这个 list 的数量级是几千甚至上万的, 会导致卡顿成倍的增加, 
<Foo list={list}/>

This conclusion is applicable whether it is in vue or react, so the transfer of a large amount of data must be assigned and obtained in memory, not through conventional methods such as modules and render.

If the order of magnitude is 100, we can also consider optimization, which can add up to a lot

startTransition

There will also be new APIs in startTransition :

 startTransition(() => {
    setList([...new Array(10000)]);
})

The function of this API is the same as what I said above requestAnimationFrame , it does not enhance performance, but it can avoid stuck, give priority to rendering other components, and avoid white screen

virtual list

The concept of virtual list is officially introduced here

Stores the positions of all list elements, and only renders the list elements in the viewport. When the viewport is scrolled, it calculates which elements should be rendered in the viewport according to the scrolling offset size and the positions of all list elements.

A moving picture to understand the principle:

minimal implementation

Here we try to implement a minimal virtual list scheme by ourselves:

 // 这是一个 react demo, 在 vue 项目中, 原理类似, 除了数据源的设置外基本没什么变化

// 数据源以及配置属性
const totalData = [...new Array(10000)].map((item, index)=>({
    index
}))
const total = totalData.length
const itemSize = 30
const maxHeight = 300

function App() {
  const [list, setList] = useState(() => totalData.slice(0, 20));

  const onScroll = (ev) => {
    const scrollTop = ev.target.scrollTop
    const startIndex = Math.max(Math.floor(scrollTop / itemSize) -5, 0);
    const endIndex = Math.min(startIndex + (maxHeight/itemSize) + 5, total);
    setList(totalData.slice(startIndex, endIndex))
  }

  return (
          <div onScroll={onScroll} style={{height: maxHeight, overflow: 'auto',}}>
            <div style={{height: total * itemSize, width: '100%', position: 'relative',}}>
              {list.map((item) => {
                return <div style={{
                  position: "absolute",
                  top: 0,
                  left: 0,
                  width: '100%',
                  transform: `translateY(${item.index *itemSize}px)`,
                }} key={item.index}>item {item.index}</div>;
              })}
            </div>
          </div>
  );
}

Check out the online demo: https://codesandbox.io/s/agitated-shtern-phcg6z?file=/src/App.js

This is the smallest example of a virtual list, it is mainly divided into 2 parts

1. It needs to be wrapped in a container, and use CSS to increase the height, and the actual rendered item needs to use transform to display it in the correct position
2. Monitor the scrolling of the external container. When scrolling, dynamically slice the original data source and replace the list that needs to be displayed.

Let's check his performance:

Basically no blocking, occasionally a little frame loss

This demo is not a final form. There are still many places that can be optimized, such as caching, logic extraction, CSS re-simplification, control of the trigger frequency of scrolling, scrolling direction control, etc. There are many points that can be optimized.

other libraries

• The virtual list solution recommended by many react-virtualized libraries is large and comprehensive
• react-window react-virtualized Recommended library, more lightweight alternative.
• The hooks form of the react-virtual virtual list, similar to the logic encapsulation in my demo

chrome official support

virtual-scroller

At the Chrome dev summit 2018, Google Engineering Manager Gray Norton introduced us to virtual-scroller, a web scrolling component that may become a web high-level API (Layered
part of the API). Its goal is to solve performance problems with long lists and eliminate off-screen rendering.

However, after developing some parts, after internal discussions, it is still necessary to terminate this API and turn to CSS development <br>Link: https://github.com/WICG/virtual-scroller/issues/201
Chrome's introduction to virtual-scroller: https://chromestatus.com/feature/5673195159945216

content-visibility

This is the new CSS property developed later

Chromium 85 has the content-visibility property, which is probably the most effective CSS property for improving page load performance. content-visibility allows user agents to normally skip element rendering work (including layout and painting) unless needed. Do the rendering work. If the page has a lot of off-screen content, the content-visibility attribute can skip the rendering of off-screen content, speed up the rendering time of the user's first screen, and reduce the waiting time for the page to interact.

Specific introduction: https://web.dev/content-visibility/

The way to use it is to add CSS properties directly

 #content {
  content-visibility: auto;
}

It is a pity that his effect is to enhance the rendering performance, but when a large amount of data is initialized, it will still be stuck, which is not as direct and effective as the virtual list.

But we can consider using it when we reduce the rendering time of the first screen

Summarize

Performance optimization under multiple data, there are many solutions

• requestAnimationFrame
• canvas
• IntersectionObserver
• startTransition
• virtual list
• content-visibility

In general, virtual lists are the most efficient, and you can use the simplest demo level to temporarily optimize your code