In those years, the performance optimizations we did together

Introduction to is a systematic and holistic thing, imprinted in the details of the project development process, and it is also a big battlefield that reflects the depth of technology. Based on the complex system of Quick BI, the article introduces in detail the ideas and methods of performance optimization, as well as systematic thinking.

-For more information about digital intelligence transformation and data center content, please join Cloud Data Center Exchange Group-Digital Intelligence Club and follow the official WeChat official at the end of the article to join )

Cloud Data Center official website 160f79bc8025e4 https://dp.alibaba.com/index

---

Performance has always been an unavoidable topic at the technical level, especially in medium and large complex projects. Just like the performance of a car, while pursuing extreme speed, it must also ensure comfort and practicability. Every aspect of automobile manufacturing, parts integration, engine tuning, etc. will ultimately affect the user's physical perception and commercial achievement, as shown in the figure below. The impact of performance on revenue.

Performance optimization is a systematic and holistic thing, imprinted in the details of the project development process, and it is also a big battlefield that reflects the depth of technology. Next, I will use Quick BI's complex system as the background overall performance optimization ideas and methods, as well as systematic thinking.

How to locate performance issues?

Generally speaking, we are more sensitive to the frame rate of animation (within 16ms), but if there is a performance problem, our actual somatosensory may be one word: "slow", but this does not provide us with any help in solving the problem. Therefore, we need to analyze the entire link behind the word.

The above picture is the general processing flow of the browser. Combined with our scenario, I abstracted it into the following steps:

It can be seen that the main time-consuming phases are divided into two:

Phase 1: Download Code

Phase 2: Script Execution & Fetch Data

How to go deep into these two stages, we generally use the following main tools to analyze:

Network

The first tool we want to use is Chrome's Network, which can help us initially locate the bottleneck:

As shown in the example, you can see the entire page at a glance in the Network: loading time (Finish), loading resource size, number of requests, time and time-consuming points of each request, resource priority, etc. The above example can be clearly seen: the resource loaded on the entire page is very large, close to 30MB.

Coverage (code coverage)

For complex front-end projects, the products built by the project are generally redundant or even unused. These invalidly loaded codes can be analyzed in real time through the Coverage tool:

As shown in the example above: the entire page is 28.3MB, of which 19.5MB is not used (executed), and the engine-style.css file usage rate is less than 0.7%

Resource big picture

Just now we have known that the utilization of front-end resources is very low, so which invalid codes have been introduced? At this time, we need to use webpack-bundle-analyzer to analyze the entire build product (product stats can be webpack --profile --json=stats.json ):

As in the above example, combined with our current business, we can see the problems of building products:

First, the initial package is too large (common.js)

Second, there are multiple duplicate packages (momentjs, etc.)

Third, the third-party package relied on is too large

Module dependencies

With the big picture of resource construction, we also probably know the points that can be optimized, but in a system, hundreds of modules are generally organized by mutual reference, and the packaging tools are used to build them through dependencies. Together (for example, as a single file in common.js), it may not be easy to directly remove a certain module code or dependencies. Therefore, we may need to take a certain degree and use tools to sort out the dependencies of the modules in the system. Then optimize by adjusting dependencies or loading methods:

In the above picture, we are using the official analyze tool of webpack (other tools include: webpack-xray, Madge), you only need to upload the resource big picture stats.json to get the whole dependency big picture

Performance

The tools mentioned above are all tools related to resource loading, so in analyzing what we use in the "execution & access" link, Chrome provides a very powerful tool: Performance:

As shown in the example above, we can find at least a few points: main process serialization, long tasks, and high-frequency tasks.

How to optimize performance?

Combined with the analysis tools just mentioned, we have basically covered the two major stages of "resource package download" and "execution & access" just mentioned, and the fundamental problems and solutions are gradually developed in continuous analysis. Ideas, here I will combine our scenes here to give some good optimization ideas and effects

Large packages are loaded on demand

You should know that the front-end engineering construction and packaging (such as webpack) generally starts from the entry to find the entire dependency tree (direct dependency), so as to produce multiple js and css file bundles or trunks based on this tree, and once a module appears In the dependency tree, when the page loads the entry, the module will also be loaded at the same time.

So our idea is to break this direct dependency and use asynchronous dependency for the end modules, as follows:

import { Marker } from '@antv/l7' synchronous 060f79bc802a3b to asynchronous, so that during construction, the dependent Marker will form a chunk, and only when this piece of code is executed (on demand), the thunk will be loaded, thereby reducing the size of the first screen package.

However, there is a problem with the above scheme. The construction will use the entire @antv/l7 as a chunk instead of the Marker part of the code, which causes the TreeShaking of the chunk to fail and the volume is very large. We can use the construction sharding method to solve:

As above, create Marker segment file, so that with TreeShaking ability for asynchronous then introduced on this basis.

The following are the results of our optimized process comparison:

In this step, we unpack on demand and load asynchronously, saving resource download time and part of the execution time

Resource preload

In fact, we have discovered a "main process serialization" problem in the analysis stage. The execution of js is single-threaded, but the browser is actually running in multiple threads, which includes asynchronous requests (fetch, etc.), so we further The idea is to parallelize Fetch Data and resource downloading through multiple threads.

According to the current status quo, the logic of interface fetching is generally coupled in business logic or data processing logic, so the step of decoupling (decoupling from UI, business modules, etc.) is essential, and pure fetch requests (and a small amount of processing) (Logical) stripped out and put in a higher priority stage to initiate the request. So where to put it? We know that the browser's processing of resources has priority, normally in the following order:

1. HTML/CSS/FONT
2. Preload/SCRIPT/XHR
3. Image/Audio/Video
4. Prefetch

To achieve parallel resource fetching and initiating fetching, it is necessary to advance fetching to the first priority (execute immediately after HTML parsing is completed, instead of waiting for the request to be initiated during the execution of the SCRIPT tag resource loading), our process will Becomes as follows:

Need special attention: Since the execution of JS is serial, the logic that initiates the fetch must be executed before the logic of the main process, and cannot be placed in nextTick (such as using setTimeout(() => doFetch())), otherwise The main process will always occupy CPU time so that the request cannot be sent

Active task scheduling

The browser also has a priority strategy for resources, but it does not know what resources we want to load/execute first at the business level, and which resources load/execute afterwards, so let’s jump out and see if we load the entire business level resources +The execution/fetching process is divided into a small task, and these tasks are fully controlled by ourselves: packaging granularity, loading timing, execution timing, does it mean that CPU time and network resources can be maximized?

The answer is yes, but generally for simple projects, the scheduling priority strategy of the browser itself is enough to meet the needs, but if relatively extreme optimization is to be done for large and complex projects, it is necessary to introduce "custom task scheduling" Plan.

Taking Quick BI as an example, our initial goal is to make the main content on the first screen appear faster. Then, from the resource loading, code execution, and access levels, CPU/network allocation should be based on our business priority. For example, I hope that the "card drop-down menu" will only be displayed after the main content on the first screen is displayed or when the CPU is idle. Start loading (that is, lower the priority, even when the user mouses into the card, and hope it will increase the priority to start loading and display immediately). as follows:

Here we encapsulate a task scheduler, the purpose of which is to declare a piece of logic to start execution after a certain dependency (Promise) is completed. Our flowchart changes are as follows:

The yellow blocks represent part of the modules for priority downgrade processing, which helps reduce the entire first screen time

TreeShaking

Most of the methods mentioned above are based on priority. In fact, in the era of increasingly complex front-end engineering (medium and large projects have exceeded hundreds of thousands of lines of code), a more intelligent optimization scheme was born to reduce the packet size. The idea is very simple: Tool-based analysis of dependencies, and remove the unreferenced code from the final product.

It sounds cool, and it's actually very good to use, but here I want to talk about a lot of points that its official website does not mention --- TreeShaking often fails:

side effect

Side Effects usually express the code that affects the whole world (such as window objects, etc.) or the environment.

As shown in the example, the b code seems to be unused, but there console.log(b(1)) its file, and packaging tools such as webpack dare not remove it easily, so it will be entered as usual.

Solution

In the package.json or webpack configuration, clearly specify which code has side effects (for example, sideEffects: [“**/*.css”] ), and the code without side effects will be removed

IIFE code

IIFE will be immediately executed function expression (Immediately invoked function expression)

As shown in the figure, this type of code will cause TreeShaking to fail

Solution

Three principles:

• [Avoid] function call executed immediately
• [Avoid] New operation executed immediately
• [Avoid] Code that affects the whole world immediately

Lazy loading

We mentioned in the "Load on Demand" section that asynchronous import for unpacking will cause TreeShaking to fail. Here is another case:

As shown in the figure, because index.ts synchronously bar.ts in sharedStr , and then somewhere, asynchronous import('./bar') at the same time, in this case, it will cause two problems at the same time:

1. TreeShaking is invalid ( unusedStr will be broken into)
2. Asynchronous lazy loading fails ( bar.ts will be index.ts )

When the amount of code reaches a certain level, this problem is prone to occur in N-person collaborative development

Solution

• [Avoid] Import the same file synchronously and asynchronously

On-demand strategy (Lazy)

In fact, there are some on-demand loading solutions mentioned earlier, here we will extend it appropriately: Since the loading of resource packages can be on-demand, can the rendering of a certain component be on-demand? Can an object instance be used on demand? Can a certain data cache be generated on demand?

LazyComponent (LazyComponent)

As shown in the figure, PieArc.private.ts corresponds to a complex React component. PieArc is makeLazyComponent . The component will be loaded and executed only when the code is executed here. Even the second parameter (deps) can be used to declare the dependency, and it will be loaded and executed when the dependency (promise) is completed.

LazyCache

Lazy caching is used in this scenario: the result of a data conversion in a data stream (or other subscribed data) needs to be used anywhere, and the conversion is performed only at the moment of use

Lazy Object (LazyObject)

Lazy object means that the object will only be instantiated when it is used (attributes/methods are accessed, modified, deleted, etc.)

As shown in the figure, when globalRecorder was introduced, it was not instantiated, and it was instantiated only when globalRecorder.record() was called

Data flow: throttling rendering

In order to facilitate state management in medium and large projects, data flow schemes are usually used, as follows:

The data stored in the store is usually atomic, with very small granularity. For example, there are N atomic properties in state: a, b, c..., etc. A component relies on these N properties for UI rendering, assuming N properties Will be changed under different ACTIONs, and these changes all occur within 16ms, then if N=20, there will be 20 View updates within 16ms (1 frame):

This obviously will cause very big performance problems. Therefore, we need to buffer and throttle the short-term ACTION amount. After the 20 ACTION status changes, only one View update is performed, as follows:

This solution plays a role in Quick BI in the form of redux middleware, and has a good effect in complex + frequent data update scenarios

Thinking

"Gentlemen are hesitant to prevent them by thinking about problems." When we look back, more than 80% of these performance problems can be avoided in the architecture design and coding stages, and 20% can be "space <=> "Time replacement strategy" and other methods to balance. Therefore, the best performance optimization solution lies in our dedication to the quality of each piece of code: Have we taken into account such module dependencies and possible build product volume problems? Have you considered the possible execution frequency of this logic? Have you considered the controllability of space or CPU usage as data grows? and many more. There is no silver bullet for performance optimization. As a technical person, you need to cultivate in your heart (familiar with the underlying principles) and put your obsession with performance into your instinctive thinking. can click here to get a free trial of the product and experience the performance optimization of Quick BI.

Related products: Intelligent data construction and management Dataphin

---

Data center is the only way for enterprises to achieve digital intelligence. Alibaba believes that data center is a combination of methodology, tools, and organization, which is "fast", "quasi", "full", "unified", and "passed". Smart big data system.

Cloud is currently exporting a series of solutions, including general data middle station solution , retail data middle station solution , financial data middle station solution , , 160f79bc803416 internet data middle station solution , Sub-scenarios such as and other subdivision scenarios for government data middle-office solutions.

Among them, the Alibaba Cloud Data Center product matrix is based on Dataphin and the Quick series is used as a business scenario cut-in, including:

• -Dataphin, a one-stop, intelligent data construction and management platform ;
• -Quick BI, intelligent decision-making anytime, anywhere ;
• -Quick Audience, comprehensive insight, global marketing, intelligent growth ;
• -Quick A+, a one-stop data operation platform cross-multi-terminal global application experience analysis and insight;
• -Quick Stock, an intelligent goods operation platform ;
• -Quick Decision, an intelligent decision platform ;

official site:

Data Zhongtai official website https://dp.alibaba.com

Dingding Communication Group and WeChat Official Account

Copyright Statement: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users. The copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.