"🌟Project actual combat🌟" Optimize project construction time

background

When I got off work the other night, I passed by the project team next door and listened to them talking about the project construction:

Now the online packaging time is too long. Fix a bug for 1 minute and release it for half an hour, which is uncomfortable for thieves.

Their project is relatively large, and the online construction time is particularly long, basically more than 15 minutes .

After a brief chat with them, I went back and took a look at the build time of my project:

It's actually quite long, so I took the time to optimize it, and the effect is quite obvious:

In the body part, the main content I will share is:

• Some configurations to improve webpack packaging performance
• Some thoughts on optimizing the construction time of large-scale projects

Hope to inspire everyone.

text

Our project is not very big, it is a medium-sized international project with about a hundred pages.

The local build time was quite long before, and the initial startup took three minutes. Later, I configured Vite , and the local startup time was reduced to about 20s . If you are interested, please refer to this article:

Webpack to Vite, speed up development!

I took a look. The online build time is five or six minutes, which is not painful or itchy, but there should be room for optimization, so I am going to optimize it.

1. Find the problem

As to optimize build time, the first step of course is to identify problems, find out the comparative time-consuming stage, coupled with optimization.

Here I used the SMP plug-in.

SMP usage of the 06093c531eef90 plug-in is very simple, here is also a brief mention:

// webpack.config.js
const SpeedMeasurePlugin = require('speed-measure-webpack-plugin');
const smp = new SpeedMeasurePlugin();

module.exports = smp.wrap({
  // ...
});

Use the SMP plug-in to get the packaging time for each stage:

Found two more obvious problems:

1. IgnorePlugin took nearly 20 seconds.
2. less-loader part was executed twice, and more than a minute was wasted.
3. ts-loader takes a minute and a half and is quite long.

2. Solve the problem

1. IgnorePlugin

After checking the configuration, I found that IgnorePlugin in the configuration did not achieve the expected effect, so I deleted it.

2. less-loader

After checking the configuration, it is found that less is indeed processed again.

For the processing of less files, you can directly read the official document, the document address:

https://webpack.docschina.org/loaders/less-loader/

My configuration:

{
  test: /\.less$/,
  use: [
    'style-loader',
    'css-loader',
    {
      loader: 'less-loader',
      options: {
        javascriptEnabled: true,
        sourceMap: true,
        modifyVars: {
          // inject our own global vars
          // https://github.com/ant-design/ant-design/issues/16464#issuecomment-491656849
          hack: `true;@import '${require.resolve('./src/vars.less')}';`,
          ...themeVariables,
        },
        limit: 10000,
        name: '[name].[hash:7].[ext]',
        outputPath: 'styles/',
      },
    },
  ],
},
{
  test: /\.css$/,
  use: ['style-loader', 'css-loader'],
},

3. ts-loader

For ts-loader , you can refer to:

https://webpack.js.org/guides/build-performance/#typescript-loader

The document also has a clear description:

The documentation suggests that we turn on the transpileOnly option and turn off the type checking.

If you want type checking, you can use ForkTsCheckerWebpackPlugin , this plug-in will do related checks in another process.

With this plugin, we have also explored the problem of memory overflow when optimizing the build. If you are interested, you can move to my article:

project build memory overflow? Take a look at Node memory limit

Now we also turn on this option.

After opening, when building locally, a warning was reported locally:

This error is very familiar. It is the problem import type

"import type" you don't know

Fix it:

problem solved.

Rebuild, and get the following results:

After optimization, we found:

• IgnorePlugin、HtmlWebpackPlugin time is greatly shortened.
• less-loader and so on returned to normal, only executed once.
• ts-loader greatly shortened, from 1 minute 30 seconds to 40 seconds.

The local effect is obvious and needs to be verified online.

3. Confirmation is valid

After executing it online, the following results are obtained:

Then I checked the page, and it was all normal.

perfect!

Looking back, it is not difficult to find that, in fact, not much has been changed, and good results have been achieved.

For small and medium-sized projects, changing the configuration can often meet our requirements, but what about the large-scale project?

For example, projects with dozens of modules and hundreds of pages.

Back to the question at the beginning: fix a bug for 1 minute, and release it for half an hour.

Simple modification of the configuration cannot bring down the time. What should I do at this time?

Some thoughts on optimizing the construction time of large-scale projects

Split sub-application

Assuming we have a project, there are nearly 30 large modules:

There are dozens of pages in each large module. This kind of system will take a long time to build and needs to be optimized.

And in the later stages of the project, problems will become more and more obvious, such as:

• The code is getting bloated
• The business module itself is not related
• Build is getting slower and slower
• Cannot be deployed independently

Faced with this situation, a feasible approach is: split sub-applications.

Architecture after split:

Each sub-project has a separate entrance, which is a project that can be deployed independently.

The sub-projects are umd into a separate 06093c531ef600 package:

When the main project starts, load these sub-projects again:

After loading, you need to process the route and store , sample code:

// base
export const bootstrap = () => {
  // ...
  ReactDOM.render((
    <Provider store={store}>
      <Router history={history}>
        <App defaultInitialData={_initialData} />
      </Router>
    </Provider>
  ), document.getElementById('root'));
  return Promise.resolve();
};

// main
const loadSubApp = (htmlEntry: string) => {
  return importHTML(`${htmlEntry}?${Date.now()}`)
    .then((res: any) => res.execScripts())
    .then((exportedValues: any) => {
      console.log(`importHTML: ${htmlEntry} loaded, exports:`, exportedValues);
      const { store, router } = exportedValues || {} as any;
      router && addCustomRouter(router);
      store && addCustomStore(store);
    })
    .catch(e => {
      console.error('importHTML: ${htmlEntry} load error:', e);
    });
};

const load = () => {
  if (__ENV__ !== 'dev') {
    const paths: string[] = [];
    subAppConfig.subApps.forEach(item => {
      if (item.project === localStorage.getItem('ops_project')) {
        paths.push(...item.paths);
      }
    });
    Promise.all(paths.map(path => loadSubApp(path)))
      .catch(e => console.log(e))
      .finally(setAllLoaded);
  } else {
    setAllLoaded();
  }
};

const init = () => {
  console.log('init: Start to bootstrap the main APP');
  addCustomStore(rootStore);
  bootstrap().then(() => {
    load();
  });
};

init();

Code sharing

• common package
• • component
• • utils
• • typings
• • ..
• externals
• • react family bucket
• • moment
• • antd
• • ..

Style isolation

namespace with the name of the sub-item to the style:

Development and debugging

Take the ops project as an example.

Make the development and debugging of the ops-common package as convenient as a local file:

1. Let the project compile the common package

1. wepback alias

1. TS alias

Standalone deployment

Apply for a separate module for each sub-project on the same project

Advantages and disadvantages of splitting sub-applications

advantage:

• Each sub-application can be independently at 16093c532110b2, and the sub-module and the main module are decoupled.
• The sub-projects can be compiled separately at , and the main project only needs to be imported, so that reduce the construction time of the main module.

Disadvantages:

• Additional complexity and maintenance costs

in conclusion

Generally speaking, for small and medium-sized projects, optimizing the packaging configuration can solve some of the problems.

To optimize the construction time of large projects, consider the mode of splitting sub-applications.

It's just that this model needs to consider some maintenance issues, such as how to maintain the version tag, how to quickly roll back and so on.

These need to be combined with the actual situation of decision.

Today's content is so much, I hope to inspire everyone.

At last

Happy May Day everyone~~

If you find the content of the article helpful, you can follow me to keep up with the latest developments.

You can also add my WeChat "scaukk" to discuss together.