workspaces - monorepo in action

foreword

Since v7, npm has introduced a very powerful feature, which is workspaces . In addition, yarn and pnpm also have the ability workspaces However, in terms of usage, it is almost identical. Therefore, if you learn npm workspaces, you will naturally learn yarn and pnpm.

Overview

This article will be divided into four parts:

1. What are workspaces;
2. Multi-package management;
3. multi-project management;
4. avoid pit;
5. Summarize;

What are workspaces?

As the name suggests, workspaces is the concept of multiple spaces, which can be understood as multiple packages in npm. Its original intention is for multi-package management. It makes it very convenient to develop and manage multiple npm packages in the same project:

• It will upgrade all dependent packages in subpackages to the root directory for installation, which improves the speed of package installation;
• After it is initialized, it will automatically associate dependencies between subpackages (soft links);
• Because of the relationship of the same project, each sub-package can share some processes, such as: eslint, stylelint, git hooks, publish flow, etc.;

This design pattern originally came from Lerna, but Lerna has stronger capabilities for multi-package management, and the latest version of Lerna is fully compatible with the workspaces mode of npm or yarn. However, because this article is about workspaces, students who are interested in Lerna can go to Lerna official website learn.

Multi-package management

The advantages of multi-package management have been mentioned above compared to single-package management. Therefore, we let the students feel why the workspaces are so bragging by me through real examples.

example demonstration

Project address I hung on github, interested students can view their own source .

1. Upgrade npm to 7 or the latest version

npm i -g npm@latest

2. Create a project

mkdir demo-workspaces-multi-packages

3. Initialize the project

npm init -y
.
└── package.json

4. Declare that this project is in workspaces mode

package.json New configuration:

"private":"true",
"workspaces": [
  "packages/*"
],

The packages/* here means that our subpackages are all under the packages folder. (For the details and more usage of workspaces, this article will not introduce them one by one, the documentation is very clear, and this article is about actual combat)

5. Initialize subpackage m1

Create subpackage m1 :

npm init -w packages/m1 -y
.
├── package.json
└── packages
    └── m1
        └── package.json

Creating m1 master file index.js :

echo "exports.name = 'kitty'" >> packages/m1/index.js
.
├── package.json
└── packages
    └── m1
        ├── index.js
        └── package.json

6. Initialize subpackage m2

In the same way, create subpackage m2 :

npm init -w packages/m2 -y
.
├── package.json
└── packages
    ├── m1
    │   ├── index.js
    │   └── package.json
    └── m2
        └── package.json

Creating m2 master file index.js :

echo "const { name } = require('m1')\nexports.name = name" >> packages/m2/index.js
.
├── package.json
└── packages
    ├── m1
    │   ├── index.js
    │   └── package.json
    └── m2
        ├── index.js
        └── package.json

Because require('m1') here, you need to add m1 dependency to m2 of package.json :

npm i -S m1 --workspace=m2

7. Initialize subpackage demo

In order to facilitate us to see the effect, create a demo folder (multi-package management recommends a demo subpackage for overall effect testing):

npm init -w packages/demo -y
echo "const { name } = require('m2')\nconsole.log(name)" >> packages/demo/index.js
.
├── package.json
└── packages
    ├── demo
    │   ├── index.js
    │   └── package.json
    ├── m1
    │   ├── index.js
    │   └── package.json
    └── m2
        ├── index.js
        └── package.json

In addition, this demo package, we do not release it like him, in order to prevent accidental release, we added in demo of package.json :

"private":"true",

Because require('m2') here, you need to add m2 dependency to demo of package.json :

npm i -S m2 --workspace=demo

Let's take a look at node_modules project root directory at this time:
<img width="300px" src="https://user-images.githubusercontent.com/17001245/148692468-e94beeca-3205-40f9-9e2b-b4c145a2f4a4.png">
Is it interesting? All are soft links, and the links point to the sub-packages under the packages

OK, after a long time, let's run demo see the effect:

node packages/demo/index.js
# 输出：
kitty

From the above example, we can see that workspaces dependencies between local subpackages very cleverly, which also makes it more convenient for developers, especially when multiple people develop. After the other person pulls the project, he only needs to run npm install to develop, and the soft link will be automatically established.

Next, let's see if the tripartite package is installed in the workspaces

8. Install two different versions of the package

npm i -S vue@2 --workspace=m1
npm i -S vue@3 --workspace=m2

In the example, we want to see, because our packages will be promoted to the root directory for installation, so what will it do with vue Will it only install the vue3 package?

result:
<img width="300px" src="https://user-images.githubusercontent.com/17001245/148693126-3426d7b8-a011-4634-87e4-e1e52e5c798b.png">
In this way, we don't need to worry about version conflicts, workspaces is obviously well resolved.

Key parameters --workspace

In the workspaces project, a very core parameter is --workspace , because the commands from the previous installation package to subpackage can be found, like the traditional installation package, use npm i -S package name or npm i -D The package name, the only difference is that --workspace is added at the end.

Is it the same for other commands, such as run , version , publish etc.? The answer is: Yes!

In addition, if package.json in scprits have a test , and we want to run this command for all subpackages at one time, we can use npm run test --workspaces .
In this case, it is very convenient Lint check or single test

At this point, the role of workspaces in multi-package management has basically been introduced. It is worth mentioning that for multi-package management, Lerna is still recommended in actual projects. It has a very mature process mechanism for version dependency automatic upgrade, package notification, automatic generation of Log (Change Log / Release Note), CI, etc. .

Multi-project management

The current npm workspaces , I personally think it is very suitable for multi-project integration (Monorepo) management.

example demonstration

Project address I hung on github, and interested students can view their own source .

1. Create a project

mkdir demo-workspaces-multi-project

2. Initialize the project

npm init -y
.
└── package.json

3. Declare that this project is in workspaces mode

package.json New configuration:

"private":"true",
"workspaces": [
  "projects/*"
],

4. Initialize subproject zoo

Create subproject zoo :

npm init -w projects/zoo -y
.
├── package.json
└── packages
    └── zoo
        └── package.json

Create a template file index.html , the main content is:

<!-- projects/zoo/index.html -->
<body>
  <h1>Welcome to Zoo!</h1>
  <div id="app"></div>
</body>

Create the project entry js file index.js with the content:

console.log('Zoo')

Install the project build dependencies:

npm i -S webpack webpack-cli webpack-dev-server html-webpack-plugin webpack-merge --workspace=zoo

# projects/zoo/package.json
"private":"true",
"dependencies": {
  "html-webpack-plugin": "^5.5.0",
  "webpack": "^5.65.0",
  "webpack-cli": "^4.9.1",
  "webpack-dev-server": "^4.7.2"
}

Create webpack configuration:

// projects/zoo/webpack/base.config.js
const HtmlWebpackPlugin = require('html-webpack-plugin')
const path = require('path')

function resolve(dir) {
  return path.join(__dirname, '../' + dir)
}

exports.config = {
  entry: resolve('src/index.js'),

  plugins: [
    new HtmlWebpackPlugin({
      title: 'Zoo',
      filename: 'index.html',
      template: resolve('src/index.html')
    })
  ],
}

exports.resolve = resolve

// projects/zoo/webpack/dev.config.js
const { config, resolve } = require('./base.config')
const { merge } = require('webpack-merge')

exports.default = merge(config, {
  mode: 'development',

  output: {
    filename: 'bundle.js',
  },
})

// projects/zoo/webpack/prod.config.js
const { config, resolve } = require('./base.config')
const { merge } = require('webpack-merge')

exports.default = merge(config, {
  mode: 'production',

  output: {
    filename: 'bundle.js',
  },
})

New commands package.json under zoo:

"scripts": {
  "dev": "webpack-dev-server --config webpack/dev.config.js --open",
  "prod": "webpack --config webpack/prod.config.js"
},

Then you can run it, just use it in the project root directory:

npm run dev --workspace=zoo

Local development is available.

Effect:
<img width="200px" src="https://user-images.githubusercontent.com/17001245/148756703-5d1a71e7-ecf3-4d56-947d-2a8ad2f2c4d1.png">

Running prod is the same.

5. Initialize subproject shop

Create subproject shop :

npm init -w projects/shop -y

The rest of the steps are zoo , so I won't repeat them.

Final directory structure:
<img width="150px" src="https://user-images.githubusercontent.com/17001245/148758564-84a086c0-caf1-4042-b6e7-d0e232787490.png">

shared

For Monorepo, sharing is one of the most important advantages. So, let's do something shared.

share folder in the root directory as a shared resource directory, and create a shared file Fish.js :

mkdir share
mkdir share/js
touch share/js/Fish.js

// share/js/Fish.js
class Fish {
  constructor(name, age) {
    this.name = name
    this.age = age
  }

  swim() {
    console.log('swim~')
  }

  print() {
    return '🐟 '
  }
}

module.exports = Fish

2. Added alias webpack configuration in the sub-project

Subprojects zoo and shop can both add the same alias :

resolve: {
  extensions: ['.js'],
  alias: {
    '$share': resolve('../../share'),
  },
},

The entry file of subproject zoo

// projects/zoo/src/index.js
const Fish = require('$share/js/Fish')
const fish = new Fish()
document.getElementById('app').textContent = fish.print()

Run zoo of dev see the effect:
<img width="150px" src="https://user-images.githubusercontent.com/17001245/148770309-3fe66464-6b1e-4780-948b-6873ee4cab5e.png">

After modifying shop , the same effect will appear.

That is to say, the things in the share folder, zoo and shop can be shared, all that needs to be done is to add a new webpack alias ! 🎉

🤔Thinking - Why workspaces for collection projects, can't we use the traditional way?

Traditional way:

1. All sub-projects are grouped into one project. The difference from the above is that package.json only one copy of 061dd2b0069ef5. In the root directory, all npm packages in the project are installed in the root directory. In package.json in the root directory, define the develop and deploy sub-projects;
2. All sub-projects are grouped into one project. The difference from the above is that although the root directory and each subpackage each have a copy of package.json , the basic build tools are installed in the root directory, such as webpack , webpack-cli , webpack-dev-server , html-webpack-plugin , webpack-merge Install, and install business-related npm packages into their respective subprojects;
3. All sub-projects are grouped into one project. The difference from the above is that each subpackage has a copy of package.json , and the root directory does not have package.json ;

Method 1 - Disadvantages:

• confusion of commands;
• Unable to deal with the problem of npm package conflicts between sub-projects; (for example, project A wants to use webpack4, project B wants to use webpack5; or project A wants to use Vue2, and project B wants to use Vue3)

Method 2 - Disadvantages:

• If subprojects have the same package, you have to repeat the installation in each subproject;
• It is also impossible to deal with the problem of npm package conflicts between sub-projects; (for example, project A wants to use webpack4, and project B wants to use webpack5)
• If you want to remove the B project one day, the cost is very high;

Method 3 - Disadvantages:

• If subprojects have the same package, you have to repeat the installation in each subproject;

Then use workspaces to solve all the above problems!

In addition, for existing projects, such as the project I took over this year, one is Web and the other is Wap, and then I found that because they belong to the same business, there is a lot of code that can be reused, and because only It only involves these two projects, and making the public code into an npm package is a bit tricky. Therefore, the copy and paste mode has been used in the past. This is obviously very inefficient. In addition, the mock service is also a separate set of word projects, but the data of most interfaces can be shared, but the url prefix is different. The most outrageous thing is that hundreds of bank icons are exactly the same. So, I'm going to combine the two into one project. workspaces is a solution with the least amount of changes to the original project.

How to deploy separately?

We want to deploy only project zoo on the build machine, what should we do?

1. Install dependencies

npm install --production --workspace=zoo 

In this case, only the dependencies under the zoo project will be installed on the build machine.

2. Build

npm run prod --workspace=zoo 

In this case, the build is successful!

avoid pit

The workspaces of npm actually have hidden pits, so I will also list them.

Pit 1: The pit of the default mode of npm install

Since npm v7, install will install the package declared by peerDependencies New projects may have little impact, however, if you are retrofitting an existing project. Because the unified management method is used, the lock files in the sub-projects are generally deleted, and unified lock management is used in the root directory. Then, when you do that, things can happen that suck.
webpack4 is used in my subproject, and then our build-related tools (webpack, babel, postcss, eslint, stylint, etc.) will be packaged into the base package. There is a package in the dependencies of these packages, which is written like this package.json

"peerDependencies": {
  "webpack": "^5.1.0"
},

Then, in the root directory npm install , and then run the sub-project to find that the project cannot run. The reason is that the project actually installed the version webpack5

solution

• Scenario 1: In the sub-project package.json display statement with the webpack version;
• Option 2: Go to github and discuss with the author to repair the dependent package. If his package is compatible with both webpack4 and webpack5 , it should be written, and the statement should be changed to: "webpack": "^4.0.0 || ^5.0.0"
• Scenario 3: npm install --legacy-peer-deps

Personally, I really think this is the npm author kicked by a donkey in the head. For yarn or pnpm, their workspaces will not use this default installation mode of peerDependencies
The author originally thought, because if the developer of the npm package declared peerDependencies , if we did not install the matching version of the package during use, the project may not be able to run. For the convenience of use, he adopted the default installation mode.
However, this approach will cause those peerDependencies do not conform to the writing specification, and there will be problems with the use of the project. Moreover, even if the authors of the new packages start to pay attention to the writing specifications, they cannot deal with the old packages that have been released. It is impossible to recycle them all, and then re-release them one by one!

Pit 2: Small version package conflict

This is actually caused by personal carelessness.

For example: zoo uses the command npm i -S @vue2.2.1 import vue, and shop uses the command npm i -S @vue2.2.2 import vue. So, will the project have two versions of vue ? Will not.
The reason we can see zoo under the package.json project:

"dependencies": {
  "html-webpack-plugin": "^5.5.0",
  "vue": "^2.2.1",
  "webpack": "^5.65.0",
  "webpack-cli": "^4.9.1",
  "webpack-dev-server": "^4.7.2",
  "webpack-merge": "^5.8.0"
}

It dawned on me.

solution

• Option 1: In fact, remove ^ ;
• Option 2: We can use npm i --save-exact vue@2.2.1 --workspace=zoo

Summarize

In this article, workspaces for multi-package management and multi-project management, which reflects the power of workspaces . Because the projects I am personally responsible for have always been managed by npm, there are unknown risks in migrating to yarn or pnpm, and I have also tried because some old packages, yarn2 and pnpm, cannot run. For new projects, I also recommend yarn2 or pnpm for management, they are more powerful than npm.

This original text comes from personal github blog , if you think a good
<(￣▽￣)/

The source code of multi-package management and multi-project management in this article are:

• Multi-package management
• Multiple Project Management

Interested students can download and study by themselves.