npm and yarn installation packages are inconsistent

problem

Problem Description

Run a CRA project, use npm and yarn to install the package, and found that the package installed by npm has @babel/plugin-proposal-optional-chaining , but the package installed by yarn does not have @babel/plugin-proposal-optional-chaining . After the local npm installs the package, it starts normally, but the yarn used in the production environment causes the build to fail.

reason

The yarn.lock file generated by yarn install installation and the package-lock.json file generated by npm install have a long time difference, which causes the package version of yarn.lock to be lower than the package version of package-lock.json. Because ˆx.x.x is installed at different times, the generated lock file package version is inconsistent.

The version of yarn installation @babel/preset-env has: "7.5.5", "^7.4.5", the actual installed version is "7.5.5".

npm installs @babel/preset-env version: "7.9.0", "^7.4.5", the actual installed version is "7.9.0" and "7.11.5". In the "7.8.3" version, the @babel/plugin-proposal-optional-chaining plug-in was relied on for the first time.

Solution: If you execute yarn upgeade yark.lock file will be updated and the package version will be updated.

yarn upgrade [package]
This command will update the package to the latest version that matches the version set in package.json and rebuild yarn.lock. This is similar to npm update.

In actual projects, the use of npm and yarn should be avoided, which will cause inconsistent versions and cause a series of problems.

expand

package.json

package.json is mainly used to record information fields such as dependent package name, version, and running instructions. The packege.json file allows npm and yarn to track the dependencies of the project, but the version number is uncertain.

The dependencies field specifies the modules that the project depends on;
devDependencies specify the modules required for project development;
They all point to an object, and each member of the object is composed of the module name and the corresponding version, and represents the dependent module and its version range.

Version limitation methods [semantic version], there are mainly the following:

• Designated version: For example, 1.2.2, follow the format of "major version, minor version, minor version", and only install the specified version during installation.
• Tilde (tilde) + specified version: For example, ~1.2.2, which means that the latest version of 1.2.x (not lower than 1.2.2) is installed, but 1.3.x is not installed, which means that the major version number and The minor version number.
• Caret + specified version: For example ˆ1.2.2, it means that the latest version of 1.xx is installed (not lower than 1.2.2), but 2.xx is not installed, that is, the major version number is not changed during installation. It should be noted that if the major version number is 0, the behavior of the caret is the same as the tilde. This is because it is in the development stage at this time, even if the minor version number changes, it may also bring about program incompatibility.
• latest: Install the latest version.

When we use, for example, npm install package -save install a dependency package, the version is in the form of a caret. In this way, the "minor version" and "minor version" will pull the latest when reinstalling the dependent package npm install. Generally, if the main version remains unchanged, no core function changes will be brought about, and the API should be compatible with the old version.

Lock the dependency tree version

Since the version of the dependency tree module is uncertain every time it is reinstalled, a corresponding version locking mechanism appears.

npm-shrinkwrap

Before npm5, it can be achieved through npmshrinkwrap. By running npm shrinkwrap, a npm-shrinkwrap.json file will be generated in the current directory. It is a large list of each dependency listed in package.json, the specific version that should be installed, the location of the module (URI), and verification A hash of module integrity, a list of packages it needs, and a list of dependencies.
When running npm install, npm-shrinkwrap.json will be used first for installation, and package.json will be used for installation if not.

package-lock.json

After the npm5 version, running npm intall will generate a new file package-lock.json.
When there is a package-lock.json file in the project, when the project dependencies are installed, the file will be used as the main basis to parse and install the specified version dependency package, instead of using package.json to parse and install the module. Because package-lock specifies the version, location, and integrity hash for each module and each of its dependencies, it creates the same installation every time. It doesn't matter what device you use or installing it in the future, it should give you the same result every time.

yarn.lock

The main goal of the emergence of yarn is to solve the uncertainty of npm installation caused by semantic version control.
Each yarn installation will generate a yarn.lock file similar to package-lock.json, which is created by default. In addition to general information, the yarn.lock file also contains a checksum of the content to be installed to ensure that the versions of the libraries used are the same.

the Yarn major optimization

• Parallel installation: Whether npm or yarn performs package installation, they will perform a series of tasks. Npm executes each package according to the queue, which means that the installation of the current package must be completed before the subsequent installation can continue. And yarn executes all tasks synchronously, which improves performance.
• Offline mode: If a software package has been installed before, it will be retrieved from the cache when it is installed again with yarn, so there is no need to download it from the network like npm.
• Unified installation version: In order to prevent different versions from being pulled, yarn has a lock file that records the version number of the installed module. Every time a new module is added, yarn will create (or update) the yarn.lock file. This ensures that the same module version is used every time the same project dependency is pulled.
• Better semantics: yarn has changed the names of some npm commands, such as yarn add/remove, which is clearer than the original install/uninstall of npm.

Installation process of dependency tree

1. Execute the preinstall of the project itself
   If a preinstall hook is defined, it will be executed at this time.
2. Determine the first-level dependencies
   The first thing you need to do is to determine the first-level dependencies in the project, that is, the modules directly specified in the dependencies and devDependencies attributes (assuming that the npm install parameter is not added at this time).
   The project itself is the root node of the entire dependency tree. Each first-level dependent module is a subtree below the root node. npm will start multiple processes starting from each first-level dependent module to gradually find deeper nodes.

3. Download module
   Downloading modules is a recursive process, divided into the following steps:

   • Get module version information. Before downloading a module, you must first determine its version, because package.json is often a semantic version.
     If there is the module information in the version description file (npm-shrinkwrap.json or package-lock.json), you can use it directly, if not, get it from the warehouse. If the version of a package in package.json is ^1.1.0, npm will go to the warehouse to obtain the latest version that conforms to the 1.xx format.
   • Get the module entity. In the previous step, the module's compressed package address (resolved field) will be obtained. Npm will use this address to check the local cache. If it is in the cache, it will be taken directly, and if it is not, it will be downloaded from the warehouse.
   • Find the module dependencies, if there are dependencies, go back to step 1, if not, stop.
4. Flat module (dedupe).
   The last step obtained is a complete dependency tree, which may contain a large number of duplicate modules. For example, module A depends on loadsh, and module B also depends on lodash. Before npm3, the installation will be strictly in accordance with the structure of the dependency tree, which will cause module redundancy. Yarn and from npm5 have added a dedupe process by default, which will traverse all nodes and place modules one by one under the root node, which is the first layer of node-modules. When a duplicate module is found, it is discarded.

repeat module
It means that the module names are the same and semantically compatible. Each semantic version corresponds to a section of the allowable range of versions. If the allowable ranges of the two modules have an intersection, then a compatible version can be obtained without having the same version numbers, which can make more redundant modules in the dedupe process Was removed.

5. Install the module
   This step will update the node_modules in the project and execute the life cycle functions in the module (in the order of preinstall, install, and postinstall).
6. Execute the project's own life cycle
   If the hook is defined in the current npm project, it will be executed at this time (in the order of install, postinstall, prepublish, prepare)

Install dependent instances

Plug-ins htmlparser2@^3.10.1 and dom-serializer@^0.2.2 both use the entities dependency package, but the versions used are different, and we install a version of the entities package ourselves. details as follows:

--htmlparser2@^3.10.1
  |--entities@^1.1.1

--dom-serializer@^0.2.2
  |--entities@^2.0.0

--entities@^2.1.0

After installing through npm install, the generated package-lock.json file content and its node_modules directory structure:

It can be found:

The dependency package entities@^2.0.0 of dom-serializer@^0.2.2 and the entities@^2.1.0 installed by ourselves are actually installed as entities@^2.2.0 and placed in the first layer of node_modules. Because the semver range of the two versions is the same, they are traversed first, and all will be merged and installed on the first layer;
The dependency package entities@^1.1.1 of htmlparser2@^3.10.1 is actually installed in the node_modules of the dom-serializer package, and is consistent with the description structure of package-lock.json.
After installing through yarn, the generated yarn.lock file content and its node_modules directory structure:

It can be found that the difference from npm install is:

All dependency descriptions in yarn.lock are flat, that is, there is no nested relationship of dependency descriptions;
In yarn.lock, dependent packages with the same name and different version numbers will be merged if the semver range is the same, otherwise, there will be multiple version descriptions.

Reference: npm/yarn lock is really fragrant