🚀Turborepo: A surge of 3.8k Stars in the month of its release. Don't you plan to try this super new Monorepo solution?

1. It’s 2022, the best way to expand your codebase 🚀🚀

1.1 What is a monorepo?

Monorepo is a project management method. Before Monorepo, the code repository management method was MultiRepo, that is, each project corresponds to a separate code repository. Each project is managed in a decentralized manner. This will lead to many drawbacks, such as the possibility of each project's Infrastructure and tool libraries are similar, the problem of repeated reuse of basic code, etc...

Monorepo is to put multiple projects in one warehouse. There are many articles about monorepo, and there are many tools that can build Monorepo, such as

tool	Briefly
Bit	Toolchain for Component Driven Development
Turborepo	High-performance build system for JavaScript and TypeScript codebases.
Rush	An extensible web single repository manager.
Nx	Next-generation build system with best-in-class monorepo support and powerful integrations.
Lerna	For managing projects with multiple packages

Today we bring you a high-performance build Monorepo tool, Turborepo

2. TurboRepo

2.1 What is Turborepo

TurboRepo is a monorepo high-performance build system for building Javascript and Typescript. Turborepo abstracts all annoying configurations, scripts and tools, reduces the complexity of project configuration, and allows us to focus on business development

We usually need to use appropriate tools to expand monorepo when building monorepo. Turborepo uses advanced construction technology and ideas to speed up development, and builds complex work that does not require configuration. We only need to use scripts and tools to quickly build your Monorepo. TurboRepo supports using Yarn Npm Pnpm,
We use Pnpm this time to show you the charm of Tuborepo!

2.2 Advantages of TurboRepo

2.2.1 Multitasking Parallel Processing

Turbo supports the parallel operation of multiple tasks. In the process of compiling and packaging multiple subpackages, turbo will process multiple tasks at the same time.

In a traditional monorepo task runner, like the lerna or yarn own built-in workspaces run command, each project's script life cycle scripts are organized in topology run in parallel (which is the mathematical term for "dependency-first" ordering) or individually and in parallel. According to the monorepo's dependency graph, CPU cores may be idle - which wastes valuable time and resources. []( https://turborepo.org/docs/glossary#topological-order )

What is topology?

Topological Order
It is a sort of topological sorting is a term of dependency precedence. If A depends on B, and B depends on C, then the topological order is C, B, A.

For example, a larger project is often divided into many sub-projects, and we call these sub-projects activities . In the whole project, some sub-projects (activities) can only be started after other related sub-projects are completed, that is, the start of a sub-project is a prerequisite for the end of all its pre-order sub-projects

In order to understand how powerful turbo is, the following figure compares the turbo vs lerna task execution timeline:

Turbo it can schedule tasks efficiently similar to waterfall which can execute multiple tasks asynchronously at the same time, while lerna can only execute one task at a time so Turbo has poor performance It goes without saying.

2.2.2 Faster incremental builds

If our project is too large, building multiple subpackages will cause a waste of time and performance. The caching mechanism in turborepo can help us remember the content of the build and skip the content that has already been calculated, optimizing the packaging efficiency.

2.2.3 Cloud Cache

Turbo enables faster builds with its remote caching feature that helps multiple people build remotely. Cloud caching.

2.2.4 Task pipeline

Use configuration files to define relationships between tasks, and let Turborepo optimize build content and timing.

2.2.5 Convention-Based Configuration

Reduce complexity through conventions, configure entire project dependencies with just a few lines of JSON, and execute the sequential structure of scripts.

2.2.6 Configuration files in the browser

Generate build profiles and import them into Chrome or Edge to see which tasks are taking the longest.

3 Turbo core concepts

3.1 Pipeline

Turborepo provides a way for developers to explicitly specify task relationships in a conventional manner.

1. New developers can check out the Turborepo pipeline and see how tasks are related.
2. turbo This explicit declaration can be used to perform optimizations and scheduled executions based on the rich availability of multi-core processors.

turbo In defining the task dependency graph of monorepo, we need to define it in the root directory turbo.json Execute various functions such as scheduling, output, cache dependency, packaging, etc.

turbo.json Located in the root directory of the turborepo project The next part of the actual combat will lead you to create a project from scratch

Each key in pipeline package.json points to our script execution command defined in pipeline can be accessed turbo run The name of the script that executed pipeline . You can specify its dependencies using the keys below it and some other cache -related options.

When we execute the turbo run *** command, turbo will follow our defined Pipelines
The various configurations of the commands in each of our packages package.json 中 对应的script execute scripts for orderly execution and cache output files.

 // turbo.json
{
  "$schema": "https://turborepo.org/schema.json",
  "pipeline": {
    "build": {
      // A package's `build` script depends on that package's
      // dependencies' and devDependencies'
      // `build` tasks  being completed first
      // (the `^` symbol signifies `upstream`).
      "dependsOn": ["^build"],
      // note: output globs are relative to each package's `package.json`
      // (and not the monorepo root)
      "outputs": [".next/**"]
    },
    "test": {
      // A package's `test` script depends on that package's
      // own `build` script being completed first.
      "dependsOn": ["build"],
      "outputs": [],
      // A package's `test` script should only be rerun when
      // either a `.tsx` or `.ts` file has changed in `src` or `test` folders.
      "inputs": ["src/**/*.tsx", "src/**/*.ts", "test/**/*.ts", "test/**/*.tsx"]
    },
    "lint": {
      // A package's `lint` script has no dependencies and
      // can be run whenever. It also has no filesystem outputs.
      "outputs": []
    },
    "deploy": {
      // A package's `deploy` script depends on the `build`,
      // `test`, and `lint` scripts of the same package
      // being completed. It also has no filesystem outputs.
      "dependsOn": ["build", "test", "lint"],
      "outputs": []
    }
  }
}

<br/>Next, let's analyze what the key in each object is used for to help us better understand pipeline

3.1.1 DependsOn

For example, we currently have three subpackages, two tool packages, and one playground for testing the other two packages.

 "pipeline": {
    "build": {
      "dependsOn": ["^build"]
    }
  }

3.1.2 General dependencies

If the execution of a task only depends on its own package of other tasks, then you can put the dependent tasks in the dependsOn array

 {
    "turbo": {
        "pipeline": {
            "deploy": {
                "dependsOn": ["build", "test", "lint"]           
            } 
        }    
    }
}

3.1.3 Topological dependencies

You can use the ^ symbol to explicitly declare that the task has topology dependencies, and the dependent packages can start executing their own tasks after the corresponding tasks are executed.

 {
    "turbo": {
        "pipeline": {
            "build": {
                "dependsOn": ["^build"],           
            }
        }    
    }
}

Because playground depends on @relaxed/utils and @relaxed/hook, the build of our current playground subpackage has dependencies. According to the build's dependsOn configuration, the build command of the dependencies will be executed first, that is, @relaxed/utils and @relaxed /hook build command, the playground build command will be executed after the dependencies are executed.
If we don't add "dependsOn": ["^build"] ‘^’ the array, then it means that we only need to execute our own build command.

dependsOn表示当前命令所依赖的命令， ^表示dependencies devDependencies执行完build , execute build

3.1.4 Null dependencies

If a task's dependsOn is [] or does not declare this attribute, it indicates that the task can be executed at any time

3.1.5 Output

outputs represents the file cache directory for command execution output

The default value is ["dist/**", "build/**"]

We can also tell turbo that the task is a side effect by passing an empty array so that we don't enter any files

 "pipeline": {
  "build": {
    // "Cache all files emitted to package's dist/** or .next
    // directories by a `build` task"
    "outputs": ["dist/**", ".next/**"],
    "dependsOn": ["^build"]
  },
 }

3.1.6 Cache

cache ， dev命令的时候watch模式，所以我们一般在项目启动模式下不需要开启turbo Cache mechanism

 {
  "$schema": "https://turborepo.org/schema.json",
  "pipeline": {
    "dev": {
      "cache": false
    }
  }
}

3.1.7 Input

The default is [] . Tells turbo the set of files to consider when determining if a task-specific package has changed. Setting this to the file input address will cause the task to be rerun only if the file matching the required config input in these real subpackages changes. This is helpful if, for example, you want to skip running tests unless the source files have changed.

Specifying [] means that the task is rerun if any file changes.

 {
  "$schema": "https://turborepo.org/schema.json",
  "pipeline": {
   "test": {
      // A package's `test` task should only be rerun when
      // either a `.tsx` or `.ts` file has changed.
      "inputs": ["src/**/*.tsx", "src/**/*.ts", "test/**/*.ts"]
    }
  }
}

3.1.8 OutputMode

outputMode represents the output pattern type is a string

type
full
new-only
hash-only
none

full is also the default value representing the entire output of the display task

hash-only Show only the calculated task hash

new-only Shows the full output of cache misses and the calculated hash of cache hits. It means to return the log with hash and if there is a missed sub-package cache or a packaging error that causes the cache miss to be packaged again, the complete task output log of the sub-package of the last cache miss will be output.

none Use "none" to hide task output. This means that the log of our topological order and packaged input will not be printed in the console, but the build command will still be executed correctly

 {
  "$schema": "https://turborepo.org/schema.json",
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputMode": "new-only"
    },
    "test": {
      "outputs": [],
      "dependsOn": ["build"],
    },
  }
}

3.2 Filtering Packages

The second core concept in turbo, filtering packages, allows you to target tasks only to the packages you want to operate on. Turborepo supports a filter command similar to pnpm in --filter but it is different from the traditional pnpm --filter command. We need to add an equal sign after --filter to determine the need to separate the filtered packets A directive that allows you to select packages that will act as "entry points" into the monorepo's package/task graph, you can filter your changes by package name, package directory, whether the package contains dependencies/dependencies, and git history project.
Syntax pnpm build --filter=@relaxed/utils

There are still many scenarios for filter syntax. I will not show the specific visible turbo filter packet syntax here.

3.3 Caching

When Turborepo checks file content changes, it will generate Hash based on the content for comparison
turbo It can cache the logs of issued files and previously run commands. It does this to save a lot of time by skipping work that has already been done.
For example, when we execute the build command of three subpackages, we only need to execute turbo run build

During each packaging process, turbo will generate log files in each build subpackage, cache the build content in the next packaging, and skip the calculated content to determine whether it needs to be rebuilt

When we enter the build command, we first enter the pipeline to see if the cache is turned off, and then use the turbo log file in the current subpackage to hash and compare whether the current code has changed. If there is no change, then skip the build.

Force overwrite cache

Conversely, if you want to force turbo to re-execute a previously cached task, add the --force flag:

 #在所有包和应用中运行`Build`N脚本
#忽略缓存命中。
turbo run build --force

3.4 Remote Caching

Turborepo速度的一个主要关键🔑 是它既懒惰又高效——它做的工作量尽可能少，并且它试图从不重做以前已经完成的工作。 This is a summary of the turbo remote cache on the official website. Under normal circumstances, when we use turbo, we can only cache our tasks on the local system during the construction process. turbo supports a shared cache for multi-person development mode

Developer teams and/or continuous integration (CI) systems use remote caching to share build output. If your build is reproducible, the output from one machine can be safely reused on another machine, which can significantly speed up the build.

If you want to link your local turborepo to a remote cache, first authenticate to the Turborepo CLI with your Vercel account:

 npx turbo login

Next, you can link your turborepo to the remote cache by running:

 npx turbo link

Once enabled, make some changes to the currently cached package or application and use turbo run . Your cache artifacts will now be stored locally and in your remote cache.

To verify, delete the local Turborepo cache with the following command:

 rm -rf ./node_modules/.cache/turbo

Then run the same build again. If all is well, turbo then the task should not be executed locally, but instead download logs and artifacts from remote cache and replay them to you.

4. Use of the command line

4.1 Option syntax

options Option syntax can be passed in different ways via turbo

• Option to pass value

 --<option>=<value>
  // like
  pnpm build --filter=vue-devui
  
  pnpm build --filter=@relaxed/hook

4.2 Global parameters (common commands)

4.2.1 --continue

Defaults to false . This flag tells turbo whether to continue execution on an error (i.e. a non-zero exit code for the task). By default, specifying the --parallel flag will automatically set the --continue to, true unless explicitly set to false . When --continue is true , turbo will exit with the highest exit code value encountered during execution.

 turbo run build --continue

4.2.2 --parallel

Default false . The script runs commands in parallel and ignores the dependency graph. This is useful for developing with live reloading. For example, when we start the vite project, we need to ignore other possible dependencies dependsOn dependencies

 "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["dist/**", "build/**"],
      "outputMode": "new-only"
    },
    "lint": {
      "outputs": []
    },
    "dev": {
      "cache": false,
      //
      "dependsOn": ["^build"]
    }
  }
}

Since we set the dependson build command dependency priority in the pipeline, we can specify the --parallel parallel execution and prevent the default dependency build instruction

 turbo run lint --parallel --no-cache
turbo run dev --parallel --no-cache

4.2.3 --filter

Specify a combination of package/application, directory, and git commit as the entry point for execution.

Multiple filters can be combined to select different target sets. In addition, filters can also exclude targets. Targets that match any filter and are not explicitly excluded will be in the final entry point selection.

For more details on --filter flags and filtering, see the dedicated page in our documentation

 turbo run build --filter=my-pkg
turbo run test --filter=...^@scope/my-lib
turbo run build --filter=./apps/* --filter=!./apps/admin

4.2.4 --force

Ignore existing cache artifacts and force re-execution of all tasks (overwrite overlapping artifacts)

 turbo run build --force

4.2.5 --no-cache

Default false . Do not cache the results of tasks. next dev This is useful for watch commands like or react-scripts start .

 turbo run build --no-cache
turbo run dev --parallel --no-cache

4.2.6 --only

Default false . Limit execution to the specified tasks in the specified package. This is very similar to how lerna or pnpm run tasks by default. If we specify that the build command needs to be executed before dependencies but if we set --only will be excluded by default build command

Given this pipeline turbo.json :

 {
  "$schema": "https://turborepo.org/schema.json",
  "pipeline": {
    "build": {
      "dependsOn": [
        "^build"
      ]
    },
    "test": {
      "dependsOn": [
        "^build"
      ]
    }
  }
}

 turbo run test --only

Only the test task in each package will be executed. It won't build .

This article introduces several commonly used turbo commands, of course turbo provides many commands used in the command line, you can check the details

command line reference

5. Demo combat ✨✨

quick start

We can transform the existing monorepo or directly create the turbo project,

Another major feature of turbo is that it is also very easy to modify your existing monorepo, just install the turbo dependency root directory and add turbo.json and everything is under control

This time, we will experience using the turbo command line to create a project. The demo only shows the combined use of turbo and monorepo, the specific construction method, and the complete project can view the github template

We enter on the command line

 npx create-turbo@latest

Create our project name

 >>> TURBOREPO

>>> Welcome to Turborepo! Let's get you set up with a new codebase.

? Where would you like to create your turborepo? (./my-turborepo)

We recommend building with Pnpm

 ? Which package manager do you want to use?
  npm
> pnpm
  yarn

turbo will automatically create a corresponding project for us according to the package manager we choose and then we enter the project

For a better experience, we use vite to display more intuitively

We enter npm init vue@3 in the root directory command line to create a vue project named playground

Then add a playground field in pnpm-workspace.yaml to add the playground package to the management of pnpm monorepo

 packages:
  - 'packages/*'
  - 'playground'

Then in the packages, create a new @relaxed/hook , @relaxed/utils , @relaxed/tsconfig The first two folders are used as test tool libraries and the rest are used as our tsconfig shared configuration library

• @relaxed/utils

Create a new index.ts in the root directory

 export function Accumulation(...value: any[]) {
  return value.reduce((t, v) => t + v, 0)
}

export function Multiplication(...value: any[]) {
  return value.reduce((t, v) => t * v, 1)
}

Then we add tsc in package.json in order to test we use tsc to package the new build command

• @relaxed/hook the same

Create a new index.ts in the root directory

 import { ref } from 'vue'
export default function useBoolean(initValue = false) {
  const bool = ref(initValue)

  function setBool(value: boolean) {
    bool.value = value
  }
  function setTrue() {
    setBool(true)
  }
  function setFalse() {
    setBool(false)
  }
  function toggle() {
    setBool(!bool.value)
  }
  return {
    bool,
    setBool,
    setTrue,
    setFalse,
    toggle
  } 
}

• @relaxed/tsconfig

Create a new base.json in the tsconfig directory

 {
  "$schema": "https://json.schemastore.org/tsconfig",
  "display": "Default",
  "compilerOptions": {
    "composite": false,
    "declaration": true,
    "declarationMap": true,
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "inlineSources": false,
    "isolatedModules": true,
    "moduleResolution": "node",
    "noUnusedLocals": false,
    "noUnusedParameters": false,
    "preserveWatchOutput": true,
    "skipLibCheck": true,
    "strict": true
  },
  "exclude": ["node_modules"]
}

package.json

 {
  "name": "@relaxed/tsconfig",
  "version": "0.0.0",
  "private": true,
  "files": [
    "base.json"
  ]
}

First we need to add the tsconfig shared module to the two tool libraries

It can be added directly in the package.json of @relaxed/hook @relaxed/utils

 "devDependencies": {
    "@relaxed/tsconfig": "workspace:*",
    "vue": "^3.2.37"
  }

or

 pnpm add @relaxed/tsconfig --filter=@relaxed/hook

pnpm add @relaxed/tsconfig --filter=@relaxed/utils

Then how do we use it in the playground?

The first method: Add dependencies of two tool subpackages to the package in the playground

 "dependencies": {
    "@relaxed/hook": "workspace:1.0.0",
    "@relaxed/utils": "workspace:*"
  },

Then we execute
pnpm install

The second method: We install directly through the command

 pnpm add @relaxed/utils  @relaxed/hook --filter=playground

* The current working directory represents the latest version and then we execute pnpm dev turbo first looks for the dev command in the pipeline and then according to whether you have configured the execution tasks in the pipeline, and finally executes the sub-package dev command, it means to execute the playground dev command in ---733c22cd0ac2032562ca67f563b5657f---

 <script setup lang="ts">
// This starter template is using Vue 3 <script setup> SFCs
// Check out https://vuejs.org/api/sfc-script-setup.html#script-setup
import HelloWorld from './components/HelloWorld.vue'
import { Accumulation, Multiplication } from '@relaxed/utils'
import useBoolean from '@relaxed/hook'
const { bool, setBool, setTrue, setFalse, toggle } = useBoolean(false)
</script>

<template>
  <div>
    <a href="https://vitejs.dev" target="_blank">
      <img src="/vite.svg" class="logo" alt="Vite logo" />
    </a>
    <a href="https://vuejs.org/" target="_blank">
      <img src="./assets/vue.svg" class="logo vue" alt="Vue logo" />
    </a>
    <div style="margin: 10px 0">
      <d-tag type="warning">
        {{ bool }}
      </d-tag>
    </div>
    <d-button type="tertiary" @click="setFalse"> {{ bool }} </d-button>
    <d-button type="primary" @click="setTrue"> {{ bool }} </d-button>
    <d-button type="info" @click="toggle"> Toogle </d-button>
  </div>
  <HelloWorld msg="Vite + Vue" />
</template>

Then when it comes to the build link, we execute pnpm build. At this time, we need the build settings in the pipeline in turbo.json

 "dependsOn": ["^build"],

Only by adding ^ so that we can first execute the build command of the three sub-packages that depend on it. Since our playground depends on other sub-packages, we must add ^ otherwise turbo will block build because we have sub-package dependencies,
So Turborepo can help us manage the logic between projects in an orderly manner

4. Summary

Turborepo can help us better manage the Monorepo project. With its excellent task scheduling management and incremental build cache, etc., it can help us solve some of the current problems in monorepo in the future, thereby improving our development efficiency and improving the whole The performance of the project in terms of builds, etc.

5. Reference

• Turborepo
• Pnpm