50 lines of code serial Promise, how is the koa onion model originally implemented?

1 Introduction

Hello everyone, my name is . Welcome to follow my 161567fad7facd public , and recently organized the source code reading activity "1 month, 200+ people, read the source code together for 4 weeks" , interested can add me 161567 Participate, exchange and learn for a long time.

The previously written "Learning Source Code Overall Architecture Series" contains jQuery , underscore , lodash , vuex , sentry , axios , redux , koa , vue-devtools , vuex4 The two latest ones are:

Vue 3.2 is released. How did You Yuxi release Vue.js?

Those practical basic tool functions in the Vue3 source code that beginners can understand

Writing relatively difficult source code consumes my time and energy, and I didn't get many reading likes. In fact, it was quite a blow. From the perspective of reading volume and reader benefits, it cannot promote the author's continuous output of articles.

So change your mind and write some relatively easy-to-understand articles. is not as difficult as imagined. At least many of them can understand .

I wrote the koa source code article learn the overall architecture of the koa source code before, analyze the koa onion model principle and the co principle koa-compose , so this article describes from the source code of 061567fad7fbaa 50 lines.

koa-compose warehouse file involved in this article, the entire index.js file has less than 50 lines of code, and the test case test/test.js file 300 more than 6 lines, but it is worth learning.

Goethe once said: Reading a good book is talking to a noble person. The same is true: reading the source code can also be regarded as a way of learning and communicating with the author.

After reading this article, you will learn:

1. 熟悉 koa-compose 中间件源码、可以应对面试官相关问题
2. 学会使用测试用例调试源码
3. 学会 jest 部分用法

2. Environmental preparation

2.1 Clone the koa-compose project

This article warehouse address koa-compose-analysis , ask for a star ~

# 可以直接克隆我的仓库，我的仓库保留的 compose 仓库的 git 记录
git clone https://github.com/lxchuan12/koa-compose-analysis.git
cd koa-compose/compose
npm i

By the way: how do I keep the git record in the compose

# 在 github 上新建一个仓库 `koa-compose-analysis` 克隆下来
git clone https://github.com/lxchuan12/koa-compose-analysis.git
cd koa-compose-analysis
git subtree add --prefix=compose https://github.com/koajs/compose.git main
# 这样就把 compose 文件夹克隆到自己的 git 仓库了。且保留的 git 记录

For more information about git subtree , you can read this article Use Git Subtree to synchronize sub-projects between multiple Git projects in both directions, with a concise user manual

Then we look at how to debug the source code according to the test cases provided in the open source project.

2.2 Debug compose source code based on test cases

Open the project with VSCode (my version is 1.60 compose/package.json , find scripts and test commands.

// compose/package.json
{
    "name": "koa-compose",
    // debug （调试）
    "scripts": {
        "eslint": "standard --fix .",
        "test": "jest"
    },
}

scripts there should be debug or debugging words. Click debug (Debug) and select test .

Then the test case test/test.js file will be executed. The terminal output is shown in the figure below.

Then we debug the compose/test/test.js file.
We can hit a breakpoint on line package.json => srcipts => test enter debug mode.
As shown below.

Then press the button above to continue debugging. compose/index.js breakpoints in key places in the 061567fad7fe2d file, and debug and learn the source code with less effort.

more nodejs debugging related, please refer to the official document

By the way, explain in detail the next few debugging related buttons.

• 1. Continue (F5): After clicking, the code will be executed directly to the location of the next breakpoint. If there is no next breakpoint, it is considered that the code execution is completed.
• 2. Single step skip (F10): After clicking, it will jump to the next line of the current code to continue execution without entering the function.
• 3. Single step debugging (F11): Click to enter the internal debugging of the current function. For example, if you execute single step debugging compose compose function for debugging.
• 4. Step out (Shift + F11): Click to jump out of the currently debugged function, which corresponds to single step debugging.
• 5. Restart (Ctrl + Shift + F5): As the name implies.
• 6. Disconnect link (Shift + F5): As the name implies.

Next, we follow the test case to learn the source code.

3. Follow the test case to learn the source code

Share a test case tip: We can add only modification to the test case.

// 例如
it.only('should work', async () => {})

In this way, we can only execute the current test case, do not care about the others, and will not interfere with debugging.

3.1 Normal process

Open the compose/test/test.js file and see the first test case.

// compose/test/test.js
'use strict'

/* eslint-env jest */

const compose = require('..')
const assert = require('assert')

function wait (ms) {
  return new Promise((resolve) => setTimeout(resolve, ms || 1))
}
// 分组
describe('Koa Compose', function () {
  it.only('should work', async () => {
    const arr = []
    const stack = []

    stack.push(async (context, next) => {
      arr.push(1)
      await wait(1)
      await next()
      await wait(1)
      arr.push(6)
    })

    stack.push(async (context, next) => {
      arr.push(2)
      await wait(1)
      await next()
      await wait(1)
      arr.push(5)
    })

    stack.push(async (context, next) => {
      arr.push(3)
      await wait(1)
      await next()
      await wait(1)
      arr.push(4)
    })

    await compose(stack)({})
    // 最后输出数组是 [1,2,3,4,5,6]
    expect(arr).toEqual(expect.arrayContaining([1, 2, 3, 4, 5, 6]))
  })
}

After reading this test case, context is next and what is 061567fad80022.

In koa document has a very typical middleware gif map.

The compose functions added to the middleware array in the order of the gif

3.1.1 compose function

Simply put, the compose function mainly does two things.

• 1. Receive a parameter, the check parameter is an array, and each item in the check array is a function.
• 2. Return a function, this function receives two parameters, which are context and next , this function finally returns Promise .

/**
 * Compose `middleware` returning
 * a fully valid middleware comprised
 * of all those which are passed.
 *
 * @param {Array} middleware
 * @return {Function}
 * @api public
 */
function compose (middleware) {
  // 校验传入的参数是数组，校验数组中每一项是函数
  if (!Array.isArray(middleware)) throw new TypeError('Middleware stack must be an array!')
  for (const fn of middleware) {
    if (typeof fn !== 'function') throw new TypeError('Middleware must be composed of functions!')
  }

  /**
   * @param {Object} context
   * @return {Promise}
   * @api public
   */

  return function (context, next) {
    // last called middleware #
    let index = -1
    return dispatch(0)
    function dispatch(i){
      // 省略，下文讲述
    }
  }
}

Then we look at the dispatch function.

3.1.2 dispatch function

function dispatch (i) {
  // 一个函数中多次调用报错
  // await next()
  // await next()
  if (i <= index) return Promise.reject(new Error('next() called multiple times'))
  index = i
  // 取出数组里的 fn1, fn2, fn3...
  let fn = middleware[i]
  // 最后 相等，next 为 undefined
  if (i === middleware.length) fn = next
  // 直接返回 Promise.resolve()
  if (!fn) return Promise.resolve()
  try {
    return Promise.resolve(fn(context, dispatch.bind(null, i + 1)))
  } catch (err) {
    return Promise.reject(err)
  }
}

It is worth mentioning that: The bind function returns a new function. The first parameter is the this point in the function (if the function does not need to use this , it will generally be written as null ).
This sentence fn(context, dispatch.bind(null, i + 1) , i + 1 is for let fn = middleware[i] take the next function in middleware
That is, next is the function in the next middleware. Can also explain the above gif graph function execution order.
The final order of the array in the test case is [1,2,3,4,5,6] .

3.1.3 Simplify compose for easy understanding

After debugging by yourself, you will find that compose is similar to this structure after execution (omit try catch judgment).

// 这样就可能更好理解了。
// simpleKoaCompose
const [fn1, fn2, fn3] = stack;
const fnMiddleware = function(context){
    return Promise.resolve(
      fn1(context, function next(){
        return Promise.resolve(
          fn2(context, function next(){
              return Promise.resolve(
                  fn3(context, function next(){
                    return Promise.resolve();
                  })
              )
          })
        )
    })
  );
};

That is to say, koa-compose returns a Promise . Take the first function from the middleware (the incoming array) context and the first next function to execute.

The first next function also returns a Promise . Take the second function from the middleware (the incoming array), and pass in the context and the second next function to execute.

The second next function also returns a Promise , the third function is taken middleware (the incoming array) context and the third next function are passed in for execution.

The third...

And so on. next function 061567fad802b4 is called in the last middleware, it returns Promise.resolve . If not, the next function is not executed.
In this way, all middleware are connected in series. This is what we often call the onion model.


to be said to be very amazing, "It's still a great god to play" .

3.2 Error trap

it('should catch downstream errors', async () => {
  const arr = []
  const stack = []

  stack.push(async (ctx, next) => {
    arr.push(1)
    try {
      arr.push(6)
      await next()
      arr.push(7)
    } catch (err) {
      arr.push(2)
    }
    arr.push(3)
  })

  stack.push(async (ctx, next) => {
    arr.push(4)
    throw new Error()
  })

  await compose(stack)({})
  // 输出顺序 是 [ 1, 6, 4, 2, 3 ]
  expect(arr).toEqual([1, 6, 4, 2, 3])
})

I believe that if I understand the first test case and the compose function, it is also a better understanding of this test case. This part is actually the corresponding code here.

try {
    return Promise.resolve(fn(context, dispatch.bind(null, i + 1)))
} catch (err) {
  return Promise.reject(err)
}

3.3 The next function cannot be called multiple times

it('should throw if next() is called multiple times', () => {
  return compose([
    async (ctx, next) => {
      await next()
      await next()
    }
  ])({}).then(() => {
    throw new Error('boom')
  }, (err) => {
    assert(/multiple times/.test(err.message))
  })
})

This piece corresponds to:

index = -1
dispatch(0)
function dispatch (i) {
  if (i <= index) return Promise.reject(new Error('next() called multiple times'))
  index = i
}

After two calls, i and index are both 1 , so an error will be reported.

compose/test/test.js are a total of more than 300 lines in the 061567fad803b4 file, and there are many test cases that can be debugged according to the method in the article.

4. Summary

Although the koa-compose less than 50 lines, if it is the first time to look at the source code and debug the source code, it will still be difficult. There are mixed basic knowledge of higher-order functions, closures, Promise , bind and so on.

Through this article, we are familiar with the koa-compose by the 061567fad80407 middleware, learned part of the jest usage, and also learned how to use the ready-made test cases to debug the source code.

believes that after learning to debug the source code through test cases, you will feel that the source code is not as difficult as you imagined .

Open source projects generally have very comprehensive test cases. In addition to the convenience of learning source code and debugging source code, it can also give us inspiration: You can gradually introduce testing tools for your projects, such as jest .

In addition, reading the source code of open source projects is a better way for us to learn the design ideas and source code implementation of the industry's big cows.

After reading this article, I very much hope that I can practice debugging the source code to learn, easy to absorb and digest. In addition, if you have more energy, you can continue to read my koa-compose source code article: Learn the overall architecture of koa source code, analyze the principle of koa onion model and co principle