Promise concurrency control

problem

It is required to write a method to control the number of concurrent Promises, as follows:

promiseConcurrencyLimit(limit, array, iteratorFn)

limit is performed at the same time the number of promise, array array of parameters, iteratorFn asynchronous operation performed in each of promise.

background

In development, post logic needs to be executed after multiple promises are processed, usually Promise.all :

Primise.all([p1, p2, p3]).then((res) => ...)

But there is a problem, because the promise will be executed immediately after it is created, that is to promise.all , the multiple promise instances passed in to 061260da998f96 have already been executed when they are created. If the asynchronous operations performed in these instances are all http Request, then n http requests will be issued in an instant, which is obviously unreasonable; a more reasonable way is to Promise.all , and only allow limit asynchronous operations to be executed at the same time.

Ideas & Realization

As mentioned in the background, promises will be executed immediately after creation, so the core of controlling concurrency lies in controlling the generation of promise instances. At first, only generate limit instances of promise, and then wait for the state of these promises to change. As long as the state of one of the promise instances changes, another promise instance will be created immediately... and the loop will continue until all promises are created and executed.

There are many libraries on npm that implement this function. I personally think that the tiny-async-pool library is better because it directly uses the native Promise to achieve this function, while most other libraries reimplement the promise. The core code is as follows:

async function asyncPool(poolLimit, array, iteratorFn) {
  const ret = []; // 用于存放所有的promise实例
  const executing = []; // 用于存放目前正在执行的promise
  for (const item of array) {
    const p = Promise.resolve(iteratorFn(item)); // 防止回调函数返回的不是promise，使用Promise.resolve进行包裹
    ret.push(p);
    if (poolLimit <= array.length) {
      // then回调中，当这个promise状态变为fulfilled后，将其从正在执行的promise列表executing中删除
      const e = p.then(() => executing.splice(executing.indexOf(e), 1));
      executing.push(e);
      if (executing.length >= poolLimit) {
        // 一旦正在执行的promise列表数量等于限制数，就使用Promise.race等待某一个promise状态发生变更，
        // 状态变更后，就会执行上面then的回调，将该promise从executing中删除，
        // 然后再进入到下一次for循环，生成新的promise进行补充
        await Promise.race(executing);
      }
    }
  }
  return Promise.all(ret);
}

The test code is as follows:

const timeout = (i) => {
  console.log('开始', i);
  return new Promise((resolve) => setTimeout(() => {
    resolve(i);
    console.log('结束', i);
  }, i));
};

(async () => {
    const res = await asyncPool(2, [1000, 5000, 3000, 2000], timeout);
    console.log(res);
  })();

The core idea of the code is:

1. First initialize the limit promise instances and put them in the executing array
2. Use Promise.race wait for the execution results of limit
3. Once the status of a certain promise changes, executing , and then execute the loop to generate a new promise, and put it in executing
4. Repeat steps 2 and 3 until all promises have been executed
5. Finally, use Promise.all to return the execution results of all promise instances