How to efficiently and compactly limit concurrency in JavaScript's Promise.all. A simplified and convincing demo for https://stackoverflow.com/a/51020535/ that you can run on TypeScript Playground

demo.ts

const sleep = (t: number) => new Promise((rs) => setTimeout(rs, t));

/**
 * This function will return an array of the same length as the input,
 * but not in the same order. It's straightforward to extend it to
 * preserve order, but I left it like this because this way the timing
 * is clearly visible. This helped me confirm that it works.
 */
async function processArray(input: number[], numWorkers: number) {
  const ret: { date: number; description: string }[] = [];
  const iterator = input.values();

  await Promise.all(
    Array.from(Array(numWorkers), async () => {
      for (const x of iterator) {
        const ms = x % 3 === 0 ? 1000 : 500;
        await sleep(ms);
        ret.push({
          date: Date.now(),
          description: `x=${x}, ${x % 3 === 0 ? "slow" : "fast"}`,
        });
      }
    })
  );

  return ret;
}

/**
 * Given an array of promise-generating functions, run them concurrently with a
 * maximum of `numWorkers` running at the same time.
 *
 * (This is useful for example for fetching lots of URLs and avoiding spamming
 * servers.)
 */
async function promiseAllLimited<T>(
  generators: (() => Promise<T>)[],
  numWorkers: number
): Promise<T[]> {
  // This has to be done upfront. We *CANNOT* do this in a loop because we want
  // all workers to pull from a single iterator. That's key to the magic.
  const iterator = generators.entries();

  // This is what actually gets returned. It'll grow as the workers add to it,
  // so potentially later elements will be added first and earlier ones filled
  // in later. JavaScript makes this kind of array-growth manageable.
  const ret: T[] = [];

  // Sentinel value telling other workers to bail. There's not an easy way to
  // tell the `Array.prototype.entries` iterator to finish "early" (other than
  // iterating over it, which might be memory-wasteful), so we resort to this
  // ugly hack.
  let done = false;

  // This Promise.all doesn't generate anything data-relevant to the input or
  // output. It's just coordinating the `numWorkers` async functions pulling at
  // an iterator of `generators` like individual strings of a twizzler. Each
  // string (each worker) is a promise but one that consumes potentially many of
  // the promises in `generators`.

  await Promise.all(
    Array.from(Array(numWorkers), async (_, numWorker) => {
      for (const [idx, f] of iterator) {
        if (done) {
          break;
        }
        try {
          ret[idx] = await f();
        } catch (e) {
          done = true;
          throw e;
        }
      }
    })
  );
  return ret;
}

(async function main() {
  {
    console.log("Initial demo");
    const numbers = Array.from(Array(10), (_, n) => 1 + n);
    const start = Date.now();

    const ret = await processArray(numbers, 2);

    for (const { date, description } of ret) {
      console.log(`- ${date - start} ms after start: ${description}`);
    }
  }

  {
    console.log("Utility function promiseAllLimited");

    const generators = Array.from(Array(10), (_, n) => async () => {
      const x = 1 + n;
      // the children promises can throw, no problem: other workers will stop.
      // Try setting this to 2
      if (x === 20) {
        throw new Error("OH THE AGONY!");
      }
      await sleep(x % 3 === 0 ? 1000 : 500);
      return {
        date: Date.now(),
        n: n,
        description: `x=${x}, ${x % 3 === 0 ? "slow" : "fast"}`,
      };
    });

    const secondStart = Date.now();
    try {
      const second = await promiseAllLimited(generators, 2);
      for (const o of second) {
        console.log(
          `* ${o.date - secondStart} ms after start: ${o.description}`
        );
      }
    } catch (e) {
      console.error("Exception encountered!", e);
    }
  }
})();

Iterates over numbers 1 through 10, sleeping 500 ms for each except those divisible by 3, when we sleep for 1000 ms ("slow" is divisible by 3, "fast" is not). Outputs the following:

Initial demo
- 503 ms after start: x=1, fast
- 504 ms after start: x=2, fast
- 1006 ms after start: x=4, fast
- 1505 ms after start: x=3, slow
- 1506 ms after start: x=5, fast
- 2009 ms after start: x=7, fast
- 2508 ms after start: x=6, slow
- 2508 ms after start: x=8, fast
- 3011 ms after start: x=10, fast
- 3510 ms after start: x=9, slow

which nicely shows that there are two workers, and as soon as one finishes, the other starts.

The reusable method promiseAllLimited returns the array in the same order as the input promise-generating functions. Here's the output from using that utility:

Utility function promiseAllLimited
* 504 ms after start: x=1, fast
* 504 ms after start: x=2, fast
* 1505 ms after start: x=3, slow
* 1005 ms after start: x=4, fast
* 1507 ms after start: x=5, fast
* 2506 ms after start: x=6, slow
* 2009 ms after start: x=7, fast
* 2509 ms after start: x=8, fast
* 3509 ms after start: x=9, slow
* 3011 ms after start: x=10, fast

(Maybe also see https://gist.github.com/fasiha/c04744393e0aafd07b0603593daacaa1)