crash-test.ts

//////////////////
// RUN THIS FIRST:  npm install [email protected]
//
// The example below crashes NodeJS (it runs out of memory)

// operator source: https://github.com/vitaly-t/iter-ops/blob/wait-cache/src/ops/async/wait-cache.ts
//////////////////

import {map, pipeAsync, waitCache} from 'iter-ops';

function* gen(n) {
    while (n > 0) {
        yield n--;
    }
}

const i = pipeAsync(gen(1_000_000), map(a => Promise.resolve(a)), waitCache(100));

(async function () {
    const start = Date.now();

    for await(const a of i) {
        // console.log(a);
    }

    console.log('duration:', Date.now() - start);
})();

Note it uses beta of iter-ops package: npm install [email protected], with the operator source here.

If we reduce cache size to 10, for example, it works correctly, though not very performant. I cannot figure out what is so heavy in this algorithm that causes NodeJS to crash.

A bit more tests, and I'm beginning to see that Promise.race is the culprit, it is godawful slow, and eats up all memory.

There are some outstanding issues about it.