Skip to content

Instantly share code, notes, and snippets.

@lupomontero
Created June 20, 2018 03:47
Show Gist options
  • Save lupomontero/0f536d4d2f4f92acfa87f4becd09b2b6 to your computer and use it in GitHub Desktop.
Save lupomontero/0f536d4d2f4f92acfa87f4becd09b2b6 to your computer and use it in GitHub Desktop.
Process promise-based tasks in batches, with a delay between batches
const splitArrayIntoBatches = (arr, limit) => arr.reduce((memo, item) => {
if (memo.length && memo[memo.length - 1].length < limit) {
memo[memo.length - 1].push(item);
return memo;
}
return [...memo, [item]];
}, []);
const throttled = (tasks, concurrency, interval = 0) => {
const processBatches = (batches, prevResults = []) => {
if (!batches.length) {
return Promise.resolve(prevResults);
}
return Promise.all(batches[0].map(fn => fn())).then((batchResults) => {
const results = [...prevResults, ...batchResults];
return (batches.length <= 1)
? results
: new Promise((resolve, reject) => setTimeout(
() => processBatches(batches.slice(1), results).then(resolve, reject),
interval,
));
});
};
return processBatches(splitArrayIntoBatches(tasks, concurrency));
};
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment