Skip to content

Instantly share code, notes, and snippets.

@ptdecker
Last active December 6, 2016 15:05
Show Gist options
  • Save ptdecker/1e30d6324e84ea03d276 to your computer and use it in GitHub Desktop.
Save ptdecker/1e30d6324e84ea03d276 to your computer and use it in GitHub Desktop.
Node.js Fully Concurrent Execution of an Async Function without Rate Limiting
/* Control Flow 2 - Fully Concurrent Executions of a Function (not rate limited)
*
* Based on:
* - http://book.mixu.net/node/ch7.html
* - https://github.com/mixu
*
* Characteristics:
*
* - Runs a number of operations concurrently
* - Starts all async operations near simultaneously (full concurrency)
* - No guarantee of order, only that all the operations have been completed
*
* Variations:
*
* - The way in which the result is collected (manual or via a
* “stashing” callback)
* - How error handling is done (via the first argument of the final function,
* manually in each subfunction, or via a dedicated, additional function)
* - Whether a final callback can be specified
*
* Tags: parallel, full-concurrency, no-concurrency-control
*/
/*
* A Simulated Asynchronous Function
*/
function async(arg, callback) {
var delay = Math.floor(1000 * Math.random());
console.log('Do something with "' + arg + '", and return ' + delay + 'ms later');
setTimeout(function() {
result = arg; // keep the result the same as the argument so async impact can be easily observed
callback(result);
}, delay);
}
/*
* A Simulated Completion Function
*/
function final(results) {
console.log('Done', results);
console.log(process.hrtime(start)[0] + "." + (process.hrtime(start)[1] / 1000000).toFixed(0) + " sec(s) total time");
}
/*
* 'Parallel' Flow Control Construct
*/
function parallel(remaining_items, results) {
items.forEach(function(item) {
async(item, function(result) {
results.push(result);
console.log('Results so far: ', results);
if (results.length == items.length) {
final(results);
}
});
});
}
/*
* Main
*/
var items = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]; // Work array
var results = []; // Results array
var start = process.hrtime(); // Start a timer
parallel(items, results);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment