Skip to content

Instantly share code, notes, and snippets.

@nicolasbrugneaux
Created March 30, 2016 09:53
Show Gist options
  • Save nicolasbrugneaux/b8b1f3f715d202e020a6d3d57cbfd043 to your computer and use it in GitHub Desktop.
Save nicolasbrugneaux/b8b1f3f715d202e020a6d3d57cbfd043 to your computer and use it in GitHub Desktop.
true parallelism in node
import { cpus } from 'os';
import cluster from 'cluster';
if (cluster.isMaster) {
getAllRows()
.then(rows => {
const CPUS = cpus().length;
const chunked =chunks(rows, CPUS); // n rows into an array of length CPUS containing n/CPUS-ish rows (tries to balance)
const statuses = chunked
.filter(x => x.length)
.map(chunk => new Promise((resolve, reject) => {
const worker = cluster.fork({
__chunk__: JSON.stringify(chunk)
});
worker.on('exit', resolve);
}));
Promise.all(statuses)
.then(codes => {
process.exit(codes.every(x => x === 0) ? 0 : 1);
});
})
}
else {
const rows = JSON.parse(process.env.__chunk__);
// do stuff
// ...
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment