Skip to content

Instantly share code, notes, and snippets.

@mikekunze
Last active December 27, 2015 01:19
Show Gist options
  • Save mikekunze/7243738 to your computer and use it in GitHub Desktop.
Save mikekunze/7243738 to your computer and use it in GitHub Desktop.
This is a timer for running multiple compiled matlab binaries in parallel on a server with more than 30 cores. It uses async to wait before all processes exit before calculating total processing time.
# To run this, grab the x64 node.js binary tarball and extract it to ~/node
#
# Then:
# ln -s ~/node/bin/node ~/bin/node; ln -s ~/node/bin/npm ~/bin/npm
# npm install coffee-script
# ln -s ~/node_modules/coffee-script/bin/coffee ~/bin/coffee
#
# Run:
# coffee timer.coffee
# Modules
async = require 'async'
spawn = require('child_process').spawn
fs = require 'fs'
# Start our timer
startTime = Date.now()
startTimeString = Date()
endTime = Date.now()
endTimeString = Date()
# Grab full path of working directory
cwd = process.cwd()
# Create an array with 30 items, with key value starting at 1
processes = [1...30]
# Create our process holder object, so it is global to eachProcess fn
activeProcesses = {}
# This is our process runner function.
# It defines what we do for each processor value
eachProcess = (process, cb)->
# Spawn the process, provide the batch file location as a string argument
activeProcesses[process] = spawn "#{cwd}/Processor_Read", ["#{cwd}/Processor_Batch_Files/Processor_Batch_#{process}"]
# Set the encoding so that we see it in text
activeProcesses[process].stdout.setEncoding 'utf8'
activeProcesses[process].stderr.setEncoding 'utf8'
# Define our stream eventes for stderr and stdout on data
activeProcesses[process].stdout.on 'data', (data)->
console.log "#{Date()} - #{process} working..."
activeProcesses[process].stderr.on 'data', (data)->
console.log "#{Date()} - #{process} err working... #{data}"
# Define what we do when the spawned process finished or ends unexpectedly
activeProcesses[process].on 'close', (c)->
console.log "#{Date()} - #{process} EXIT with #{c}"
cb()
# Here is when we are actually putting everything together.
# For each spawned process on close event, a callback is executed.
# When all callbacks are executed, the anonymous function
# attached at the end of the forEach call will be executed.
async.forEach processes, eachProcess, ()->
endTime = Date.now()
endTimeString = Date()
console.log "Start Time: #{startTimeString}"
console.log "End Time: #{endTimeString}"
console.log "Total Time #{(endTime - startTime) / 1000}"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment