Skip to content

Instantly share code, notes, and snippets.

@christopher-beckham

christopher-beckham/cc_workflow.md

Last active September 25, 2019 23:39
Show Gist options
  • Save christopher-beckham/add2a00d43730d683cba1d67b9f8727e to your computer and use it in GitHub Desktop.
Save christopher-beckham/add2a00d43730d683cba1d67b9f8727e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
cc_workflow.md

launch_any.sh:

#!/bin/bash
#SBATCH --ntasks=1
#SBATCH -c 4
#SBATCH --gres=gpu
#SBATCH --mem=12G

# ^ Flags here only are used if sbatch is invoked
# on this script directly, otherwise the sbatch
# flags are in the flow script.

# (Optional)
export RESULTS_DIR=`cd ../results_new; pwd`

# Activate environment
source activate pytorch-env

for script in "$@"
do
    echo "launching $script ..."
    ./${script} &
done

wait

flow_submit_xx.sh:

#!/bin/bash

echo "Submitting scripts: " "$@"

flow-submit \
--resume \
--prolog "source ../envs/env_imagenet.sh" \
--options "mem=12G;time=12:00:00;account=...;ntasks=1;cpus-per-task=4;gres=gpu:1" \
--root=`pwd` \
launch launch_any.sh "$@"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment