Skip to content

Instantly share code, notes, and snippets.

View georgegu1997's full-sized avatar

Qiao Gu georgegu1997

View GitHub Profile
@georgegu1997
georgegu1997 / parallel_jobs.md
Created June 14, 2023 03:46
Run multiple jobs with a max number of parallel processes

This is a function to run multiple jobs in parallel, but with a max number of parallel processes. New jobs will be run only if some previous jobs end and free up resources.

Credit to GPT-4.

First declare the following function in bash

function wait_for_jobs() {
  local num_jobs=$1
 while true; do
@georgegu1997
georgegu1997 / useful_slurm.md
Last active May 2, 2025 03:29
Useful slurm commands and tricks

Show the full submit command for each job in the queue

alias sqx='for J in $(squeue -h -u $USER -o "%i"); do
             printf "\n\nJob %s\n" "$J"
             scontrol show job $J | grep -E "TresPerNode|Command"
             sacct -j $J -o submitline -n -P
           done'