Wrapper around GNU parallel with advanced options for better control flow, and human- and machine-readable results

README.md

run-parallel-cmds.sh

Execute commands in parallel with structured output, logging, and progress tracking.

A bash wrapper around GNU parallel that simplifies running multiple commands concurrently with:

• Structured output (JSON or text format)
• Per-command logs (stdout/stderr capture)
• Timeout handling with signal detection
• Progress tracking and summary statistics
• Color output and progress bars
• Flexible input (CLI args, file, or stdin)

Requirements

• GNU parallel — required
  • macOS: brew install parallel
  • Ubuntu/Debian: apt install parallel
  • Arch: pacman -S parallel
  • Alpine: apk add parallel

Installation

Copy run-parallel-cmds.sh to a directory in your $PATH and make it executable:

chmod +x run-parallel-cmds.sh

Or run directly:

bash run-parallel-cmds.sh [OPTIONS] [--] cmd1 cmd2 ...

Usage

Three ways to provide commands

1. Command-line arguments:

run-parallel-cmds.sh "echo hello" "sleep 2" "curl https://example.com"

2. File (one command per line):

run-parallel-cmds.sh -f commands.txt

3. Standard input (pipe):

cat urls.txt | run-parallel-cmds.sh --prepend "curl -sS"

Options

Option	Short	Argument	Description
--timeout	-t	SEC	Timeout per command (default: 30)
--prepend		STR	Prepend string to every command
--append		STR	Append string to every command
--logs			Include stdout/stderr in output
--tail		N	Keep only last N log lines (implies --logs)
--no-exit-code	-e		Omit exit codes from output
--format		FMT	Output format: json or text (default: json)
--short			Minimal stdout; save full output to file
--output-file	-o	PATH	File path for --short mode
--jobs	-j	N	Max parallel jobs (default: CPU count)
--file	-f	FILE	Read commands from file, one per line
--no-color			Disable colored output
--help	-h		Show usage

Examples

Basic: Run three commands with timeout and logs

run-parallel-cmds.sh -t 10 --logs "sleep 2 && echo ok" "false" "echo hi"

Output:

{
  "results": [
    {
      "command": "sleep 2 && echo ok",
      "exit_code": 0,
      "logs": "ok"
    },
    {
      "command": "false",
      "exit_code": 1
    },
    {
      "command": "echo hi",
      "exit_code": 0,
      "logs": "hi"
    }
  ],
  "summary": {
    "total": 3,
    "succeeded": 2,
    "failed": 1,
    "timed_out": 0,
    "elapsed": "2s",
    "elapsed_seconds": 2
  }
}

Prepend/append: Run with sudo

run-parallel-cmds.sh --prepend sudo --format text \
  "systemctl status nginx" \
  "systemctl status redis"

Pipe input: Parallel curl requests

cat urls.txt | run-parallel-cmds.sh --prepend "curl -sS" --short -o results.json

Saves full results to results.json, prints summary to stderr:

✓ Results saved to /home/user/parallel-results.20260320_143022.json
Total: 100  OK: 98  Fail: 2  (3m42s)

File input: Run system diagnostics

# Create command file
cat > diagnostics.txt << 'EOF'
df -h
free -h
ps aux | head -20
netstat -an | grep LISTEN
EOF

# Run with 5-minute timeout
run-parallel-cmds.sh -t 300 -f diagnostics.txt --logs --format text

Text output with progress bar

run-parallel-cmds.sh --format text --logs "sleep 1 && true" "sleep 2 && true" "sleep 3 && true"

Output:

exit code: 0 | sleep 1 && true

exit code: 0 | sleep 2 && true

exit code: 0 | sleep 3 && true

──────────────────────────────────────────────────
Total: 3  Passed: 3  Failed: 0  (3s)

  █████████████████████████████ 100%  3/3 succeeded
■ Done — 3 commands in 3s

Limit concurrency: Run with max 2 jobs

run-parallel-cmds.sh -j 2 "long_task_1" "long_task_2" "long_task_3" "long_task_4"

Timeout detection

run-parallel-cmds.sh -t 2 "sleep 5"

Outputs:

{
  "results": [
    {
      "command": "sleep 5",
      "exit_code": 137,
      "timed_out": true
    }
  ],
  "summary": {
    "total": 1,
    "succeeded": 0,
    "failed": 1,
    "timed_out": 1,
    "elapsed": "2s",
    "elapsed_seconds": 2
  }
}

Tail logs: Keep only last 5 lines

run-parallel-cmds.sh --tail 5 "command_with_long_output"

Output Formats

JSON (default)

Structured output with one object per command plus summary:

• command: The executed command
• exit_code: Exit code (0 = success)
• timed_out: Boolean flag if command exceeded timeout
• logs: Combined stdout/stderr (if --logs enabled)

Summary includes:

• total, succeeded, failed, timed_out
• elapsed: Human-readable duration
• elapsed_seconds: Total seconds

Text

Human-readable format with logs, exit codes, and summary:

• One command result per block
• Color-coded exit codes (green=0, red=non-zero, yellow=timeout)
• Summary with totals and elapsed time
• Progress bar with percentage

Exit Code

• 0: All commands succeeded
• 1: One or more commands failed

Tips

• For large command lists: Use --short to save full output to file and print summary to stderr
• For streaming input: Pipe from curl, find, or similar: find . -name "*.log" | run-parallel-cmds.sh --prepend "gzip"
• For monitoring: Parse JSON output with jq for custom reporting
• Disable colors: Use --no-color in scripts or non-TTY environments

License

This script is provided as-is for use in your projects.

Cudos

Claude Code for Ansible-my repository.

run-parallel-cmds.sh

#!/usr/bin/env bash
#
# run-parallel-cmds.sh — Execute commands in parallel via GNU parallel
#                         with structured output, logging, and progress.
#
# Usage:
#   run-parallel-cmds.sh [OPTIONS] [--] "cmd1" "cmd2" ...
#   run-parallel-cmds.sh [OPTIONS] -f commands.txt
#   echo -e "cmd1\ncmd2" | run-parallel-cmds.sh [OPTIONS]
#
# Requires: GNU parallel

set -uo pipefail

# ═══════════════════════════════════════════════════════════════════════════════
# Defaults
# ═══════════════════════════════════════════════════════════════════════════════
TIMEOUT=30
PREPEND=""
APPEND=""
SHOW_LOGS=false
TAIL_LINES=0
SHOW_EXIT_CODE=true
FORMAT="json"           # json | text
SHORT_OUTPUT=false
OUTPUT_FILE=""
MAX_JOBS=0              # 0 = let parallel decide (CPU count)
NO_COLOR=false
declare -a COMMANDS=()

# ═══════════════════════════════════════════════════════════════════════════════
# Colors (set once after arg parsing)
# ═══════════════════════════════════════════════════════════════════════════════
RED="" GRN="" YLW="" BLU="" CYN="" BLD="" DIM="" RST=""

init_colors() {
  if [[ "$NO_COLOR" == true ]] || [[ ! -t 2 ]]; then
    return
  fi
  RED=$'\033[31m'   GRN=$'\033[32m'   YLW=$'\033[33m'
  BLU=$'\033[34m'   CYN=$'\033[36m'   BLD=$'\033[1m'
  DIM=$'\033[2m'    RST=$'\033[0m'
}

# ═══════════════════════════════════════════════════════════════════════════════
# Helpers
# ═══════════════════════════════════════════════════════════════════════════════
die()  { printf '%s\n' "${RED}Error:${RST} $*" >&2; exit 1; }

fmt_elapsed() {
  local s=$1
  if (( s >= 3600 )); then printf '%dh%02dm%02ds' $((s/3600)) $((s%3600/60)) $((s%60))
  elif (( s >= 60 )); then printf '%dm%02ds' $((s/60)) $((s%60))
  else                      printf '%ds' "$s"
  fi
}

# Pure-bash JSON string escape (handles \, ", newline, CR, tab)
json_escape() {
  local s="$1"
  s="${s//\\/\\\\}"
  s="${s//\"/\\\"}"
  s="${s//$'\n'/\\n}"
  s="${s//$'\r'/\\r}"
  s="${s//$'\t'/\\t}"
  printf '%s' "\"$s\""
}

strip_ansi() { sed $'s/\033\[[0-9;]*m//g'; }

progress_bar() {
  local pct=$1 width=${2:-30}
  local filled=$(( pct * width / 100 ))
  local empty=$(( width - filled ))
  printf '%s' "${GRN}"
  printf '█%.0s' $(seq 1 $filled 2>/dev/null)   || true
  printf '%s' "${DIM}"
  printf '░%.0s' $(seq 1 $empty 2>/dev/null)     || true
  printf '%s %3d%%' "${RST}" "$pct"
}

# ═══════════════════════════════════════════════════════════════════════════════
# Usage
# ═══════════════════════════════════════════════════════════════════════════════
usage() {
  cat <<'EOF'
Usage: run-parallel-cmds.sh [OPTIONS] [--] "cmd1" "cmd2" ...
       run-parallel-cmds.sh [OPTIONS] -f commands.txt
       echo -e "cmd1\ncmd2" | run-parallel-cmds.sh [OPTIONS]

Options:
  -t, --timeout SEC       Timeout per command              (default: 30)
      --prepend STR       Prepend string to every command
      --append  STR       Append  string to every command
      --logs              Include stdout/stderr in output   (default: off)
      --tail N            Keep only last N log lines        (implies --logs)
  -e, --no-exit-code      Omit exit codes from output
      --format FMT        Output format: json | text        (default: json)
      --short             Minimal stdout; save full output to file
  -o, --output-file PATH  File path for --short mode        (default: $PWD/parallel-results.<ts>.<ext>)
  -j, --jobs N            Max parallel jobs                 (default: CPU count)
  -f, --file FILE         Read commands from file, one per line
      --no-color          Disable colored output
  -h, --help              Show this help

Examples:
  # Run three commands with 10s timeout, show logs
  run-parallel-cmds.sh -t 10 --logs "sleep 2 && echo ok" "false" "echo hi"

  # Prepend sudo to every command, text output
  run-parallel-cmds.sh --prepend sudo --format text "systemctl status nginx" "systemctl status redis"

  # Pipe commands, short output saved to file
  cat urls.txt | run-parallel-cmds.sh --prepend "curl -sS" --short -o results.json
EOF
}

# ═══════════════════════════════════════════════════════════════════════════════
# Parse arguments
# ═══════════════════════════════════════════════════════════════════════════════
while [[ $# -gt 0 ]]; do
  case "$1" in
    -t|--timeout)      TIMEOUT="$2";        shift 2 ;;
    --prepend)         PREPEND="$2";         shift 2 ;;
    --append)          APPEND="$2";          shift 2 ;;
    --logs)            SHOW_LOGS=true;       shift   ;;
    --tail)            TAIL_LINES="$2"; SHOW_LOGS=true; shift 2 ;;
    -e|--no-exit-code) SHOW_EXIT_CODE=false; shift   ;;
    --format)          FORMAT="$2";          shift 2 ;;
    --short)           SHORT_OUTPUT=true;    shift   ;;
    -o|--output-file)  OUTPUT_FILE="$2";     shift 2 ;;
    -j|--jobs)         MAX_JOBS="$2";        shift 2 ;;
    --no-color)        NO_COLOR=true;        shift   ;;
    -f|--file)
      while IFS= read -r _line; do
        [[ -z "$_line" || "$_line" =~ ^[[:space:]]*# ]] && continue
        COMMANDS+=("$_line")
      done < "$2"
      shift 2 ;;
    -h|--help) usage; exit 0 ;;
    --)        shift; COMMANDS+=("$@"); break ;;
    -*)        printf 'Unknown option: %s\n' "$1" >&2; usage >&2; exit 2 ;;
    *)         COMMANDS+=("$1"); shift ;;
  esac
done

# Read from stdin when no commands given and stdin is piped
if [[ ${#COMMANDS[@]} -eq 0 ]] && [[ ! -t 0 ]]; then
  while IFS= read -r _line; do
    [[ -z "$_line" || "$_line" =~ ^[[:space:]]*# ]] && continue
    COMMANDS+=("$_line")
  done
fi

# -o without --short implies --short
[[ -n "$OUTPUT_FILE" ]] && SHORT_OUTPUT=true

init_colors

# ═══════════════════════════════════════════════════════════════════════════════
# Validate
# ═══════════════════════════════════════════════════════════════════════════════
command -v parallel &>/dev/null || die "GNU parallel is required but not found.
  Install: apt install parallel | brew install parallel | pacman -S parallel"

[[ ${#COMMANDS[@]} -eq 0 ]] && { die "No commands provided. See --help."; }

case "$FORMAT" in
  json|text) ;;
  *) die "Unknown format '$FORMAT'. Use json or text." ;;
esac

# ═══════════════════════════════════════════════════════════════════════════════
# Prepare workspace
# ═══════════════════════════════════════════════════════════════════════════════
WORK_DIR=$(mktemp -d)
trap 'rm -rf "$WORK_DIR"' EXIT

JOBLOG="$WORK_DIR/joblog"
OUT_DIR="$WORK_DIR/out"
mkdir -p "$OUT_DIR"

# Build constructed commands (apply prepend / append)
declare -a CONSTRUCTED=()
for cmd in "${COMMANDS[@]}"; do
  built=""
  [[ -n "$PREPEND" ]] && built+="${PREPEND} "
  built+="$cmd"
  [[ -n "$APPEND" ]]  && built+=" ${APPEND}"
  CONSTRUCTED+=("$built")
done

TOTAL=${#CONSTRUCTED[@]}

# Write one script per command (avoids quoting issues in parallel)
for i in "${!CONSTRUCTED[@]}"; do
  printf '%s\n' "${CONSTRUCTED[$i]}" > "$WORK_DIR/cmd_$((i+1)).sh"
done

# Wrapper: runs a numbered command and captures stdout/stderr to files
cat > "$WORK_DIR/_run.sh" << 'WRAPPER'
#!/usr/bin/env bash
idx="$1"; dir="$2"
bash "$dir/cmd_${idx}.sh" \
  >"$dir/out/${idx}.stdout" \
  2>"$dir/out/${idx}.stderr"
WRAPPER
chmod +x "$WORK_DIR/_run.sh"

# ═══════════════════════════════════════════════════════════════════════════════
# Execute
# ═══════════════════════════════════════════════════════════════════════════════
printf '\n' >&2
printf '%s\n' \
  "${BLD}${CYN}▶ Running ${TOTAL} command(s) in parallel${RST}  ${DIM}timeout=${TIMEOUT}s${RST}" >&2
printf '%s\n' "${DIM}$(printf '─%.0s' $(seq 1 50))${RST}" >&2

# Assemble parallel flags
PARALLEL_ARGS=(
  --timeout "$TIMEOUT"
  --joblog  "$JOBLOG"
  --halt    never
)
[[ -t 2 ]] && PARALLEL_ARGS+=(--bar)
(( MAX_JOBS > 0 )) && PARALLEL_ARGS+=(-j "$MAX_JOBS")

START_EPOCH=$(date +%s)

seq 1 "$TOTAL" | parallel "${PARALLEL_ARGS[@]}" \
  "bash \"$WORK_DIR/_run.sh\" {} \"$WORK_DIR\"" || true

END_EPOCH=$(date +%s)
ELAPSED=$(( END_EPOCH - START_EPOCH ))

# ═══════════════════════════════════════════════════════════════════════════════
# Parse job log
# ═══════════════════════════════════════════════════════════════════════════════
declare -A EXIT_CODES=() SIGNALS=() JOB_TIMES=()
SUCCEEDED=0  FAILED=0  TIMED_OUT=0

while IFS=$'\t' read -r seq host starttime runtime send receive exitval signal _rest; do
  [[ "$seq" == "Seq" ]] && continue        # header row
  EXIT_CODES[$seq]="$exitval"
  SIGNALS[$seq]="$signal"
  JOB_TIMES[$seq]="$runtime"

  if [[ "$signal" -ne 0 ]]; then
    (( TIMED_OUT++ ));  (( FAILED++ ))
  elif [[ "$exitval" -ne 0 ]]; then
    (( FAILED++ ))
  else
    (( SUCCEEDED++ ))
  fi
done < "$JOBLOG"

# ═══════════════════════════════════════════════════════════════════════════════
# Collect logs for a command index (1-based)
# ═══════════════════════════════════════════════════════════════════════════════
get_logs() {
  local idx=$1 combined=""
  local f_out="$OUT_DIR/${idx}.stdout"
  local f_err="$OUT_DIR/${idx}.stderr"

  [[ -s "$f_out" ]] && combined=$(< "$f_out")
  if [[ -s "$f_err" ]]; then
    [[ -n "$combined" ]] && combined+=$'\n'
    combined+=$(< "$f_err")
  fi

  if (( TAIL_LINES > 0 )) && [[ -n "$combined" ]]; then
    combined=$(printf '%s' "$combined" | tail -n "$TAIL_LINES")
  fi

  printf '%s' "$combined"
}

# ═══════════════════════════════════════════════════════════════════════════════
# Format: JSON
# ═══════════════════════════════════════════════════════════════════════════════
format_json() {
  local buf
  buf=$'{\n  "results": [\n'

  local i
  for (( i = 1; i <= TOTAL; i++ )); do
    local cmd="${CONSTRUCTED[$((i-1))]}"
    local ec="${EXIT_CODES[$i]:-255}"
    local sig="${SIGNALS[$i]:-0}"
    local is_timeout="false"
    (( sig != 0 )) && is_timeout="true"

    buf+="    {"
    buf+=$'\n'"      \"command\": $(json_escape "$cmd")"

    if [[ "$SHOW_EXIT_CODE" == true ]]; then
      buf+=","$'\n'"      \"exit_code\": ${ec}"
      [[ "$is_timeout" == "true" ]] && \
        buf+=","$'\n'"      \"timed_out\": true"
    fi

    if [[ "$SHOW_LOGS" == true ]]; then
      local logs
      logs=$(get_logs "$i")
      buf+=","$'\n'"      \"logs\": $(json_escape "$logs")"
    fi

    buf+=$'\n'"    }"
    (( i < TOTAL )) && buf+=","
    buf+=$'\n'
  done

  buf+='  ],'$'\n'
  buf+='  "summary": {'$'\n'
  buf+="    \"total\": ${TOTAL},"$'\n'
  buf+="    \"succeeded\": ${SUCCEEDED},"$'\n'
  buf+="    \"failed\": ${FAILED},"$'\n'
  buf+="    \"timed_out\": ${TIMED_OUT},"$'\n'
  buf+="    \"elapsed\": \"$(fmt_elapsed "$ELAPSED")\","$'\n'
  buf+="    \"elapsed_seconds\": ${ELAPSED}"$'\n'
  buf+='  }'$'\n'
  buf+='}'

  printf '%s\n' "$buf"
}

# ═══════════════════════════════════════════════════════════════════════════════
# Format: text
# ═══════════════════════════════════════════════════════════════════════════════
format_text() {
  local buf="" i
  for (( i = 1; i <= TOTAL; i++ )); do
    local cmd="${CONSTRUCTED[$((i-1))]}"
    local ec="${EXIT_CODES[$i]:-255}"
    local sig="${SIGNALS[$i]:-0}"

    # Logs
    if [[ "$SHOW_LOGS" == true ]]; then
      local logs
      logs=$(get_logs "$i")
      [[ -n "$logs" ]] && buf+="${logs}"$'\n'
    fi

    # Exit code | command
    if [[ "$SHOW_EXIT_CODE" == true ]]; then
      if (( sig != 0 )); then
        buf+="${YLW}exit code: TIMEOUT${RST} | ${cmd}"$'\n'
      elif (( ec == 0 )); then
        buf+="${GRN}exit code: ${ec}${RST} | ${cmd}"$'\n'
      else
        buf+="${RED}exit code: ${ec}${RST} | ${cmd}"$'\n'
      fi
    else
      buf+="${cmd}"$'\n'
    fi

    buf+=$'\n'
  done

  # Summary
  buf+="${BLD}$(printf '─%.0s' $(seq 1 50))${RST}"$'\n'

  buf+="${BLD}Total:${RST} ${TOTAL}  "
  buf+="${GRN}Passed:${RST} ${SUCCEEDED}  "
  buf+="${RED}Failed:${RST} ${FAILED}"
  (( TIMED_OUT > 0 )) && buf+="  ${YLW}Timeout:${RST} ${TIMED_OUT}"
  buf+="  ${DIM}($(fmt_elapsed "$ELAPSED"))${RST}"$'\n'

  printf '%s\n' "$buf"
}

# ═══════════════════════════════════════════════════════════════════════════════
# Emit output
# ═══════════════════════════════════════════════════════════════════════════════
RESULT=""
if [[ "$FORMAT" == "json" ]]; then
  RESULT=$(format_json)
else
  RESULT=$(format_text)
fi

if [[ "$SHORT_OUTPUT" == true ]]; then
  # Determine output file path
  if [[ -z "$OUTPUT_FILE" ]]; then
    local_ext="json"
    [[ "$FORMAT" != "json" ]] && local_ext="txt"
    OUTPUT_FILE="${PWD}/parallel-results.$(date +%Y%m%d_%H%M%S).${local_ext}"
  fi

  # Strip ANSI when writing to file
  printf '%s\n' "$RESULT" | strip_ansi > "$OUTPUT_FILE"

  printf '%s\n' "${GRN}✓${RST} Results saved to ${BLD}${OUTPUT_FILE}${RST}" >&2

  # Short summary on stderr
  printf '%s' "${BLD}Total:${RST} ${TOTAL}  ${GRN}OK:${RST} ${SUCCEEDED}  ${RED}Fail:${RST} ${FAILED}" >&2
  (( TIMED_OUT > 0 )) && printf '  %s' "${YLW}Timeout:${RST} ${TIMED_OUT}" >&2
  printf '  %s\n' "${DIM}($(fmt_elapsed "$ELAPSED"))${RST}" >&2
else
  printf '%s\n' "$RESULT"
fi

# ── Footer with progress bar ─────────────────────────────────────────────────
printf '\n' >&2

if (( TOTAL > 0 )); then
  pct=$(( SUCCEEDED * 100 / TOTAL ))
  printf '  %s  ' "$(progress_bar "$pct" 30)" >&2
  printf '%s\n' "${DIM}${SUCCEEDED}/${TOTAL} succeeded${RST}" >&2
fi

printf '%s\n\n' \
  "${BLD}${CYN}■ Done${RST} — ${TOTAL} commands in $(fmt_elapsed "$ELAPSED")" >&2

# Exit non-zero if any command failed
(( FAILED > 0 )) && exit 1
exit 0