Skip to content

Instantly share code, notes, and snippets.

@mjambon
Last active March 28, 2024 11:09
Show Gist options
  • Save mjambon/79adfc5cf6b11252e78b75df50793f24 to your computer and use it in GitHub Desktop.
Save mjambon/79adfc5cf6b11252e78b75df50793f24 to your computer and use it in GitHub Desktop.
bash: Run parallel commands and fail if any of them fails
#! /usr/bin/env bash
#
# Run parallel commands and fail if any of them fails.
#
set -eu
pids=()
for x in 1 2 3; do
ls /not-a-file &
pids+=($!)
done
for pid in "${pids[@]}"; do
wait "$pid"
done
#! /usr/bin/env bash
#
# Run parallel commands and fail if any of them fails.
#
# The expected output is something like this:
#
# $ ./parallel-explained
# ls: cannot access '/not-a-file': No such file or directory
# ls: cannot access '/not-a-file'ls: cannot access '/not-a-file': No such file or directory
# : No such file or directory
#
# Our 'parallel-explained' script exited with code '2', because it's the exit
# code of one of the failed 'ls' jobs:
#
# $ echo $?
# 2
#
# 'set -e' tells the shell to exit if any of the foreground command fails,
# i.e. exits with a non-zero status.
set -eu
# Initialize array of PIDs for the background jobs that we're about to launch.
pids=()
for x in 1 2 3; do
# Run a command in the background. We expect this command to fail.
ls /not-a-file &
# Add the PID of this background job to the array.
pids+=($!)
done
# Wait for each specific process to terminate.
# Instead of this loop, a single call to 'wait' would wait for all the jobs
# to terminate, but it would not give us their exit status.
#
for pid in "${pids[@]}"; do
#
# Waiting on a specific PID makes the wait command return with the exit
# status of that process. Because of the 'set -e' setting, any exit status
# other than zero causes the current shell to terminate with that exit
# status as well.
#
wait "$pid"
done
@Manouchehri
Copy link

Here's my solution:

#!/usr/bin/env bash

set -eu

ARG1=${1:-$(nproc --ignore=1)}

pids=()

for x in $(seq 1 ${ARG1}); do
  python3 unit_tests.py &
  pids+=($!)
done

for pid in "${pids[@]}"; do
  if wait -n; then
    :
  else
    exit_code=$?
    echo "Process exited with $exit_code, killing other tests now."
    for pid in "${pids[@]}"; do
      kill -9 "$pid" 2> /dev/null || :
    done
    exit "$exit_code"
  fi
done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment