Last active
April 28, 2021 23:20
-
-
Save kpittman-securus/9ad20bb2e0a6d73ed2429c8be6bd7f58 to your computer and use it in GitHub Desktop.
Parallel execution with find and xargs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#! /usr/bin/env bash | |
find ./src/bookmarklets -name "*.js" -print0 | xargs -I FILE -0 -P "$(nproc)" npm run rollup -- --input "FILE" --file "output/FILE" | |
# EDIT: Per my latest comment - I now prefer this: | |
for bookmarklet in ./src/bookmarklets/**/*.js | |
do | |
npm run rollup -- --input "$bookmarklet" --file "output/$bookmarklet" & | |
done | |
wait |
After some time to stew on this, and reading some alternatives, I feel that using a loop with background processes is more readable.
for thing in glob
do
npm run doThingWithArg -- --file=thing & # & sends proc to bg
done
wait # waits for bg procs to complete
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
https://serverfault.com/questions/193319/a-better-unix-find-with-parallel-processing
https://remysharp.com/2016/12/16/tricks-with-xargs
https://www.cyberciti.biz/faq/check-how-many-cpus-are-there-in-linux-system/