Last active
July 18, 2025 07:39
-
-
Save dch/b5df789dfb7ee1e8b8afa6263605efc7 to your computer and use it in GitHub Desktop.
tarsnap hacky parallel restore script
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/sh | |
| # recover all files in parallel from the most recent archive | |
| # MIT license | |
| # https://git.io/vdrbG | |
| # "works on my machine" | |
| # lots of assumptions notably path length (strip-component) | |
| # get the latest archive as our names can be sorted by time | |
| ARCHIVE=`tarsnap --keyfile /tmp/tarsnap.key --list-archives | sort | tail -1` | |
| # order the archives by descending size | |
| FILES=`tarsnap --keyfile /tmp/tarsnap.key -tvf ${ARCHIVE} | cut -w -f 5,9 | sort -rn | cut -w -f 2` | |
| # spawn 10 invocations in parallel (use -P 0 for unlimited) | |
| echo $FILES | xargs -P 10 -n 1 -t \ | |
| time tarsnap \ | |
| --retry-forever \ | |
| -S \ | |
| --strip-components 6 \ | |
| --print-stats \ | |
| --humanize-numbers \ | |
| --keyfile /tmp/tarsnap.key \ | |
| --chroot \ | |
| -xv \ | |
| -f ${ARCHIVE} | |
| # profit |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Why all this cut and sort in the FILES line?
I'm currently pretty happy with my simplified version:
I omitted -v, so no cut is necessary to cut out the path.
I added --resume-extract to skip already extracted files.
I added -C ... to extract to a specific dir and not to $PWD.
-P 20 -n 20 is reasonable for my case.
Adding " -- " to the end helps with filenames that contain " -" or " --" that might otherwise be interpreted as tarsnap option.
Other options like -S, --retry-forever, --strip-components 6, seem specific to the authors use-case. So I omitted them for mine.