Skip to content

Instantly share code, notes, and snippets.

@seebeen
Created November 8, 2023 08:30
Show Gist options
  • Save seebeen/6c61619fa66e45cb27fee6c0bb4cc6b7 to your computer and use it in GitHub Desktop.
Save seebeen/6c61619fa66e45cb27fee6c0bb4cc6b7 to your computer and use it in GitHub Desktop.
Archiving functions
# Function to store files into a tar.gz archive using pigz with compression level 1
storeit() {
local tarfile=$1
local pathtotar=$2
local cpus=${3:-12} # Default to 12 CPUs if not specified
# Count the total number of files and total size for progress bar
local totalsize=$(du -sb "$pathtotar" | awk '{print $1}')
# Archive and compress with progress bar
tar -cf - -C "$pathtotar" . | pv -s "$totalsize" -petrab -N "Compressing" | pigz -1 -p "$cpus" > "$tarfile"
}
# Function to extract files from a tar.gz archive using pigz
unstoreit() {
local tarfile=$1
local targetpath=$2
local cpus=${3:-12} # Default to 12 CPUs if not specified
# Ensure the target directory exists
mkdir -p "$targetpath"
# Get the compressed size for the progress bar
local compressedsize=$(du -sb "$tarfile" | awk '{print $1}')
# Extract with progress bar
pigz -d -p "$cpus" -c "$tarfile" | pv -s "$compressedsize" -petrab -N "Extracting" | tar -xf - -C "$targetpath"
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment