Skip to content

Instantly share code, notes, and snippets.

@Jan-Zeiseweis
Last active September 14, 2015 17:15
Show Gist options
  • Save Jan-Zeiseweis/d7c85da5bfdeca3bcbfa to your computer and use it in GitHub Desktop.
Save Jan-Zeiseweis/d7c85da5bfdeca3bcbfa to your computer and use it in GitHub Desktop.
split and gzip bash script
#!/bin/bash -e
# usage: bash ./split_and_gzip.sh file.txt
FILE=$1
LINES_PER_FILE=10000000 #10m
$(split -l $LINES_PER_FILE -d $FILE ${FILE}.)
FILES=${FILE}.*
for f in $FILES
do
echo "GZipping file $f ..."
number=$(echo $f | grep -oE "[[:digit:]]{1,}$")
filename="${FILE}.gz.${number}"
$(gzip $f -c > $filename)
$(rm $f)
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment