Skip to content

Instantly share code, notes, and snippets.

@damc-dev

damc-dev/gist:29d3a162c53265cd9e3590fa28f756c7

Forked from looselycoupled/gist:4f23a6ed698b9c62387c596f5613fd6a
Created May 20, 2019 14:45
Show Gist options
  • Save damc-dev/29d3a162c53265cd9e3590fa28f756c7 to your computer and use it in GitHub Desktop.
Save damc-dev/29d3a162c53265cd9e3590fa28f756c7 to your computer and use it in GitHub Desktop.
Download ZIP
Bash CSV 'split' Example
Raw
gistfile1.txt
# The following will split a CSV (file.csv) into multiple parts of 1 million lines each
# with each part having its own header.
#
# PREFIX denotes the filename to use for the parts. A number will be added to the end.
tail -n +2 file.csv |
split -d -l 1000000 - --filter='sh -c "{ head -n1 file.csv; cat; } > $FILE"' PREFIX
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment