Skip to content

Instantly share code, notes, and snippets.

@urielaero
Created January 19, 2022 18:09
Show Gist options
  • Save urielaero/a6569ecd865b25c25ac2ed7f70c47f6b to your computer and use it in GitHub Desktop.
Save urielaero/a6569ecd865b25c25ac2ed7f70c47f6b to your computer and use it in GitHub Desktop.
Split CSV with bash
#!/bin/bash
# usage $1: number_files $2: filename $3: [output_path]
# example:
# $ ./split_csv.sh 2 my_dir/large_file.csv ./output_dir
if [ $# -ge 3 ]
then
mkdir -p $3
fi
output=${3:-./}
split -nl/$1 -d --additional-suffix=.csv $2 $output/part_
# TODO support more 00000
# TODO better argument list (--help)
for file in $output/part_*
do
if [[ $file != *art_00.csv ]]
then
head -n 1 $2 > tmp_file
cat $file >> tmp_file
mv -f tmp_file $file
fi
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment