Skip to content

Instantly share code, notes, and snippets.

@avishekrk
Created May 31, 2018 10:03
Show Gist options
  • Select an option

  • Save avishekrk/723810de774b09dec1d8ccae59e13d75 to your computer and use it in GitHub Desktop.

Select an option

Save avishekrk/723810de774b09dec1d8ccae59e13d75 to your computer and use it in GitHub Desktop.
Clean food inspections
!/bin/bash
wget https://github.com/avishekrk/where_not_to_eat/archive/master.zip;
unzip master.zip;
rm -v master.zip
mv -v where_not_to_eat-master raw
cd ./raw/
pwd
find -name "* *" | while read f; do echo ${f}; new=$(echo $f | sed "s/ /_/g"); echo ${new}; mv -v "$f" $new; done
mkdir -v ./../staging
head -1 food_inspection_2018-01-01.csv| tr '[:upper:]' '[:lower:]' | sed -e "s/#//g" -e "s/ ,/,/g" -e "s/ /_/g" -e s"/^,//g" > header
cat header > all_inspections.csv
for f in food_inspection_2018-0*.csv; do echo ${f}; awk 'NR > 1 {print}' ${f} >> all_inspections.csv; done
mv -v all_inspections.csv ./../staging
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment