Skip to content

Instantly share code, notes, and snippets.

@cquest
Last active April 22, 2016 09:26
Show Gist options
  • Save cquest/665dab89a55ec9e3d3d9 to your computer and use it in GitHub Desktop.
Save cquest/665dab89a55ec9e3d3d9 to your computer and use it in GitHub Desktop.
Géocodage en batch de GROS fichiers CSV via addok...
#!/bin/bash
# exemple: ./bigaddok.sh in.csv out.csv "columns=numero columns=voie"
# needs httpie (pip install httpie)
lines=`wc -l $1| grep ^[0-9]* -o`
maxlines=10000
export chunks=`echo "$lines/$maxlines"|bc`
api="http://api-adresse.data.gouv.fr/search/csv/"
for chunk in `seq 0 $chunks`; do
head --lines=1 $1 > temp.csv
tail $1 --lines=+2 | tail --lines=+`echo $chunk*$maxlines|bc` | head -n $maxlines >> temp.csv
http --timeout 600 -f POST $api $3 [email protected] > out.csv
if [ "$chunk" = "0" ]; then
mv out.csv $2
else
tail -n +2 out.csv >> $2
fi
echo "`date +%H:%M:%S` $chunk/$chunks"
done
@cquest
Copy link
Author

cquest commented Apr 22, 2016

http come from httpie, see http://radek.io/2015/10/20/httpie/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment