Skip to content

Instantly share code, notes, and snippets.

@cpdean
Created December 3, 2014 22:08
Show Gist options
  • Save cpdean/ae94bc802825b332aa46 to your computer and use it in GitHub Desktop.
Save cpdean/ae94bc802825b332aa46 to your computer and use it in GitHub Desktop.
parallel demo
#!/bin/bash
# pull out census data per city file and put it into its own file
for i in $(ls raw_city_data/);
do
echo "parsing $i"
grep -i "census:population" raw_city_data/$i > census_populations/$i
done
# do the exact same thing, but takes a third the time (and makes your computer very very warm)
ls raw_city_data/ | parallel "grep -i \"census:population\" raw_city_data/{} > census_populations/{}"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment