Skip to content

Instantly share code, notes, and snippets.

@neilkod
Created December 15, 2011 19:57
Show Gist options
  • Save neilkod/1482599 to your computer and use it in GitHub Desktop.
Save neilkod/1482599 to your computer and use it in GitHub Desktop.
bash-3.1$ cat test.py
import gzip
datafile="/var/opt/sports_dw/dev/nz_sports/done/flat_customer_sport_preference_fct1.out.gz"
opn=gzip.open(datafile)
for i, line in enumerate(opn):
pass
print i+1
bash-3.1$ time python test.py
33725715
real 5m8.971s
user 5m6.272s
sys 0m0.644s
bash-3.1$ time zcat /var/opt/sports_dw/dev/nz_sports/done/flat_customer_sport_preference_fct1.out.gz|wc -l
33725715
real 0m16.134s
user 0m14.855s
sys 0m1.743s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment