Skip to content

Instantly share code, notes, and snippets.

@acompa

acompa/scrapegist.py

Created February 6, 2011 04:08
Show Gist options
  • Save acompa/813116 to your computer and use it in GitHub Desktop.
Save acompa/813116 to your computer and use it in GitHub Desktop.
Download ZIP
General idea behind opening and DLing BBs.
Raw
scrapegist.py
import urllib2, re
targetURL = "http://www.minneapolisfed.org/bb/reports/1990/90-01-su.cfm"
site = urllib2.urlopen(targetURL)
out = "/Users/alexc/Dropbox/Develop/BB/data/1990-01.html"
with open(out, 'w') as BBFile:
for line in site.readlines():
p = re.compile(r'<.*?>')
BBFile.write(p.sub('', line))
site.close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment