Skip to content

Instantly share code, notes, and snippets.

@voldyman
Created June 24, 2013 09:14
Show Gist options
  • Save voldyman/5848780 to your computer and use it in GitHub Desktop.
Save voldyman/5848780 to your computer and use it in GitHub Desktop.
Simple python API scraper
from BeautifulSoup import BeautifulSoup
from urllib2 import urlopen
def get_page(pageToScrape):
return urlopen(pageToScrape).read()
def main():
pagesToScrape = ['http://example.com/data1/api.xml' , 'http://example.com/data2/api.xml']
for page in pagesToScrape:
content = get_page(page)
parsedContent = BeautifulSoup(content)
for data in parsedContent.findAll('info'):
print data
if __name__ == '__main__':
main()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment