Skip to content

Instantly share code, notes, and snippets.

@cstrouse
Created March 14, 2012 01:38
Show Gist options
  • Select an option

  • Save cstrouse/2033271 to your computer and use it in GitHub Desktop.

Select an option

Save cstrouse/2033271 to your computer and use it in GitHub Desktop.
Scrape beeradvocate with Python, Requests, and BS
import requests
from BeautifulSoup import BeautifulSoup
soup = BeautifulSoup(requests.get("http://beeradvocate.com/search?q=ipa&qt=beer").content)
# Find the beer links
results = soup.findAll('div')
# How many beers did it find
beer_count = results[12].find('b').string.split(' ')[1]
beer_links = results[13].findAll('li')
links = []
for link in beer_links:
links.append("http://beeradvocate.com%s" % (link.find('a')['href']))
for beer in links:
soup = BeautifulSoup(requests.get(beer).content)
# Beer name
print soup.find('h1', {'class': 'norm'}).string
# Beer rating
print soup.find('span', {'class': 'BAscore_big'}).string
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment