Skip to content

Instantly share code, notes, and snippets.

@enginebai

enginebai/ipeen_crawler1.py

Created September 3, 2016 03:01
Show Gist options
  • Save enginebai/69530aa7292f3f37cfb71d937b6e6ff9 to your computer and use it in GitHub Desktop.
Save enginebai/69530aa7292f3f37cfb71d937b6e6ff9 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
ipeen_crawler1.py
def get_shop_link_list():
list_req = requests.get(LIST_URL)
if list_req.status_code == requests.codes.ok:
soup = BeautifulSoup(list_req.content, HTML_PARSER)
shop_links_a_tags = soup.find_all('a', attrs={'data-label': '店名'})
shop_links = []
for link in shop_links_a_tags:
print(ROOT_URL + link['href'])
shop_links.append(ROOT_URL + link['href'])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment