Skip to content

Instantly share code, notes, and snippets.

@zacbrac

zacbrac/scraper.py

Last active December 12, 2018 23:25
Show Gist options
  • Save zacbrac/4a007e96510a2d81173d to your computer and use it in GitHub Desktop.
Save zacbrac/4a007e96510a2d81173d to your computer and use it in GitHub Desktop.
Download ZIP
Web Scraping with Python
Raw
scraper.py
from bs4 import BeautifulSoup
import urllib3
http = urllib3.PoolManager()
r = http.request('GET', 'https://www.wsj.com/news/technology')
soup = BeautifulSoup(r.data, 'html.parser')
for element in soup.select('h3.wsj-headline'):
print(element.get_text().encode('utf-8').strip())
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment