Skip to content

Instantly share code, notes, and snippets.

@montenegrodr

montenegrodr/letmecrawl.py

Created February 26, 2018 09:27
Show Gist options
  • Save montenegrodr/985d930490bf22bc1c7a0c762f30b38f to your computer and use it in GitHub Desktop.
Save montenegrodr/985d930490bf22bc1c7a0c762f30b38f to your computer and use it in GitHub Desktop.
Download ZIP
LetMeCrawl — Web Scraping sem ser bloqueado
Raw
letmecrawl.py
import requests
from letmecrawl import letmecrawl
for proxy in letmecrawl():
response = requests.get(
url='http://ifconfig.co/json',
proxies={'http': str(proxy)}
)
print response.content
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment