Skip to content

Instantly share code, notes, and snippets.

@cathoderay

cathoderay/get_urls.py

Created August 19, 2012 02:21
Show Gist options
  • Save cathoderay/3391054 to your computer and use it in GitHub Desktop.
Save cathoderay/3391054 to your computer and use it in GitHub Desktop.
Download ZIP
get urls from an url
Raw
get_urls.py
import requests
from lxml.html import fromstring
def get_urls(url):
return fromstring(requests.get(url).text).xpath('//a/@href')
print get_urls('http://www.google.com')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment