Skip to content

Instantly share code, notes, and snippets.

@ibnesayeed
Created December 4, 2017 00:29
Show Gist options
  • Select an option

  • Save ibnesayeed/68fb3cdcdf1de0a0abb6ae750dcb4eca to your computer and use it in GitHub Desktop.

Select an option

Save ibnesayeed/68fb3cdcdf1de0a0abb6ae750dcb4eca to your computer and use it in GitHub Desktop.
A basic page scraping script that prints out all the hyper references of the given web page.
#!/usr/bin/env python
import sys
import requests
from bs4 import BeautifulSoup
res = requests.get(sys.argv[-1])
soup = BeautifulSoup(res.text, "html.parser")
for link in soup.find_all("a"):
print(link.get("href"))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment