Created
November 3, 2019 21:33
-
-
Save chadfurman/6b045518862a88d2b34e2ebbbf1d664f to your computer and use it in GitHub Desktop.
A short script to download files from a webpage you saved in your browser. Currently set to download PDFs.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import re | |
from urllib.request import urlretrieve | |
from bs4 import BeautifulSoup | |
page = open('index.html', 'r') | |
page_text = page.read() | |
soup = BeautifulSoup(page_text, 'html.parser') | |
zipfiles = soup.find_all('a', href=re.compile('\.pdf')) | |
for z in zipfiles: | |
url = z['href'] | |
parts = url.split('/') | |
dst = parts[-1] | |
print("Retrieving " + dst + ": " + url) | |
try: | |
urlretrieve(url, dst) | |
except: | |
print("Error, skipping.") |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment