Skip to content

Instantly share code, notes, and snippets.

@sivagao
Last active December 14, 2015 08:59
Show Gist options
  • Select an option

  • Save sivagao/5061672 to your computer and use it in GitHub Desktop.

Select an option

Save sivagao/5061672 to your computer and use it in GitHub Desktop.
py_downloader_list_urls.py. download a list of urls to a location if it does not exist.
import os
import urllib
def ensure_dir(f):
d = os.path.dirname(f)
if not os.path.exists(d):
os.makedirs(d)
DOWNLOADS_DIR = 'F:/1Codes/0Web/bootstrap-awesome' #change window splash
#var xxx = "";[].slice.apply($('link[href]')).forEach(function(e){xxx += e.href; xxx+=','})
#xxx =
prefix = 'WB0663265/'
urls = xxx.split(',')
Durls = [item[item.find(prefix)+len(prefix):len(item)] for item in urls if item.find(prefix) > 0]
# For every line in the file
for name in Durls:
# Combine the name and the downloads directory to get the local filename
filename = os.path.join(DOWNLOADS_DIR, name)
url = 'http://wbpreview.com/previews/WB0663265/' + filename
# Download the file if it does not exist
if not os.path.isfile(filename):
ensure_dir(filename)
urllib.urlretrieve(url, filename)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment