Skip to content

Instantly share code, notes, and snippets.

@Archenoth
Last active August 29, 2015 14:06
Show Gist options
  • Select an option

  • Save Archenoth/a192353dc7de02b63e5a to your computer and use it in GitHub Desktop.

Select an option

Save Archenoth/a192353dc7de02b63e5a to your computer and use it in GitHub Desktop.
A little Perl script to scan a webpage for PDFs and download them all.
#!/usr/bin/perl
#
# Site scraping PDF downloader: By Matthew (Archenoth) MacLean.
#
# This downloader will scan for all PDF links on the URL you pass in
# and download them to their respective files in your current working
# directory.
#
# You will need to install the Crypt::SSLeay CPAN module to download PDFs
# on HTTPS servers.
use warnings;
use strict;
use LWP::Simple;
print "Usage $0 <URL To download PDFs from>\n" and exit if !@ARGV;
my $root = $ARGV[0] =~ s/\/[^\/]+$/\//r;
for(get($ARGV[0]) =~ m/(?:http:|\.\.|\/)[^"']+\.pdf/gi){
print "Cannot download from $ARGV[0]\n" and exit if not defined $_;
print "Downloading " . (my $name = (/([^'"\/]+\.pdf)/i)[0]) . "\n";
if(s/^[\.\/]/$root./){
(my $size =()= m/\.\./g)++;
for(my $i = 0; $i < $size; $i++) {
s/\/[^\/\.]+[^\/]+\/\.\.//;
}
}
print "Download failed for $_!\n\n" if is_error getstore $_, $name;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment