Skip to content

Instantly share code, notes, and snippets.

@Romern
Created June 16, 2020 13:49
Show Gist options
  • Save Romern/f384064f7210752e5cf325b05cb82463 to your computer and use it in GitHub Desktop.
Save Romern/f384064f7210752e5cf325b05cb82463 to your computer and use it in GitHub Desktop.
Bash script which downloads files from archive.org
#!/bin/bash
if [ $# -le 0 ]; then
echo "Usage: $0 url (outputfilename)"
exit
fi
if [ $# -eq 2 ]; then
outputfilename="--output-document=$2"
fi
json=$(curl -s "http://web.archive.org/cdx/search/cdx?url=$1&limit=2&output=json")
if [ "$json" = "[]" ]; then
echo "Not found"
exit
fi
datestring=$(echo $json | jq .[1][1] | sed -e 's/^"//' -e 's/"$//')
url=$(echo $json | jq .[1][2] | sed -e 's/^"//' -e 's/"$//')
finalurl="https://web.archive.org/web/${datestring}if_/$url"
wget "$outputfilename" "$finalurl"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment