Skip to content

Instantly share code, notes, and snippets.

@rraallvv
Created November 3, 2016 22:56
Show Gist options
  • Save rraallvv/b0a083dcb18f6590ef3cf0264c5e769b to your computer and use it in GitHub Desktop.
Save rraallvv/b0a083dcb18f6590ef3cf0264c5e769b to your computer and use it in GitHub Desktop.
Quick and dirty files downloader
#!/bin/bash
remotepath="http://www.example.com"
urldecode() {
# urldecode <string>
local url_encoded="${1//+/ }"
printf '%b' "${url_encoded//%/\\x}"
}
list="$1"
while read -r line
do
#echo $line
file="${line%\"}"
file="${file#\"}"
file=${file#file://}
file=${file#$PWD}
file=${file#$remotepath}
#file=$(urldecode $file)
#echo $file
dir=$(dirname "${file}")
dir=${dir%/}
#dir=$(echo "$dir" | tr '[:upper:]' '[:lower:]')
#dir=${dir#/}
#echo $dir
filename=$(basename "${file}")
#filename=$(echo "$filename" | tr '[:upper:]' '[:lower:]')
#echo $filename
#echo "$PWD/$dir"
output=".$dir/${filename%\?*}"
#echo $output
if [ ! -f $output ]; then
mkdir -p "$PWD/$dir" && wget "$remotepath$dir/$filename" -O $output
fi
done < "$list"
# remove empty files
find . -size 0 -delete
# remove empty directolries
find . -type d -empty -delete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment