Skip to content

Instantly share code, notes, and snippets.

@kmlawson
Created July 27, 2020 13:32
Show Gist options
  • Save kmlawson/9f4279c0117800b7d71662a4dd4896e6 to your computer and use it in GitHub Desktop.
Save kmlawson/9f4279c0117800b7d71662a4dd4896e6 to your computer and use it in GitHub Desktop.
# Here is a workaround for lack of pause in curl:
# example:
# kurl 20 “http://myurl.com/[1-10].jpg”
# Will download with a 20 second pause
# Assumes $1 is pause in seconds between requests
# Assumes $2 is URL, with [x-y] as range inside it
# Assumes output is .jpg file
if [ -z "$1" ] 2> /dev/null; then
echo "No parameters found. Exiting."
return 1
fi
if [ -z "$2" ] 2> /dev/null; then
echo "No second parameter found with URL. Exiting."
return 1
fi
if ! [ "$1" -eq "$1" ] 2>/dev/null; then
echo "The first parameter does not appear to be an integer. Exiting."
return 1
fi
# Doesn't check if there is a missing or valid [x-y] range in the URL
rangestart=`echo "$2" | gsed -E 's/.*\[([^-]*)-.*/\1/'`
rangeend=`echo "$2" | gsed -E 's/.*\[.*-([^]]*)\].*/\1/'`
i=$rangestart
while [ "$i" -le "$rangeend" ]; do # no variables in {x..y} loop
url=`echo "$2" | gsed -E "s/\[[^]]*\]/$i/"`
echo "Downloading $i of $rangeend."
curl "$url" -o "$i.jpg"
sleep $1
i=$(($i+1))
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment