Skip to content

Instantly share code, notes, and snippets.

@talos
Last active August 29, 2015 14:25
Show Gist options
  • Save talos/28e735732e81eb98b57c to your computer and use it in GitHub Desktop.
Save talos/28e735732e81eb98b57c to your computer and use it in GitHub Desktop.
parallel wget
#!/bin/bash
# escape sequences http://ascii-table.com/ansi-escape-sequences.php
# http://www.climagic.org/mirrors/VT100_Escape_Codes.html
wget http://www.opendatacache.com/bronx.lehman.cuny.edu/api/views/4iu9-f7ju/rows.csv --progress=bar:force -O rows1.csv 2>log1.log &
wget http://www.opendatacache.com/bronx.lehman.cuny.edu/api/views/4iu9-f7ju/rows.csv --progress=bar:force -O rows2.csv 2>log2.log &
wget http://www.opendatacache.com/bronx.lehman.cuny.edu/api/views/4iu9-f7ju/rows.csv --progress=bar:force -O rows3.csv 2>log3.log &
for logfile in $( ls *.log ); do
echo ''
done
while True; do
echo -e "\033[4A"
for logfile in $( ls *.log ); do
echo -e "\033[K$(tail -n 3 $logfile | grep -m 1 '%') $logfile"
done
if [ -z "$(jobs)" ]; then
# one last update
echo -e "\033[4A"
for logfile in $( ls *.log ); do
echo -e "\033[K$(tail -n 4 $logfile | grep -m 1 '%') $logfile"
done
break
else
jobs >/dev/null
sleep 0.1
fi
done
wait
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment