Skip to content

Instantly share code, notes, and snippets.

@innermond

innermond/parallel wget from a list

Last active February 10, 2016 14:57
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save innermond/ac55b6ee96b453493e41 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save innermond/ac55b6ee96b453493e41 to your computer and use it in GitHub Desktop.
Download ZIP
download from a list with wget in parallel
Raw
parallel wget from a list
cat .list | xargs -I % -P 10 wget -t 3 --header="Accept: text/html" --user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:21.0) Gecko/20100101 Firefox/21.0" ‐‐referer="https://www.google.com" %
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment