Created
April 15, 2013 15:01
-
-
Save smajda/5388745 to your computer and use it in GitHub Desktop.
wget mirror with options I have to google for everytime.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/sh | |
url='http://someurl.com/' | |
wait=2 # wait between each request | |
cutdirs=2 # i.e. if '/foo/bar/data' -> start at 'data' | |
wget \ | |
--execute="robots=off" \ | |
--mirror \ | |
--convert-links \ | |
--no-parent \ | |
--no-host \ | |
--cut-dirs=$cutdirs \ | |
--wait=$wait \ | |
${url} | |
# Other handy options not used here: | |
# --reject index.html* # if you don't actually want index.htmls |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment