Skip to content

Instantly share code, notes, and snippets.

@ryanlong1004
Created March 16, 2023 17:22
Show Gist options
  • Save ryanlong1004/b7fbd9f24938cec9160c77ada0731157 to your computer and use it in GitHub Desktop.
Save ryanlong1004/b7fbd9f24938cec9160c77ada0731157 to your computer and use it in GitHub Desktop.
Download website with wget

wget -r -np -k https://www.google.com

Other useful options:

-nd (no directories): download all files to the current directory
-e robots=off: ignore restrictions in robots.txt file and don't download robots.txt files
-A png,jpg: accept only files with the extensions png or jpg
-m (mirror): -r --timestamping --level inf --no-remove-listing
-nc, --no-clobber: Skip download if files exist

from https://apple.stackexchange.com/questions/100570/getting-all-files-from-a-web-page-using-curl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment