Created
June 10, 2016 19:09
-
-
Save anonymous/ae10b42c8242e95a3c03ba6c404aec81 to your computer and use it in GitHub Desktop.
Curl custom recursive download example
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Was searching for a certain lyric, couldn't find a site that allowed you to search all of a specific artists lyrics | |
| # thanks to http://apple.stackexchange.com/a/100576 | |
| curl -O -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36" http://www.azlyrics.com/19/50cent.html | |
| for file in $(cat 50cent.html | grep href | sed 's/.*href="//' | sed 's/".*//' | grep '50cent' | grep '\.\.' | sed 's/\.\./www.azlyrics.com/'); do curl -O -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36" $file ; done | |
| grep -nir --color=always watch | less -r | |
| # seems ip got banned after 50 or so requests so maybe need to add some delays |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment