Skip to content

Instantly share code, notes, and snippets.

@yuxel

yuxel/test.sh

Created March 27, 2020 09:44
Show Gist options
  • Save yuxel/47d6587755d59c12d61b22c8b2f92e7d to your computer and use it in GitHub Desktop.
Save yuxel/47d6587755d59c12d61b22c8b2f92e7d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
test.sh
for i in `curl https://mestores.com/sitemap-$1.xml | grep loc | cut -d '>' -f3 | grep ^http | cut -d '<' -f1`; do
echo $i
time curl --silent --output /dev/null --compressed $i -H 'user-agent: Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1'
time curl --silent --output /dev/null --compressed $i -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36'
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment