Skip to content

Instantly share code, notes, and snippets.

@thatgameguy
Forked from lotsofcode/spider-seperated-urls.sh
Created December 16, 2011 21:39
Show Gist options
  • Select an option

  • Save thatgameguy/1488118 to your computer and use it in GitHub Desktop.

Select an option

Save thatgameguy/1488118 to your computer and use it in GitHub Desktop.
Spider response header for comma separated urls
#!/bin/bash
THEROOTURL="http://www.domain.com"
THEPATHS=".php,blur,foo,bar,rar.htm,car.txt"
IFS=","
for THEPATH in $THEPATHS; do
echo "------------------------------------------------------------";
echo "$THEROOTURL/$THEPATH";
wget $THEROOTURL/$THEPATH --spider 2>&1 | grep "HTTP"
done;
@lotsofcode

Copy link
Copy Markdown

Haha. Perfect, that's what I like to see.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment