Skip to content

Instantly share code, notes, and snippets.

@lotsofcode
Created September 26, 2011 15:49
Show Gist options
  • Select an option

  • Save lotsofcode/1242554 to your computer and use it in GitHub Desktop.

Select an option

Save lotsofcode/1242554 to your computer and use it in GitHub Desktop.
Spider response header for comma separated urls
#!/bin/bash
THEROOTURL="http://www.domain.com"
THEPATHS="foo,bar,rar.htm,car.txt"
IFS=","
for THEPATH in $THEPATHS; do
echo "------------------------------------------------------------";
echo "$THEROOTURL/$THEPATH";
wget $THEROOTURL/$THEPATH --spider 2>&1 | grep "HTTP"
done;
@thatgameguy
Copy link

Could this be more generic?

@lotsofcode
Copy link
Author

Better?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment