thatgameguy

thatgameguy / spider-seperated-urls.sh

Created December 16, 2011 21:39 — forked from lotsofcode/spider-seperated-urls.sh

Spider response header for comma separated urls

	#!/bin/bash

	THEROOTURL="http://www.domain.com"
	THEPATHS=".php,blur,foo,bar,rar.htm,car.txt"

	IFS=","
	for THEPATH in $THEPATHS; do
	echo "------------------------------------------------------------";
	echo "$THEROOTURL/$THEPATH";
	wget $THEROOTURL/$THEPATH --spider 2>&1 \| grep "HTTP"