Skip to content

Instantly share code, notes, and snippets.

@simonw

simonw/bots-from-twitter.txt

Created November 29, 2009 00:28
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save simonw/244718 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save simonw/244718 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
bots-from-twitter.txt
174.129.50.201 - - [28/Nov/2009:18:20:45 -0600] "HEAD / HTTP/1.0" 200 - "-" "@hourlypress"
70.32.121.67 - - [29/Nov/2009:00:20:46 +0000] "GET / HTTP/1.0" 200 8826 "-" "Java/1.6.0"
66.249.71.186 - - [28/Nov/2009:18:20:48 -0600] "GET / HTTP/1.0" 200 3110 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
208.74.66.39 - - [29/Nov/2009:00:20:49 +0000] "GET / HTTP/1.0" 200 8832 "-" "Mozilla/5.0 (compatible; Butterfly/1.0; +http://labs.topsy.com/butterfly.html) Gecko/2009032608 Firefox/3.0.8"
174.129.107.161 - - [28/Nov/2009:18:20:50 -0600] "HEAD / HTTP/1.0" 200 - "-" "PycURL/7.18.2"
174.129.89.199 - - [29/Nov/2009:00:20:51 +0000] "GET / HTTP/1.0" 200 8797 "-" "Python-urllib/2.5"
174.129.58.57 - - [28/Nov/2009:18:20:53 -0600] "HEAD / HTTP/1.0" 200 - "-" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"
89.151.84.35 - - [29/Nov/2009:00:20:53 +0000] "GET / HTTP/1.0" 200 8912 "-" "Mozilla/5.0 (compatible; MSIE 6.0b; Windows NT 5.0) Gecko/2009011913 Firefox/3.0.6 TweetmemeBot"
89.151.84.35 - - [28/Nov/2009:18:20:53 -0600] "GET / HTTP/1.0" 200 8900 "-" "Mozilla/5.0 (compatible; MSIE 6.0b; Windows NT 5.0) Gecko/2009011913 Firefox/3.0.6 TweetmemeBot"
89.151.84.35 - - [29/Nov/2009:00:20:53 +0000] "GET / HTTP/1.0" 200 8775 "-" "Mozilla/5.0 (compatible; MSIE 6.0b; Windows NT 5.0) Gecko/2009011913 Firefox/3.0.6 TweetmemeBot"
174.129.95.55 - - [28/Nov/2009:18:20:54 -0600] "GET / HTTP/1.0" 200 8839 "-" "bitlybot"
89.151.84.35 - - [29/Nov/2009:00:20:54 +0000] "GET / HTTP/1.0" 200 8937 "-" "Mozilla/5.0 (compatible; MSIE 6.0b; Windows NT 5.0) Gecko/2009011913 Firefox/3.0.6 TweetmemeBot"
79.125.48.47 - - [28/Nov/2009:18:21:01 -0600] "HEAD / HTTP/1.0" 200 - "-" "Mozilla/5.0 (compatible; uriplay/1.0; +http://uriplay.org)"
98.210.159.9 - - [29/Nov/2009:00:21:06 +0000] "GET / HTTP/1.0" 200 3142 "-" "EventBox/582 CFNetwork/454.5 Darwin/10.2.0 (i386) (MacBookPro5%2C5)"
174.129.141.109 - - [29/Nov/2009:00:21:13 +0000] "HEAD / HTTP/1.0" 200 - "-" "PostRank/2.0 (postrank.com)"
72.44.43.230 - - [28/Nov/2009:18:21:25 -0600] "HEAD / HTTP/1.0" 200 - "-" "LWP::Simple/5.808"
72.44.43.230 - - [29/Nov/2009:00:21:25 +0000] "GET / HTTP/1.0" 200 8878 "-" "LWP::Simple/5.808"
75.101.205.207 - - [28/Nov/2009:18:21:39 -0600] "GET / HTTP/1.0" 200 8768 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
174.143.213.49 - - [28/Nov/2009:18:21:42 -0600] "HEAD / HTTP/1.0" 200 - "http://twitturls.com/" "Mozilla/5.0 (compatible; Twitturls; +http://twitturls.com)"
174.143.213.49 - - [29/Nov/2009:00:21:42 +0000] "HEAD / HTTP/1.0" 200 - "http://twitturls.com/" "Mozilla/5.0 (compatible; Twitturls; +http://twitturls.com)"
174.143.213.49 - - [28/Nov/2009:18:21:42 -0600] "HEAD / HTTP/1.0" 200 - "http://twitturls.com/" "Mozilla/5.0 (compatible; Twitturls; +http://twitturls.com)"
174.143.213.49 - - [29/Nov/2009:00:21:42 +0000] "HEAD / HTTP/1.0" 200 - "http://twitturls.com/" "Mozilla/5.0 (compatible; Twitturls; +http://twitturls.com)"
174.129.199.128 - - [28/Nov/2009:18:21:51 -0600] "GET / HTTP/1.0" 200 8821 "-" "-"
82.132.139.203 - - [29/Nov/2009:00:22:05 +0000] "HEAD / HTTP/1.0" 200 - "-" "TwitterFon/107 CFNetwork/459 Darwin/10.0.0d3"
212.92.18.15 - - [29/Nov/2009:00:22:18 +0000] "GET / HTTP/1.0" 200 8763 "-" "Jakarta Commons-HttpClient/3.1"
87.74.15.2 - - [29/Nov/2009:00:22:19 +0000] "GET / HTTP/1.0" 200 3111 "-" "Socialite/968 CFNetwork/454.5 Darwin/10.2.0 (i386) (MacBook1%2C1)"
75.101.232.27 - - [28/Nov/2009:18:22:20 -0600] "GET / HTTP/1.0" 200 8863 "-" "MetaURI API +metauri.com"
87.74.15.2 - - [29/Nov/2009:00:22:23 +0000] "GET / HTTP/1.0" 200 3100 "-" "Socialite/968 CFNetwork/454.5 Darwin/10.2.0 (i386) (MacBook1%2C1)"
89.151.116.59 - - [29/Nov/2009:00:22:37 +0000] "GET / HTTP/1.0" 200 8832 "-" "Mozilla/5.0 (compatible; MSIE 6.0b; Windows NT 5.0) Gecko/2009011913 Firefox/3.0.6 TweetmemeBot"
70.37.66.245 - - [28/Nov/2009:18:22:45 -0600] "GET / HTTP/1.0" 200 3132 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)"
174.129.22.161 - - [28/Nov/2009:18:23:01 -0600] "GET / HTTP/1.0" 200 8937 "-" "Jakarta Commons-HttpClient/3.0.1"
174.133.61.66 - - [29/Nov/2009:00:23:21 +0000] "GET / HTTP/1.0" 200 8824 "-" "MOZILLA/5.0 (WINDOWS; U; WINDOWS NT 5.1; EN-US; RV:1.9.0.3) GECKO/2008092417 FIREFOX/3.0.3"
64.233.172.20 - - [28/Nov/2009:18:23:24 -0600] "GET / HTTP/1.0" 200 3060 "-" "Python-urllib/1.17 AppEngine-Google; (+http://code.google.com/appengine; appid: therealurl)"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment