Skip to content

Instantly share code, notes, and snippets.

@dstrelau
Created November 5, 2009 22:36
Show Gist options
  • Save dstrelau/227472 to your computer and use it in GitHub Desktop.
Save dstrelau/227472 to your computer and use it in GitHub Desktop.
#!/usr/bin/env ruby
require 'typhoeus'
HOST = 'http://yourhost.com'
CONCURRENTS = 5
urls = File.readlines(ARGV.shift)
hydra = Typhoeus::Hydra.new
queue_req = lambda do |resp|
return if urls == []
url = HOST + urls.pop
puts "Processing: #{url}"
req = Typhoeus::Request.new(url)
req.on_complete(&queue_req)
hydra.queue(req)
end
CONCURRENTS.times { queue_req.call(0) }
hydra.run
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment