Skip to content

Instantly share code, notes, and snippets.

@a-chernykh
Created June 28, 2014 10:12
Show Gist options
  • Save a-chernykh/c0471b75d0de6e9b4a3a to your computer and use it in GitHub Desktop.
Save a-chernykh/c0471b75d0de6e9b4a3a to your computer and use it in GitHub Desktop.
require 'typhoeus'
def download_typhoeus(urls, concurrency)
hydra = Typhoeus::Hydra.new(max_concurrency: concurrency)
urls.each do |url|
request = Typhoeus::Request.new url
request.on_complete do |response|
write_file url, response.body
end
hydra.queue request
end
hydra.run
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment