Skip to content

Instantly share code, notes, and snippets.

@a-chernykh
Created June 28, 2014 10:12
Show Gist options
  • Save a-chernykh/cefe4c9540925dd46524 to your computer and use it in GitHub Desktop.
Save a-chernykh/cefe4c9540925dd46524 to your computer and use it in GitHub Desktop.
require 'em-http'
def download_em_http(urls, concurrency)
EventMachine.run do
multi = EventMachine::MultiRequest.new
EM::Iterator.new(urls, concurrency).each do |url, iterator|
req = EventMachine::HttpRequest.new(url).get
req.callback do
write_file url, req.response
iterator.next
end
multi.add url, req
multi.callback { EventMachine.stop } if url == urls.last
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment