Created
November 14, 2012 10:34
-
-
Save shesek/4071443 to your computer and use it in GitHub Desktop.
node.js mikeal/request with concurrency limit
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# I've been getting "socket hangup" errors when sending too many | |
# concurrent HTTP requests with mikeal/request library, so... | |
request_concurrent = (max) -> | |
queue = [] | |
running = 0 | |
r = (a..., cb) -> | |
# Queue the request if we have too many running | |
return queue.push [a..., cb] if running >= max | |
++running | |
request a..., (resp...) -> | |
--running | |
# Run the next queued request, if we have one | |
r queue.shift()... if queue.length | |
cb resp... | |
# Usage | |
req = request_concurrent 15 | |
# And just use `req()` like you would normally use `request()` | |
# (but without the sugary .get()/.post()/etc methods, tho they | |
# can be easily added by setting the returned req function | |
# [[Prototype]] to `request`.) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It seems like node has a pool of http clients, which should already set a limit on concurrent requests. I'm not sure if it was really my issue, but it seems to have solved it.