Created
August 12, 2018 21:24
-
-
Save txomon/5ce53233bac6c7e4439d28f1428f23e3 to your computer and use it in GitHub Desktop.
Rate limit python asyncio
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import asyncio | |
import collections | |
async def ratelimit(*, max_request, in_interval): | |
slots = collections.deque() | |
while True: | |
slots.append(time() + in_interval) | |
yield | |
while len(slots) >= max_request: | |
left = slots[0] - time() | |
if left <= 0: | |
slots.popleft() | |
else: | |
await asyncio.sleep(left) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rate_limit = ratelimit(max_request=5, in_interval=10) | |
async def ratelimit_controlled_function(): | |
await rate_limit.__anext__() | |
do_whatever() | |
async def a_lot_of_functions_to_run(): | |
async for _ in rate_limit: | |
await function() |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment