Skip to content

Instantly share code, notes, and snippets.

@gregburek
Created December 7, 2011 01:51
Show Gist options
  • Save gregburek/1441055 to your computer and use it in GitHub Desktop.
Save gregburek/1441055 to your computer and use it in GitHub Desktop.
Rate limiting function calls with Python Decorators
import time
def RateLimited(maxPerSecond):
minInterval = 1.0 / float(maxPerSecond)
def decorate(func):
lastTimeCalled = [0.0]
def rateLimitedFunction(*args,**kargs):
elapsed = time.clock() - lastTimeCalled[0]
leftToWait = minInterval - elapsed
if leftToWait>0:
time.sleep(leftToWait)
ret = func(*args,**kargs)
lastTimeCalled[0] = time.clock()
return ret
return rateLimitedFunction
return decorate
@RateLimited(2) # 2 per second at most
def PrintNumber(num):
print num
if __name__ == "__main__":
print "This should print 1,2,3... at about 2 per second."
for i in range(1,100):
PrintNumber(i)
@3lixy
Copy link

3lixy commented May 19, 2018

@oPromessa thanks for sharing and nice job on the logging and example.

@slidenerd
Copy link

Why dont you use a counter instead of a list. Each time the function is called, increment the counter by 1 and reset the counter to 0 when the second or minute has changed

@luizfzs
Copy link

luizfzs commented Mar 19, 2020

Any updates regarding licensing of these pieces of code?

@codekiln
Copy link

Note, there are other libraries on PyPI that do this, such as ratelimit

@hugokernel
Copy link

A more simple version but with async compatibility https://gist.github.com/hugokernel/c478d639849e1e772b1395a546100031

@yeus
Copy link

yeus commented Dec 2, 2021

I created a similar asyncio version but using the token bucket algorithm for
max number of calls during/time interval which results in a maximum burst rate. it can be combined with the above rate limiter without burst rate..

https://gist.github.com/yeus/dff02dce88c6da9073425b5309f524dd

@teocns
Copy link

teocns commented Sep 28, 2023

An extended version of @oPromessa 's utility. Thank you for sharing @oPromessa !

@martinandrovich
Copy link

I added Python 3.12 generic type-hinting and support for recursive calls using an RLock:

from typing import Callable, cast
from functools import wraps
import threading
import time


def rate_limited[T, **P](max_per_second: int) -> Callable[[Callable[P, T]], Callable[P, T]]:

    def decorator(fn: Callable[P, T]) -> Callable[P, T]:

        lock = threading.RLock()
        min_interval = 1.0 / max_per_second
        last_time_called = 0.0

        @wraps(fn)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
            with lock:
                nonlocal last_time_called
                elapsed = time.perf_counter() - last_time_called
                left_to_wait = min_interval - elapsed
                if left_to_wait > 0:
                    time.sleep(left_to_wait)

                last_time_called = time.perf_counter()
                return fn(*args, **kwargs)

        return cast(Callable[P, T], wrapper)

    return decorator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment