Proof of concept of distributed rate limiting multiple workers processing speed.
Rate limiting follows a leaky bucket algorithim. The bucket is implemented using a speical token-bucket queue. Max size of the bucket is enforced by using the max length of a token queue. The bucket is refilled by a single worker, which also is responsible for the refill rate.
Workers must get a token before fetching and processing any tasks.