Last active
April 27, 2018 12:53
-
-
Save nvie/4707624 to your computer and use it in GitHub Desktop.
Which API is the nicest for the new RQ with concurrency?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Alt 1: very explicit, but requires knowledge on what type of workers exist | |
from rq.workers.gevent import GeventWorker | |
w = GeventWorker(4) | |
# Alt 2: A bit like an "Abstract Factory", no need for knowing the implementing | |
# class names, backward compatible, and kind='forking' by default | |
from rq import Worker | |
w = Worker(4, kind='gevent') | |
# Alt 3: I don't think (2) works without really nasty tricks. This one is as | |
# simple as (2), but uses a convenient class method, so we can return an instance | |
# of the implementing subclass instead. Switching worker impls is as easy as | |
# changing a single string. | |
from rq import Worker | |
w = Worker.create('gevent', 4) | |
# Alt 4: A helper function that wraps (1) | |
from rq import make_worker | |
w = make_worker('gevent', 4) # will return a rq.workers.gevent.GeventWorker | |
# with max 4 processes |
I'm really interested in concurrency. Did any of this get implemented?
I'm all curious what the final decision was on this one.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@saulshanabrook: The main use case is that implementing custom worker scripts would be easier. The default
rqworker
script would look something like this:Where the
concurrency_backend
andnum_processes
arguments could be command line flags, like for example:$ rqworker -c gevent -n 4
Implementing custom worker scripts would be just as easy and can use the default
rqworker
script as a reference implementation.