Skip to content

Instantly share code, notes, and snippets.

@jmizgajski
Created May 10, 2014 13:28
Show Gist options
  • Save jmizgajski/3dc2db9d6321781f7b2b to your computer and use it in GitHub Desktop.
Save jmizgajski/3dc2db9d6321781f7b2b to your computer and use it in GitHub Desktop.
Chaining two groups of tasks in celery.
#task is a task that is not interested in any results from other tasks
#callback is a task you want to call after each group is finished (ex. log time)
# and has to take a list of results as first arguement
@app.task(base=Task)
def task(i):
#do stuff
pass
@app.task(base=Task)
def callback(results):
#do stuff with results i.e. log total time of distributed processing
pass
chain(
chord([task.si(x) for x in [1,2,3]], body=callback.s(), immutable=True),
chord([task.si(x) for x in [5,6,7]], body=callback.s(), immutable=True)
)
@miklagard
Copy link

Thank you for the trick :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment