Skip to content

Instantly share code, notes, and snippets.

@erikbern
Created March 18, 2014 20:17
Show Gist options
  • Save erikbern/9628609 to your computer and use it in GitHub Desktop.
Save erikbern/9628609 to your computer and use it in GitHub Desktop.
import luigi
# Here we are importing our own tasks, provided they are
# arranged in a python module (folder) named "components"
from components.SomeTaskA import SomeTaskA
from components.SomeTaskB import SomeTaskB
from components.SomeTaskC import SomeTaskC
# ------------------------------------------
# DEFINE THE MAIN WORKFLOW DEPENDENCY GRAPH
# ------------------------------------------
class MyTaskB(SomeTaskB):
def requires(self):
return SomeTaskA()
class MyTaskC(SomeTaskC):
def requires(self):
return SomeTaskB()
# Build (and run) the workflow
luigi.build(MyTaskC(), workers=16, local_scheduler=True)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment