Skip to content

Instantly share code, notes, and snippets.

@kaxil
Last active April 12, 2020 17:24
Show Gist options
  • Save kaxil/f32843b3547120e50ec9d91ab7d08eee to your computer and use it in GitHub Desktop.
Save kaxil/f32843b3547120e50ec9d91ab7d08eee to your computer and use it in GitHub Desktop.
Airflow generate dynamic task
# Using DummyOperator
a = []
for i in range(0,10):
a.append(DummyOperator(
task_id='Component'+str(i),
dag=dag))
if i != 0:
a[i-1] >> a[i]
# From a List
sample_list = ["val1", "val2", "val3"]
tasks_list = []
for index, value in enumerate(sample_list):
tasks_list.append(DummyOperator(
task_id='Component'+str(index),
dag=dag))
if index != 0:
tasks_list[index-1] >> tasks_list[index]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment