Skip to content

Instantly share code, notes, and snippets.

@MatrixManAtYrService

MatrixManAtYrService/dag.py

Created April 30, 2022 00:49
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save MatrixManAtYrService/76d7a061b1fb81ad29e8ba1c8b20c1bd to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save MatrixManAtYrService/76d7a061b1fb81ad29e8ba1c8b20c1bd to your computer and use it in GitHub Desktop.
Download ZIP
Raw
dag.py
from datetime import datetime
from airflow import DAG
from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import (
KubernetesPodOperator,
)
from airflow.configuration import conf
namespace = conf.get("kubernetes", "NAMESPACE")
with DAG(
dag_id="kpo_mapped_sleepfirst",
start_date=datetime(1970, 1, 1),
schedule_interval=None,
doc_md="for testing reattach_on_restart, restart scheduler before 200 seconds is up and task should complete",
) as dag:
KubernetesPodOperator(
task_id="cowsay_static",
name="cowsay_statc",
namespace=namespace,
image="docker.io/rancher/cowsay",
cmds=["sh"],
arguments=["-c", "sleep 200 && cowsay moo"],
reattach_on_restart=True,
log_events_on_failure=True,
)
KubernetesPodOperator.partial(
task_id="cowsay_mapped",
name="cowsay_mapped",
namespace=namespace,
image="docker.io/rancher/cowsay",
cmds=["sh"],
log_events_on_failure=True,
reattach_on_restart=True,
).expand(
# oops, I forgot the `-c` here, which is probably the cause for the error
arguments=[
[f"sleep 200 && cowsay {x}"] for x in ["mooooove", "cow", "get out the way"]
]
)
Raw
mapped task logs (ugly)
*** Reading local file: /usr/local/airflow/logs/dag_id=kpo_mapped_sleepfirst/run_id=manual__2022-04-30T00:38:31+00:00/task_id=cowsay_mapped/map_index=0/attempt=1.log
[2022-04-30, 00:38:34 UTC] {taskinstance.py:1158} INFO - Dependencies all met for <TaskInstance: kpo_mapped_sleepfirst.cowsay_mapped manual__2022-04-30T00:38:31+00:00 map_index=0 [queued]>
[2022-04-30, 00:38:34 UTC] {taskinstance.py:1158} INFO - Dependencies all met for <TaskInstance: kpo_mapped_sleepfirst.cowsay_mapped manual__2022-04-30T00:38:31+00:00 map_index=0 [queued]>
[2022-04-30, 00:38:34 UTC] {taskinstance.py:1355} INFO -
--------------------------------------------------------------------------------
[2022-04-30, 00:38:34 UTC] {taskinstance.py:1356} INFO - Starting attempt 1 of 1
[2022-04-30, 00:38:34 UTC] {taskinstance.py:1357} INFO -
--------------------------------------------------------------------------------
[2022-04-30, 00:38:34 UTC] {taskinstance.py:1376} INFO - Executing <Mapped(KubernetesPodOperator): cowsay_mapped> on 2022-04-30 00:38:31+00:00
[2022-04-30, 00:38:34 UTC] {standard_task_runner.py:52} INFO - Started process 10 to run task
[2022-04-30, 00:38:34 UTC] {standard_task_runner.py:79} INFO - Running: ['airflow', 'tasks', 'run', 'kpo_mapped_sleepfirst', 'cowsay_mapped', 'manual__2022-04-30T00:38:31+00:00', '--job-id', '4', '--raw', '--subdir', 'DAGS_FOLDER/dags/taskmap/kpo_mapped_sleepfirst.py', '--cfg-path', '/tmp/tmpv5xz9qsn', '--map-index', '0', '--error-file', '/tmp/tmpveaejky7']
[2022-04-30, 00:38:34 UTC] {standard_task_runner.py:80} INFO - Job 4: Subtask cowsay_mapped
[2022-04-30, 00:38:34 UTC] {task_command.py:369} INFO - Running <TaskInstance: kpo_mapped_sleepfirst.cowsay_mapped manual__2022-04-30T00:38:31+00:00 map_index=0 [running]> on host kpomappedsleepfirstcowsaymappe-ccf4aff731f540f3af294883fedf2a77
[2022-04-30, 00:38:34 UTC] {taskinstance.py:1568} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=kpo_mapped_sleepfirst
AIRFLOW_CTX_TASK_ID=cowsay_mapped
AIRFLOW_CTX_EXECUTION_DATE=2022-04-30T00:38:31+00:00
AIRFLOW_CTX_TRY_NUMBER=1
AIRFLOW_CTX_DAG_RUN_ID=manual__2022-04-30T00:38:31+00:00
[2022-04-30, 00:38:34 UTC] {kubernetes_pod.py:530} INFO - Creating pod cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff with labels: {'dag_id': 'kpo_mapped_sleepfirst', 'task_id': 'cowsay_mapped', 'run_id': 'manual__2022-04-30T0038310000-67ef4d9e9', 'kubernetes_pod_operator': 'True', 'map_index': '0', 'try_number': '1'}
[2022-04-30, 00:38:34 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff
[2022-04-30, 00:38:35 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff
[2022-04-30, 00:38:36 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff
[2022-04-30, 00:38:37 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff
[2022-04-30, 00:38:38 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff
[2022-04-30, 00:38:39 UTC] {pod_manager.py:199} INFO - sh: can't open 'sleep 200 && cowsay mooooove': No such file or directory
[2022-04-30, 00:38:39 UTC] {kubernetes_pod.py:410} ERROR - Pod Event: Scheduled - Successfully assigned tb11c-inner-kpo-mapped-sleepfirst/cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff to ip-172-28-45-24
[2022-04-30, 00:38:39 UTC] {kubernetes_pod.py:410} ERROR - Pod Event: Pulling - Pulling image "docker.io/rancher/cowsay"
[2022-04-30, 00:38:39 UTC] {kubernetes_pod.py:410} ERROR - Pod Event: Pulled - Successfully pulled image "docker.io/rancher/cowsay" in 179.029563ms
[2022-04-30, 00:38:39 UTC] {kubernetes_pod.py:410} ERROR - Pod Event: Created - Created container base
[2022-04-30, 00:38:39 UTC] {kubernetes_pod.py:410} ERROR - Pod Event: Started - Started container base
[2022-04-30, 00:38:39 UTC] {kubernetes_pod.py:423} INFO - Deleting pod: cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff
[2022-04-30, 00:38:39 UTC] {taskinstance.py:1888} ERROR - Task failed with exception
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 394, in execute
self.cleanup(
File "/usr/local/lib/python3.9/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 416, in cleanup
raise AirflowException(f'Pod {pod and pod.metadata.name} returned a failure: {remote_pod}')
airflow.exceptions.AirflowException: Pod cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff returned a failure: {'api_version': 'v1',
'kind': 'Pod',
'metadata': {'annotations': {'cni.projectcalico.org/podIP': '',
'cni.projectcalico.org/podIPs': ''},
'cluster_name': None,
'creation_timestamp': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal()),
'deletion_grace_period_seconds': None,
'deletion_timestamp': None,
'finalizers': None,
'generate_name': None,
'generation': None,
'labels': {'airflow_version': '2.3.0.dev465-astro.1',
'dag_id': 'kpo_mapped_sleepfirst',
'kubernetes_executor': 'False',
'kubernetes_pod_operator': 'True',
'map_index': '0',
'run_id': 'manual__2022-04-30T0038310000-67ef4d9e9',
'task_id': 'cowsay_mapped',
'try_number': '1'},
'managed_fields': [{'api_version': 'v1',
'fields_type': 'FieldsV1',
'fields_v1': {'f:metadata': {'f:labels': {'.': {},
'f:airflow_version': {},
'f:dag_id': {},
'f:kubernetes_executor': {},
'f:kubernetes_pod_operator': {},
'f:map_index': {},
'f:run_id': {},
'f:task_id': {},
'f:try_number': {}}},
'f:spec': {'f:affinity': {},
'f:containers': {'k:{"name":"base"}': {'.': {},
'f:args': {},
'f:command': {},
'f:image': {},
'f:imagePullPolicy': {},
'f:name': {},
'f:resources': {},
'f:terminationMessagePath': {},
'f:terminationMessagePolicy': {}}},
'f:dnsPolicy': {},
'f:enableServiceLinks': {},
'f:restartPolicy': {},
'f:schedulerName': {},
'f:securityContext': {},
'f:terminationGracePeriodSeconds': {}}},
'manager': 'OpenAPI-Generator',
'operation': 'Update',
'subresource': None,
'time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal())},
{'api_version': 'v1',
'fields_type': 'FieldsV1',
'fields_v1': {'f:metadata': {'f:annotations': {'.': {},
'f:cni.projectcalico.org/podIP': {},
'f:cni.projectcalico.org/podIPs': {}}}},
'manager': 'calico',
'operation': 'Update',
'subresource': 'status',
'time': datetime.datetime(2022, 4, 30, 0, 38, 38, tzinfo=tzlocal())},
{'api_version': 'v1',
'fields_type': 'FieldsV1',
'fields_v1': {'f:status': {'f:conditions': {'k:{"type":"ContainersReady"}': {'.': {},
'f:lastProbeTime': {},
'f:lastTransitionTime': {},
'f:message': {},
'f:reason': {},
'f:status': {},
'f:type': {}},
'k:{"type":"Initialized"}': {'.': {},
'f:lastProbeTime': {},
'f:lastTransitionTime': {},
'f:status': {},
'f:type': {}},
'k:{"type":"Ready"}': {'.': {},
'f:lastProbeTime': {},
'f:lastTransitionTime': {},
'f:message': {},
'f:reason': {},
'f:status': {},
'f:type': {}}},
'f:containerStatuses': {},
'f:hostIP': {},
'f:phase': {},
'f:podIP': {},
'f:podIPs': {'.': {},
'k:{"ip":"10.1.53.109"}': {'.': {},
'f:ip': {}}},
'f:startTime': {}}},
'manager': 'kubelite',
'operation': 'Update',
'subresource': 'status',
'time': datetime.datetime(2022, 4, 30, 0, 38, 39, tzinfo=tzlocal())}],
'name': 'cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff',
'namespace': 'tb11c-inner-kpo-mapped-sleepfirst',
'owner_references': None,
'resource_version': '8733',
'self_link': '/api/v1/namespaces/tb11c-inner-kpo-mapped-sleepfirst/pods/cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff',
'uid': 'dae6fbf2-89f8-458e-bfbc-afa8801ff067'},
'spec': {'active_deadline_seconds': None,
'affinity': {'node_affinity': None,
'pod_affinity': None,
'pod_anti_affinity': None},
'automount_service_account_token': None,
'containers': [{'args': ['sleep 200 && cowsay mooooove'],
'command': ['sh'],
'env': None,
'env_from': None,
'image': 'docker.io/rancher/cowsay',
'image_pull_policy': 'Always',
'lifecycle': None,
'liveness_probe': None,
'name': 'base',
'ports': None,
'readiness_probe': None,
'resources': {'limits': None, 'requests': None},
'security_context': None,
'startup_probe': None,
'stdin': None,
'stdin_once': None,
'termination_message_path': '/dev/termination-log',
'termination_message_policy': 'File',
'tty': None,
'volume_devices': None,
'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount',
'mount_propagation': None,
'name': 'kube-api-access-qf7rc',
'read_only': True,
'sub_path': None,
'sub_path_expr': None}],
'working_dir': None}],
'dns_config': None,
'dns_policy': 'ClusterFirst',
'enable_service_links': True,
'ephemeral_containers': None,
'host_aliases': None,
'host_ipc': None,
'host_network': None,
'host_pid': None,
'hostname': None,
'image_pull_secrets': None,
'init_containers': None,
'node_name': 'ip-172-28-45-24',
'node_selector': None,
'os': None,
'overhead': None,
'preemption_policy': 'PreemptLowerPriority',
'priority': 0,
'priority_class_name': None,
'readiness_gates': None,
'restart_policy': 'Never',
'runtime_class_name': None,
'scheduler_name': 'default-scheduler',
'security_context': {'fs_group': None,
'fs_group_change_policy': None,
'run_as_group': None,
'run_as_non_root': None,
'run_as_user': None,
'se_linux_options': None,
'seccomp_profile': None,
'supplemental_groups': None,
'sysctls': None,
'windows_options': None},
'service_account': 'default',
'service_account_name': 'default',
'set_hostname_as_fqdn': None,
'share_process_namespace': None,
'subdomain': None,
'termination_grace_period_seconds': 30,
'tolerations': [{'effect': 'NoExecute',
'key': 'node.kubernetes.io/not-ready',
'operator': 'Exists',
'toleration_seconds': 300,
'value': None},
{'effect': 'NoExecute',
'key': 'node.kubernetes.io/unreachable',
'operator': 'Exists',
'toleration_seconds': 300,
'value': None}],
'topology_spread_constraints': None,
'volumes': [{'aws_elastic_block_store': None,
'azure_disk': None,
'azure_file': None,
'cephfs': None,
'cinder': None,
'config_map': None,
'csi': None,
'downward_api': None,
'empty_dir': None,
'ephemeral': None,
'fc': None,
'flex_volume': None,
'flocker': None,
'gce_persistent_disk': None,
'git_repo': None,
'glusterfs': None,
'host_path': None,
'iscsi': None,
'name': 'kube-api-access-qf7rc',
'nfs': None,
'persistent_volume_claim': None,
'photon_persistent_disk': None,
'portworx_volume': None,
'projected': {'default_mode': 420,
'sources': [{'config_map': None,
'downward_api': None,
'secret': None,
'service_account_token': {'audience': None,
'expiration_seconds': 3607,
'path': 'token'}},
{'config_map': {'items': [{'key': 'ca.crt',
'mode': None,
'path': 'ca.crt'}],
'name': 'kube-root-ca.crt',
'optional': None},
'downward_api': None,
'secret': None,
'service_account_token': None},
{'config_map': None,
'downward_api': {'items': [{'field_ref': {'api_version': 'v1',
'field_path': 'metadata.namespace'},
'mode': None,
'path': 'namespace',
'resource_field_ref': None}]},
'secret': None,
'service_account_token': None}]},
'quobyte': None,
'rbd': None,
'scale_io': None,
'secret': None,
'storageos': None,
'vsphere_volume': None}]},
'status': {'conditions': [{'last_probe_time': None,
'last_transition_time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal()),
'message': None,
'reason': None,
'status': 'True',
'type': 'Initialized'},
{'last_probe_time': None,
'last_transition_time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal()),
'message': 'containers with unready status: [base]',
'reason': 'ContainersNotReady',
'status': 'False',
'type': 'Ready'},
{'last_probe_time': None,
'last_transition_time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal()),
'message': 'containers with unready status: [base]',
'reason': 'ContainersNotReady',
'status': 'False',
'type': 'ContainersReady'},
{'last_probe_time': None,
'last_transition_time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal()),
'message': None,
'reason': None,
'status': 'True',
'type': 'PodScheduled'}],
'container_statuses': [{'container_id': 'containerd://a337a889cedd47d6a9f079c0315cfa0f6f26a098880544a970977231f7e93248',
'image': 'docker.io/rancher/cowsay:latest',
'image_id': 'docker.io/rancher/cowsay@sha256:5dab61268bc18daf56febb5a856b618961cd806dbc49a22a636128ca26f0bd94',
'last_state': {'running': None,
'terminated': None,
'waiting': None},
'name': 'base',
'ready': False,
'restart_count': 0,
'started': False,
'state': {'running': None,
'terminated': {'container_id': 'containerd://a337a889cedd47d6a9f079c0315cfa0f6f26a098880544a970977231f7e93248',
'exit_code': 2,
'finished_at': datetime.datetime(2022, 4, 30, 0, 38, 38, tzinfo=tzlocal()),
'message': None,
'reason': 'Error',
'signal': None,
'started_at': datetime.datetime(2022, 4, 30, 0, 38, 38, tzinfo=tzlocal())},
'waiting': None}}],
'ephemeral_container_statuses': None,
'host_ip': '172.28.45.24',
'init_container_statuses': None,
'message': None,
'nominated_node_name': None,
'phase': 'Failed',
'pod_i_ps': [{'ip': '10.1.53.109'}],
'pod_ip': '10.1.53.109',
'qos_class': 'BestEffort',
'reason': None,
'start_time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal())}}
[2022-04-30, 00:38:39 UTC] {taskinstance.py:1394} INFO - Marking task as FAILED. dag_id=kpo_mapped_sleepfirst, task_id=cowsay_mapped, map_index=0, execution_date=20220430T003831, start_date=20220430T003834, end_date=20220430T003839
[2022-04-30, 00:38:39 UTC] {standard_task_runner.py:92} ERROR - Failed to execute job 4 for task cowsay_mapped (Pod cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff returned a failure: {'api_version': 'v1',
'kind': 'Pod',
'metadata': {'annotations': {'cni.projectcalico.org/podIP': '',
'cni.projectcalico.org/podIPs': ''},
'cluster_name': None,
'creation_timestamp': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal()),
'deletion_grace_period_seconds': None,
'deletion_timestamp': None,
'finalizers': None,
'generate_name': None,
'generation': None,
'labels': {'airflow_version': '2.3.0.dev465-astro.1',
'dag_id': 'kpo_mapped_sleepfirst',
'kubernetes_executor': 'False',
'kubernetes_pod_operator': 'True',
'map_index': '0',
'run_id': 'manual__2022-04-30T0038310000-67ef4d9e9',
'task_id': 'cowsay_mapped',
'try_number': '1'},
'managed_fields': [{'api_version': 'v1',
'fields_type': 'FieldsV1',
'fields_v1': {'f:metadata': {'f:labels': {'.': {},
'f:airflow_version': {},
'f:dag_id': {},
'f:kubernetes_executor': {},
'f:kubernetes_pod_operator': {},
'f:map_index': {},
'f:run_id': {},
'f:task_id': {},
'f:try_number': {}}},
'f:spec': {'f:affinity': {},
'f:containers': {'k:{"name":"base"}': {'.': {},
'f:args': {},
'f:command': {},
'f:image': {},
'f:imagePullPolicy': {},
'f:name': {},
'f:resources': {},
'f:terminationMessagePath': {},
'f:terminationMessagePolicy': {}}},
'f:dnsPolicy': {},
'f:enableServiceLinks': {},
'f:restartPolicy': {},
'f:schedulerName': {},
'f:securityContext': {},
'f:terminationGracePeriodSeconds': {}}},
'manager': 'OpenAPI-Generator',
'operation': 'Update',
'subresource': None,
'time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal())},
{'api_version': 'v1',
'fields_type': 'FieldsV1',
'fields_v1': {'f:metadata': {'f:annotations': {'.': {},
'f:cni.projectcalico.org/podIP': {},
'f:cni.projectcalico.org/podIPs': {}}}},
'manager': 'calico',
'operation': 'Update',
'subresource': 'status',
'time': datetime.datetime(2022, 4, 30, 0, 38, 38, tzinfo=tzlocal())},
{'api_version': 'v1',
'fields_type': 'FieldsV1',
'fields_v1': {'f:status': {'f:conditions': {'k:{"type":"ContainersReady"}': {'.': {},
'f:lastProbeTime': {},
'f:lastTransitionTime': {},
'f:message': {},
'f:reason': {},
'f:status': {},
'f:type': {}},
'k:{"type":"Initialized"}': {'.': {},
'f:lastProbeTime': {},
'f:lastTransitionTime': {},
'f:status': {},
'f:type': {}},
'k:{"type":"Ready"}': {'.': {},
'f:lastProbeTime': {},
'f:lastTransitionTime': {},
'f:message': {},
'f:reason': {},
'f:status': {},
'f:type': {}}},
'f:containerStatuses': {},
'f:hostIP': {},
'f:phase': {},
'f:podIP': {},
'f:podIPs': {'.': {},
'k:{"ip":"10.1.53.109"}': {'.': {},
'f:ip': {}}},
'f:startTime': {}}},
'manager': 'kubelite',
'operation': 'Update',
'subresource': 'status',
'time': datetime.datetime(2022, 4, 30, 0, 38, 39, tzinfo=tzlocal())}],
'name': 'cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff',
'namespace': 'tb11c-inner-kpo-mapped-sleepfirst',
'owner_references': None,
'resource_version': '8733',
'self_link': '/api/v1/namespaces/tb11c-inner-kpo-mapped-sleepfirst/pods/cowsay-mapped-236b13e9a8a2422cbda74f0defb5cbff',
'uid': 'dae6fbf2-89f8-458e-bfbc-afa8801ff067'},
'spec': {'active_deadline_seconds': None,
'affinity': {'node_affinity': None,
'pod_affinity': None,
'pod_anti_affinity': None},
'automount_service_account_token': None,
'containers': [{'args': ['sleep 200 && cowsay mooooove'],
'command': ['sh'],
'env': None,
'env_from': None,
'image': 'docker.io/rancher/cowsay',
'image_pull_policy': 'Always',
'lifecycle': None,
'liveness_probe': None,
'name': 'base',
'ports': None,
'readiness_probe': None,
'resources': {'limits': None, 'requests': None},
'security_context': None,
'startup_probe': None,
'stdin': None,
'stdin_once': None,
'termination_message_path': '/dev/termination-log',
'termination_message_policy': 'File',
'tty': None,
'volume_devices': None,
'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount',
'mount_propagation': None,
'name': 'kube-api-access-qf7rc',
'read_only': True,
'sub_path': None,
'sub_path_expr': None}],
'working_dir': None}],
'dns_config': None,
'dns_policy': 'ClusterFirst',
'enable_service_links': True,
'ephemeral_containers': None,
'host_aliases': None,
'host_ipc': None,
'host_network': None,
'host_pid': None,
'hostname': None,
'image_pull_secrets': None,
'init_containers': None,
'node_name': 'ip-172-28-45-24',
'node_selector': None,
'os': None,
'overhead': None,
'preemption_policy': 'PreemptLowerPriority',
'priority': 0,
'priority_class_name': None,
'readiness_gates': None,
'restart_policy': 'Never',
'runtime_class_name': None,
'scheduler_name': 'default-scheduler',
'security_context': {'fs_group': None,
'fs_group_change_policy': None,
'run_as_group': None,
'run_as_non_root': None,
'run_as_user': None,
'se_linux_options': None,
'seccomp_profile': None,
'supplemental_groups': None,
'sysctls': None,
'windows_options': None},
'service_account': 'default',
'service_account_name': 'default',
'set_hostname_as_fqdn': None,
'share_process_namespace': None,
'subdomain': None,
'termination_grace_period_seconds': 30,
'tolerations': [{'effect': 'NoExecute',
'key': 'node.kubernetes.io/not-ready',
'operator': 'Exists',
'toleration_seconds': 300,
'value': None},
{'effect': 'NoExecute',
'key': 'node.kubernetes.io/unreachable',
'operator': 'Exists',
'toleration_seconds': 300,
'value': None}],
'topology_spread_constraints': None,
'volumes': [{'aws_elastic_block_store': None,
'azure_disk': None,
'azure_file': None,
'cephfs': None,
'cinder': None,
'config_map': None,
'csi': None,
'downward_api': None,
'empty_dir': None,
'ephemeral': None,
'fc': None,
'flex_volume': None,
'flocker': None,
'gce_persistent_disk': None,
'git_repo': None,
'glusterfs': None,
'host_path': None,
'iscsi': None,
'name': 'kube-api-access-qf7rc',
'nfs': None,
'persistent_volume_claim': None,
'photon_persistent_disk': None,
'portworx_volume': None,
'projected': {'default_mode': 420,
'sources': [{'config_map': None,
'downward_api': None,
'secret': None,
'service_account_token': {'audience': None,
'expiration_seconds': 3607,
'path': 'token'}},
{'config_map': {'items': [{'key': 'ca.crt',
'mode': None,
'path': 'ca.crt'}],
'name': 'kube-root-ca.crt',
'optional': None},
'downward_api': None,
'secret': None,
'service_account_token': None},
{'config_map': None,
'downward_api': {'items': [{'field_ref': {'api_version': 'v1',
'field_path': 'metadata.namespace'},
'mode': None,
'path': 'namespace',
'resource_field_ref': None}]},
'secret': None,
'service_account_token': None}]},
'quobyte': None,
'rbd': None,
'scale_io': None,
'secret': None,
'storageos': None,
'vsphere_volume': None}]},
'status': {'conditions': [{'last_probe_time': None,
'last_transition_time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal()),
'message': None,
'reason': None,
'status': 'True',
'type': 'Initialized'},
{'last_probe_time': None,
'last_transition_time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal()),
'message': 'containers with unready status: [base]',
'reason': 'ContainersNotReady',
'status': 'False',
'type': 'Ready'},
{'last_probe_time': None,
'last_transition_time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal()),
'message': 'containers with unready status: [base]',
'reason': 'ContainersNotReady',
'status': 'False',
'type': 'ContainersReady'},
{'last_probe_time': None,
'last_transition_time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal()),
'message': None,
'reason': None,
'status': 'True',
'type': 'PodScheduled'}],
'container_statuses': [{'container_id': 'containerd://a337a889cedd47d6a9f079c0315cfa0f6f26a098880544a970977231f7e93248',
'image': 'docker.io/rancher/cowsay:latest',
'image_id': 'docker.io/rancher/cowsay@sha256:5dab61268bc18daf56febb5a856b618961cd806dbc49a22a636128ca26f0bd94',
'last_state': {'running': None,
'terminated': None,
'waiting': None},
'name': 'base',
'ready': False,
'restart_count': 0,
'started': False,
'state': {'running': None,
'terminated': {'container_id': 'containerd://a337a889cedd47d6a9f079c0315cfa0f6f26a098880544a970977231f7e93248',
'exit_code': 2,
'finished_at': datetime.datetime(2022, 4, 30, 0, 38, 38, tzinfo=tzlocal()),
'message': None,
'reason': 'Error',
'signal': None,
'started_at': datetime.datetime(2022, 4, 30, 0, 38, 38, tzinfo=tzlocal())},
'waiting': None}}],
'ephemeral_container_statuses': None,
'host_ip': '172.28.45.24',
'init_container_statuses': None,
'message': None,
'nominated_node_name': None,
'phase': 'Failed',
'pod_i_ps': [{'ip': '10.1.53.109'}],
'pod_ip': '10.1.53.109',
'qos_class': 'BestEffort',
'reason': None,
'start_time': datetime.datetime(2022, 4, 30, 0, 38, 34, tzinfo=tzlocal())}}; 10)
[2022-04-30, 00:38:39 UTC] {local_task_job.py:156} INFO - Task exited with return code 1
[2022-04-30, 00:38:39 UTC] {local_task_job.py:273} INFO - 0 downstream tasks scheduled from follow-on schedule check
Raw
static task logs (happy)
*** Reading local file: /usr/local/airflow/logs/dag_id=kpo_mapped_sleepfirst/run_id=manual__2022-04-30T00:38:31+00:00/task_id=cowsay_static/attempt=1.log
[2022-04-30, 00:38:40 UTC] {taskinstance.py:1158} INFO - Dependencies all met for <TaskInstance: kpo_mapped_sleepfirst.cowsay_static manual__2022-04-30T00:38:31+00:00 [queued]>
[2022-04-30, 00:38:40 UTC] {taskinstance.py:1158} INFO - Dependencies all met for <TaskInstance: kpo_mapped_sleepfirst.cowsay_static manual__2022-04-30T00:38:31+00:00 [queued]>
[2022-04-30, 00:38:40 UTC] {taskinstance.py:1355} INFO -
--------------------------------------------------------------------------------
[2022-04-30, 00:38:40 UTC] {taskinstance.py:1356} INFO - Starting attempt 1 of 1
[2022-04-30, 00:38:40 UTC] {taskinstance.py:1357} INFO -
--------------------------------------------------------------------------------
[2022-04-30, 00:38:40 UTC] {taskinstance.py:1376} INFO - Executing <Task(KubernetesPodOperator): cowsay_static> on 2022-04-30 00:38:31+00:00
[2022-04-30, 00:38:40 UTC] {standard_task_runner.py:52} INFO - Started process 10 to run task
[2022-04-30, 00:38:40 UTC] {standard_task_runner.py:79} INFO - Running: ['airflow', 'tasks', 'run', 'kpo_mapped_sleepfirst', 'cowsay_static', 'manual__2022-04-30T00:38:31+00:00', '--job-id', '6', '--raw', '--subdir', 'DAGS_FOLDER/dags/taskmap/kpo_mapped_sleepfirst.py', '--cfg-path', '/tmp/tmpjql3svvc', '--error-file', '/tmp/tmp2vdyv1hs']
[2022-04-30, 00:38:40 UTC] {standard_task_runner.py:80} INFO - Job 6: Subtask cowsay_static
[2022-04-30, 00:38:40 UTC] {task_command.py:369} INFO - Running <TaskInstance: kpo_mapped_sleepfirst.cowsay_static manual__2022-04-30T00:38:31+00:00 [running]> on host kpomappedsleepfirstcowsaystati-74d59059b8a446799f0b0c6931233dcd
[2022-04-30, 00:38:40 UTC] {taskinstance.py:1568} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=kpo_mapped_sleepfirst
AIRFLOW_CTX_TASK_ID=cowsay_static
AIRFLOW_CTX_EXECUTION_DATE=2022-04-30T00:38:31+00:00
AIRFLOW_CTX_TRY_NUMBER=1
AIRFLOW_CTX_DAG_RUN_ID=manual__2022-04-30T00:38:31+00:00
[2022-04-30, 00:38:40 UTC] {kubernetes_pod.py:530} INFO - Creating pod cowsay-statc-912e6d252a6645198dd7b0eab315c05d with labels: {'dag_id': 'kpo_mapped_sleepfirst', 'task_id': 'cowsay_static', 'run_id': 'manual__2022-04-30T0038310000-67ef4d9e9', 'kubernetes_pod_operator': 'True', 'try_number': '1'}
[2022-04-30, 00:38:40 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-statc-912e6d252a6645198dd7b0eab315c05d
[2022-04-30, 00:38:41 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-statc-912e6d252a6645198dd7b0eab315c05d
[2022-04-30, 00:38:42 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-statc-912e6d252a6645198dd7b0eab315c05d
[2022-04-30, 00:38:43 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-statc-912e6d252a6645198dd7b0eab315c05d
[2022-04-30, 00:38:44 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-statc-912e6d252a6645198dd7b0eab315c05d
[2022-04-30, 00:38:45 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-statc-912e6d252a6645198dd7b0eab315c05d
[2022-04-30, 00:38:46 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-statc-912e6d252a6645198dd7b0eab315c05d
[2022-04-30, 00:38:47 UTC] {pod_manager.py:159} WARNING - Pod not yet started: cowsay-statc-912e6d252a6645198dd7b0eab315c05d
[2022-04-30, 00:42:05 UTC] {pod_manager.py:199} INFO - _____
[2022-04-30, 00:42:05 UTC] {pod_manager.py:199} INFO - < moo >
[2022-04-30, 00:42:05 UTC] {pod_manager.py:199} INFO - -----
[2022-04-30, 00:42:05 UTC] {pod_manager.py:199} INFO - \ ^__^
[2022-04-30, 00:42:05 UTC] {pod_manager.py:199} INFO - \ (oo)\_______
[2022-04-30, 00:42:05 UTC] {pod_manager.py:199} INFO - (__)\ )\/\
[2022-04-30, 00:42:05 UTC] {pod_manager.py:199} INFO - ||----w |
[2022-04-30, 00:42:05 UTC] {pod_manager.py:199} INFO - || ||
[2022-04-30, 00:42:06 UTC] {kubernetes_pod.py:423} INFO - Deleting pod: cowsay-statc-912e6d252a6645198dd7b0eab315c05d
[2022-04-30, 00:42:06 UTC] {taskinstance.py:1394} INFO - Marking task as SUCCESS. dag_id=kpo_mapped_sleepfirst, task_id=cowsay_static, execution_date=20220430T003831, start_date=20220430T003840, end_date=20220430T004206
[2022-04-30, 00:42:06 UTC] {local_task_job.py:156} INFO - Task exited with return code 0
[2022-04-30, 00:42:06 UTC] {local_task_job.py:273} INFO - 0 downstream tasks scheduled from follow-on schedule check
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment