rampage644

Introduction

This document describes how Airflow jobs (or workflows) get deployed onto production system.

Directory structure

• HOME directory:/home/airflow
• DAG directory: $HOME/airflow-git-dir/dags/
• Config directory: $HOME/airflow-git-dir/configs/
• Unittest directore: $HOME/airflow-git-dir/tests/. Preferable, discoverable by both nose and py.test
• Credentials should be accessed by by some library

rampage644

My Shub Talks

29/07 - Introduce workflow manager

Brief intro

First, I'd like to say hello to everyone and thank for coming.

rampage644

Dataservices spider development process

Disclaimer: Everything described in this document is my personal opinion that doesn't have to be true for everyone.

Common

Key information

rampage644

{}