Rahul Pande rahul-pande
rahul-pande
/ pg_download_operator.py
Created
September 5, 2016 07:48
Airflow Postgres Download Operator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Airflow Operator to download results of a sql query to a file on the worker | |
| # Pass chunksize parameter to download large tables without the | |
| # worker running out of memory | |
| import logging | |
| from airflow.hooks.postgres_hook import PostgresHook | |
| from airflow.models import BaseOperator | |
| from airflow.utils.decorators import apply_defaults |