How to get Apache Airflow running on a bare AWS EC2 node

Amazon Managed Workflows for Apache Airflow (AWS MWAA) is prohibitively expensive for someone tinkering around on a personal account, and unprepared to pay a few hundred dollars a month at a minimum for their smallest environment. Here are the commands I used to get it running on Ubuntu 20.

Prepare the box

sudo apt-get update

Get Python

sudo apt-get install python3-pip
pip3 install --upgrade pip==20.2.4

Download Airflow

This home directory is actually the default, but I figured I'd put it here since it's probably relevant.

export AIRFLOW_HOME=~/airflow
pip3 install apache-airflow

Then disconnect and reconnect to have your shell restarted.

Initialize Airflow

airflow db init
airflow users create \
     --username philihp \
     --firstname Philihp \
     --lastname Busby \
     --role Admin \
     --email [email protected]

Start Processes

Do these on two separate terminals.

airflow webserver --port 8080

airflow scheduler