Robert Sanders rssanders3
🏃♂️
rssanders3
/ install-config-apache-airflow-set-fernet-conf.conf
Created
March 25, 2019 07:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [core] | |
| # Secret key to save connection passwords in the db | |
| fernet_key = {FERNET_KEY} |
rssanders3
/ install-config-apache-airflow-initdb.sh
Created
March 25, 2019 07:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| airflow initdb |
rssanders3
/ install-config-apache-airflow-centos-dependencies.sh
Last active
March 28, 2019 19:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| yum -y groupinstall "Development tools" | |
| yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel python-devel wget cyrus-sasl-devel.x86_64 unzip |
rssanders3
/ install-config-apache-airflow-create-dirs.sh
Created
March 25, 2019 08:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cd {AIRFLOW_HOME} | |
| mkdir dags | |
| mkdir logs | |
| mkdir plugins |
rssanders3
/ install-config-apache-airflow-setup-systemd.sh
Created
March 25, 2019 08:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cd /tmp/ | |
| wget https://github.com/apache/airflow/archive/{AIRFLOW_VERSION}.zip | |
| #Example: "wget https://github.com/apache/airflow/archive/1.8.0.zip" |
rssanders3
/ install-config-apache-airflow-unzip-systemd.sh
Created
March 25, 2019 08:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| unzip {AIRFLOW_VERSION}.zip | |
| # This will output extract the contents into: airflow-{AIRFLOW_VERSION} |
rssanders3
/ install-config-apache-airflow-deploy-systemd.sh
Created
March 25, 2019 09:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cd airflow-{AIRFLOW_VERSION}/scripts/systemd/ | |
| # Update the contents of the airflow file. | |
| # Set the AIRFLOW_HOME if its anything other then the default | |
| vi airflow | |
| # Copy the airflow property file to the target location | |
| cp airflow /etc/sysconfig/ | |
| # Update the contents of the airflow-*.service files | |
| # Set the User and Group values to the user and group you want the airflow service to run as | |
| vi airflow-*.service | |
| # Copy the airflow services files to the target location |
rssanders3
/ install-python-3.5.7.sh
Last active
July 1, 2019 17:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cd /opt | |
| sudo wget --no-check-certificate https://www.python.org/ftp/python/3.5.7/Python-3.5.7.tar.xz | |
| tar xf Python-3.5.7.tar.xz | |
| cd Python-3.5.7 | |
| ./configure --prefix=/usr/local | |
| make && make altinstall | |
| ls -ltr /usr/local/bin/python* | |
| vi ~/.bashrc |
rssanders3
/ install-python-mysqlclient-ubuntu.sh
Created
July 1, 2019 17:24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| apt-get install python-dev libmysqlclient-dev | |
| pip install mysqlclient |
rssanders3
/ install-python-mysqlclient-centos.sh
Created
July 1, 2019 17:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| yum install -y mysql-devel python-devel python-setuptools | |
| pip install MySQL-python |