Skip to content

Instantly share code, notes, and snippets.

View tridungle's full-sized avatar

Trí Dũng Lê tridungle

  • Independent
  • Ho Chi Minh
View GitHub Profile
@tridungle
tridungle / 82467.geojson
Created May 26, 2021 10:17 — forked from coding-chimp/82467.geojson
Building large GraphQL response is slow
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
#!/bin/bash
USUARIO_SO="$(whoami)"
ANACONDA_URL="https://repo.anaconda.com/archive/Anaconda3-2020.11-Linux-x86_64.sh"
_DB_PASSWORD="aig2Chie"
_IP=$(hostname -I | cut -d' ' -f1)
while getopts "a:p:h" opt; do
case $opt in
a) ANACONDA_URL="$OPTARG";;
p) _DB_PASSWORD="$OPTARG";;
h) cat <<EOF
@tridungle
tridungle / airflow-ec2
Created May 26, 2021 10:06 — forked from qi-qi/airflow-ec2
Airflow-ec2
sudo apt-get update && sudo apt-get -y upgrade
sudo pip3 install --upgrade apache-airflow[all]==1.10.2
sudo apt-get install -y curl build-essential autoconf libtool pkg-config git libxml2-dev libxslt1-dev libkrb5-dev libsasl2-dev libssl-dev libffi-dev locales netcat apt-utils libblas-dev libmysqlclient-dev libpq-dev liblapack-dev supervisor net-tools
sudo apt-get install -y python3.6-dev python3-distutils python3-kerberos python3-jenkins
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo python3 get-pip.py
# systemd configuration for master scheduler
[Unit]
Description=Airflow webserver daemon
After=rabbitmq-server.service
Wants=rabbitmq-server.service
[Service]
WorkingDirectory=/opt/airflow
Environment="AIRFLOW_HOME=/opt/airflow"
@tridungle
tridungle / deployment-scheduler.yaml
Created May 26, 2021 10:03 — forked from holypriest/deployment-scheduler.yaml
Deployment configuration for the Airflow scheduler
kind: Deployment
apiVersion: apps/v1
metadata:
name: airflow-scheduler
namespace: airflow-on-k8s
spec:
replicas: 1
selector:
matchLabels:
tier: airflow
#!/bin/bash
USUARIO_SO="$(whoami)"
ANACONDA_URL="https://repo.anaconda.com/miniconda/Miniconda3-py37_4.9.2-Linux-x86_64.sh"
_DB_PASSWORD="aig2Chie"
_IP=$(hostname -I | cut -d' ' -f1)
while getopts "a:p:h" opt; do
case $opt in
a) ANACONDA_URL="$OPTARG";;
p) _DB_PASSWORD="$OPTARG";;
h) cat <<EOF
@tridungle
tridungle / docker-compose.yml
Created May 26, 2021 10:02 — forked from bayborodin/docker-compose.yml
Apache Airflow Docker Compose
version: '3'
volumes:
postgres_data:
x-airflow-common:
&airflow-common
image: apache/airflow:2.0.1
environment:
- AIRFLOW__CORE__EXECUTOR=LocalExecutor
@tridungle
tridungle / TIGexecutor.yml
Created May 26, 2021 10:02 — forked from saurabhkdm/TIGexecutor.yml
TIG stack for airlfow..
version: '2.1'
services:
redis:
image: 'redis:5.0.5'
command: redis-server --requirepass redispass
postgres:
image: postgres:9.6
environment:
- POSTGRES_USER=airflow
#!/bin/sh
#instalação pacotes iniciais do sistema operacional
apt-get update
apt-get upgrade -y
apt-get autoremove
apt-get autocleanapt-get -y install build-essential binutils gcc make git htop nethogs tmux
#Instalação do postgres
apt-get -y install postgresql postgresql-contrib libpq-dev postgresql-client postgresql-client-common
@tridungle
tridungle / airflow-python3.sh
Created May 26, 2021 10:00 — forked from zacgx/airflow-python3.sh
Installing Airflow with CeleryExcuter, using PostgreSQL as metadata database and Redis for Celery message broker
# this script has been tested and worked in a freshly installed Ubuntu 16.04 and 16.10
# it assumes that you are running airflow in a private netowrk and no need to be worry about outside access
# if that's not the case, the lines for PostgreSQL and Redis in this script need to be updated accordingly
# run as root
sudo su
# initial system updates and installs
apt-get update && apt-get upgrade -y && apt-get autoremove && apt-get autoclean
apt-get -y install build-essential binutils gcc make git htop nethogs tmux