Skip to content

Instantly share code, notes, and snippets.

@frog32

frog32/DJANGO_TWISTED_HAPROXY.md

Forked from sspross/DJANGO_TWISTED_HAPROXY.md
Last active December 16, 2015 16:59
Show Gist options
  • Save frog32/5466970 to your computer and use it in GitHub Desktop.
Save frog32/5466970 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
celery.conf
[program:YOUR_PROJECT_celery]
command=python manage.py celeryd --loglevel=INFO --beat
environment=PATH="/home/www-data/YOUR-PROJECT/dj/env/bin:/usr/local/bin:/usr/bin:/bin"
directory=/home/www-data/YOUR-PROJECT/dj/
user=www-data
numprocs=1
umask=022
stdout_logfile=/home/www-data/logs/%(program_name)s.log
stdout_logfile_maxbytes=50MB
stdout_logfile_backups=10
stderr_logfile=/home/www-data/logs/%(program_name)s.error.log
stderr_logfile_maxbytes=50MB
stderr_logfile_backups=10
autorestart=True
autostart=True
startsecs=10
stopwaitsecs = 60
priority=998
Raw
DJANGO_TWISTED_HAPROXY.md

Serving Django and Twisted using HAproxy

Why?

Because we wanted to achive the following while developing a webapp using websockets:

Static serving (nginx), Django application (gunicorn) and Websockets service (twisted)

  • on the same IP

    because small vhosts come with one address only.

  • on the same Port 80

    to support companies which are allowing connections to common ports only.

Who?

This setup was mostly created by @frog32 and the guide was written by @sspross. It was developed while working at allink. Warning: This setup is not battle tested and may or may not work for you.

This demo application by @frog32 is using this setup, but is still under havy development.

How?

  1. Project Setup
  2. HAproxy Setup
  3. Gunicorn Setup
  4. Nginx Setup
  5. Twisted Setup
  6. Celery Setup

Django and Twisted

  • Our project structure looks like this:

    • /home/www-data/YOUR-PROJECT/dj/ <- Django Project
    • /home/www-data/YOUR-PROJECT/tx/ <- Twisted Project

  • Installed packages:

    apt-get install csstidy yui-compressor libpng12-dev libjpeg-dev zlib1g-dev python-virtualenv  python-setuptools  libxml2-dev git gettext python-dev libssl-dev mercurial libmysqlclient-dev redis-server rabbitmq-server mysql-server haproxy supervisor nginx node-less

Project Setup

  • Python packages (e.g. REQUIREMENTS_SERVER):

    hiredis
redis
django-redis-cache
gunicorn

  • Add to your settings.py config:

    USE_X_FORWARDED_HOST = True

  • Provide URL in urls.py for HAproxy to enable monitoring:

    url(r'^isrunning$', lambda r: HttpResponse('OK')),

HAproxy Setup

  • Config HAproxy /etc/haproxy/haproxy.cfg, e.g. this.

  • Enable HAproxy, set ENABLED=1 in /etc/default/haproxy (and maybe in /etc/init.d/haproxy) and start it with sudo /etc/init.d/haproxy start. You should see the following if it's really running:

     * Starting haproxy haproxy
   ...done.

  • If your firewall is active, open port for HAproxy stats sudo ufw allow 1936

  • Now you can access HAproxy stats over http://YOUR.IP:1936/ and gunicorn should be up and working.

Gunicorn Setup

  • There has to be a wsgi.py file in /home/www-data/YOUR-PROJECT/dj/, e.g. this.
  • Create supervisor configuration for gunicorn /etc/supervisor/conf.d/YOUR_PROJECT_gunicorn.conf, e.g. this.
  • Don't forget to create a logs directory as configured in the gunicorn config.
  • Update supervisor sudo supervisorctl run update (to restart webserver use restart YOUR_PROJECT_gunicorn)

Nginx Setup

  • Unlink default site sudo unlink /etc/nginx/sites-enabled/default
  • Create an enabled site configuration /etc/nginx/sites-enabled/YOUR-DOMAIN.ch, e.g. this.
  • Don't forget to create a isrunning file like configured e.g. touch /home/www-data/YOUR-PROJECT/dj/isrunning for HAproxy Ping.
  • Restart nginx sudo /etc/init.d/nginx restart
  • Now check HAproxy stats if nginx is pingable and up running http://YOUR.IP:1936/

Twisted Setup

  • Create an additional virtualenv for your Twisted app in /home/www-data/YOUR-PROJECT/tx/ with virtualenv --setuptools env

  • Install python packages:

    txsockjs
pyopenssl
twisted

  • Create another supervisor configuration for twisted /etc/supervisor/conf.d/YOUR_PROJECT_twisted.conf, e.g. this.

  • Update supervisor sudo supervisorctl run update (to restart twisted use restart YOUR_PROJECT_twisted)

  • Now both (gunicorn and twisted) should be running:

    supervisor> status
YOUR_PROJECT_gunicorn             RUNNING    pid 11194, uptime 0:03:34
YOUR_PROJECT_twisted              RUNNING    pid 11231, uptime 0:01:57

  • And everything should be green on the HAproxy status page http://YOUR.IP:1936/.

Celery Setup

  • Add to your settings.py file:

    # broker and celery
BROKER_URL = 'amqp://YOUR_PROJECT:YOUR_PROJECT@localhost:5672/YOUR_PROJECT'
CELERY_RESULT_BACKEND = "redis://localhost/0"
CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"
CELERYD_CONCURRENCY = 1
CELERY_SEND_EVENTS = False
CELERY_ENABLE_UTC = True

import djcelery
djcelery.setup_loader()

  • Create rabbitmq user and vhost:

    sudo rabbitmqctl add_user YOUR_PROJECT YOUR_PROJECT
sudo rabbitmqctl add_vhost YOUR_PROJECT
sudo rabbitmqctl set_permissions -p YOUR_PROJECT YOUR_PROJECT ".*" ".*" ".*"

  • Create another supervisor configuration for celery /etc/supervisor/conf.d/YOUR_PROJECT_celery.conf, e.g. this.

  • Update supervisor sudo supervisorctl run update (to restart celery use restart YOUR_PROJECT_celery)

  • Now all three (gunicorn, twisted and celery) should be running:

    supervisor> status
YOUR_PROJECT_celery               RUNNING    pid 12281, uptime 0:00:13
YOUR_PROJECT_gunicorn             RUNNING    pid 11194, uptime 1:18:23
YOUR_PROJECT_twisted              RUNNING    pid 11231, uptime 1:16:46
Raw
gunicorn.conf
[program:YOUR_PROJECT_gunicorn]
command=/home/www-data/YOUR-PROJECT/dj/env/bin/gunicorn -b 127.0.0.1:9001 wsgi:application
directory=/home/www-data/YOUR-PROJECT/dj
user=www-data
numprocs=1
umask=022
stdout_logfile=/home/www-data/logs/%(program_name)s.log
stdout_logfile_maxbytes=50MB
stdout_logfile_backups=10
stderr_logfile=/home/www-data/logs/%(program_name)s.error.log
stderr_logfile_maxbytes=50MB
stderr_logfile_backups=10
autorestart=true
autostart=True
startsecs=10
stopwaitsecs = 60
priority=998
Raw
haproxy.cfg
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
#log loghost local0 info
maxconn 4096
#chroot /usr/share/haproxy
user haproxy
group haproxy
daemon
#debug
#quiet
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
# reevaluate rules for every request
option http-server-close
# the next line is needed to not fall back to streaming
option http-pretend-keepalive
maxconn 2000
contimeout 5s
clitimeout 60s
srvtimeout 60s
option forwardfor
frontend public
bind :80
timeout client 86400s
acl is_websocket path_beg /ws
#acl is_websocket hdr(Upgrade) -i WebSocket
acl url_static path_beg /static/ /media/
use_backend twisted if is_websocket
use_backend nginx if url_static
default_backend gunicorn
backend twisted
# put the same clients allways on the same server
balance uri depth 2
option httpchk GET /ws/
timeout check 1000ms
#reqrep ^([^\ ]*)\ /ws/(.*) \1\ /\2
server twisted1 127.0.0.1:10001 check inter 50ms
#server twisted2 127.0.0.1:10002 check inter 1000ms
backend nginx
balance leastconn
option httpchk GET /isrunning
timeout check 500ms
timeout client 30s
server nginx1 127.0.0.1:8001 check inter 1s
#server nginx2 127.0.0.1:8002 check inter 1000ms
backend gunicorn
balance leastconn
option httpchk GET /isrunning
timeout check 500ms
timeout client 30s
server gunicorn1 127.0.0.1:9001 check inter 50ms
frontend stats
bind :1936
default_backend stats
backend stats
stats enable
stats hide-version
stats realm HAproxy\Statistics
stats uri /
stats auth YOUR_HAPROXY_USERNAME:YOUR_HAPROXY_PASSWORD
rspidel ^Set-cookie:\ IP= # do not let this cookie tell our internal IP address
#errorloc 502 http://192.168.114.58/error502.html
#errorfile 503 /etc/haproxy/errors/503.http
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
Raw
ngnix-enabled-site.txt
server {
listen 8001;
root /home/www-data/YOUR-PROJECT/dj/;
location /media/ {
}
location /static/ {
}
location /isrunning {
}
location / {
deny all;
}
# what to serve if upstream is not available or crashes
error_page 500 502 503 504 /media/50x.html;
}
Raw
SERVER_SETUP.md

In this guide we used a digitalocean droplet, but it is just an Ubuntu Server and not for production usage!

  1. Enable Automatic Security Updates
  2. Install Logwatch To Keep An Eye On Things
  3. Set Up a Firewall
  4. Switch to www-data User and require Public Key Authentication
  5. Lock Down SSH
  6. Colors for www-data

Ubuntu Server

  • Add your SSH key to your digital ocean control panel
  • Create a droplet (Ubuntu 12.04 x32 Server)
  • Login over ssh -A [email protected]
  • Change root password passwd (you don't have to remember it, you login with your ssh key and you can reset the root password using the control panel anytime)
  • Update distribution apt-get update && apt-get upgrade
  • Install fail2ban apt-get install fail2ban

Enable Automatic Security Updates

  • apt-get install vim unattended-upgrades and edit vim /etc/apt/apt.conf.d/10periodic:

    APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::AutocleanInterval "7";
APT::Periodic::Unattended-Upgrade "1";

Install Logwatch To Keep An Eye On Things

  • apt-get install logwatch and edit vim /etc/cron.daily/00logwatch:

    /usr/sbin/logwatch --output mail --mailto [email protected] --detail high

Set Up a Firewall

  • apt-get install ufw and

    ufw allow 22
ufw allow 80
ufw enable

    (allow only from a specific IP ufw allow from {your-ip} to any port 22, allow https ufw allow 443

Switch to www-data User and require Public Key Authentication

  • Change home folder for user www-data and add your public key to the authorized_keys file:

    mkdir /home/www-data
mkdir /home/www-data/.ssh
chmod 700 /home/www-data/.ssh
vim /home/www-data/.ssh/authorized_keys
chown www-data:www-data /home/www-data -R
usermod -d /home/www-data www-data

  • Activate sudo for www-data:

    passwd www-data

  • In visudo comment all existing user/group grant lines and add:

    root      ALL=(ALL) ALL
www-data  ALL=(ALL) ALL

    Maybe allow www-data to use supervisor without password:

    www-data  ALL=(ALL) NOPASSWD: /usr/bin/supervisorctl

Lock Down SSH

  • Configure ssh to prevent password & root logins vim /etc/ssh/sshd_config:

    PermitRootLogin no
PasswordAuthentication no

  • And restart ssh service ssh restart

Colors for www-data

  • Edit /etc/passwd and change the shell of www-data user from /bin/sh to /bin/bash
  • Copy bashrc and profile files cp /root/.bashrc /home/www-data/ && cp /root/.profile /home/www-data/ and update their owner chown www-data:www-data /home/www-data -R

References

Raw
twisted.conf
[program:YOUR_PROJECT_twisted]
command=/home/www-data/YOUR-PROJECT/tx/env/bin/twistd -n -y service.tac
directory=/home/www-data/YOUR-PROJECT/tx
user=www-data
numprocs=1
umask=022
stdout_logfile=/home/www-data/logs/%(program_name)s.log
stdout_logfile_maxbytes=50MB
stdout_logfile_backups=10
stderr_logfile=/home/www-data/logs/%(program_name)s.error.log
stderr_logfile_maxbytes=50MB
stderr_logfile_backups=10
autorestart=true
autostart=True
startsecs=10
stopwaitsecs = 60
priority=998
Raw
wsgi.py
import os
PROJECT_NAME = 'YOUR_PROJECT'
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "%s.settings" % PROJECT_NAME)
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment