gistfile1.txt

========================================================
Deployment information for http://myrepresentatives.com.
========================================================
Note::
Much of this was derived from @jacobian's Django deployment workshop:
    http://github.com/jacobian/django-deployment-workshop/blob/master/notes/pycon-2010-script.txt

Possible Deployment Options::
1. Run virtual domains on nginx and pass domain header, pointing to 1 instance of apache w/ virtual domains as usual
2. Run virtual domains on nginx, w/ multiple instance of apache (on different ports)
(This guide assumes option 1)

Setup
=====
Boot a cloud server & update packages

    aptitude update
    aptitude upgrade

Better tab completion 
    aptitude install bash-completion
    echo ". /etc/bash_completion" >> .bashrc


1. Getting Started
==================
    aptitude install git-core python-dev postgresql-dev postgresql-client build-essential libpq-dev mercurial python-setuptools

Get the GeoDjango-specific libraries::
    aptitude install binutils libgdal1-1.6.0 postgresql-8.4-postgis postgresql-server-dev-8.4 python-psycopg2 gdal-bin libgeoip1 python-gdal libproj-dev

    # Update the PROJ datum shifting files (looks like this is OK in Ubuntu 10.4)
    #wget http://download.osgeo.org/proj/proj-datumgrid-1.4.tar.gz
    #mkdir nad; cd nad
    #tar xzf ../proj-datumgrid-1.4.tar.gz
    #nad2bin null < null.lla
    #sudo cp null /usr/share/proj

Set up the app directories::
    mkdir -p /home/web/static
    mkdir -p /home/web/representproject
    
Virtualenv + virtualenvwrapper::
    mkdir -p /home/web/.virtualenvs
    easy_install virtualenv
    easy_install virtualenvwrapper
    easy_install pip

Edit .bashrc and add the following::
    export WORKON_HOME=/home/web/.virtualenvs
    source /usr/local/bin/virtualenvwrapper.sh

    See the virtualenvwrapper docs here:
    http://www.doughellmann.com/docs/virtualenvwrapper/

    source ~/.bashrc

Create a Virtualenv::
    mkvirtualenv represent --distribute
OR
    mkvirtualenv represent --no-site-packages --distribute

Install Django and Necessary Python packages::
    workon represent 
   
    #TODO: This should all go in a pip requirements file
    pip install django
    pip install psycopg2 # if using --no-site-packages
    pip install django-registration
    pip install geopy

    #TODO: write setup.py for represent so it can be installed with
    pip install django-represent-X.X.tar.gz

    # Until setup.py works, move code to server
    scp -r * root@<host>:/home/web/representproject/

2. Install Caching
====================
aptitude install python-memcache memcached libmemcache-dev

3. PostgreSQL
=============
Install PostgreSQL::
    aptitude install postgresql

#TODO: revise the postgresql configs (leaving defaults for now)
#Kill Postgresql and change it's settings::
#    invoke-rc.d postgresql-8.4 stop
#    cd /etc/postgresql/8.4/main/
#    mv postgresql.conf postgresql.conf.orig
#    mv pg_hba.conf pg_hba.conf.orig
#
#    ln -s /home/web/representproject/postgres/pg_hba.conf .
#    ln -s /home/web/representproject/postgres/postgresql.conf .
#
#Make sure the conf files have the correct IP addresses::
#    invoke-rc.d postgresql-8.4 start

Allow password-less login to PostgreSQL::
    1) edit /etc/postgresql/8.4/main/pg_hba.conf 
    2) Change:
            local all postgres ident
       To:
            local all all trust

Create DB users, including a root user for convenience::
    su postgres 
    createuser root
    createuser django_user
    
Create a PostGIS spatial database template::
This script is available from: http://geodjango.org/docs/create_template_postgis-debian.sh

    #POSTGIS_SQL_PATH=/usr/share/postgresql-8.4-postgis/
    POSTGIS_SQL_PATH=/usr/share/postgresql/8.4/contrib
    
    createdb -E UTF8 template_postgis # Create the template spatial database.
    #### IF The above fails with an encoding error, use the following:
    createdb -T template0 -E UTF8 template_postgis

    createlang -d template_postgis plpgsql # Adding PLPGSQL language support.
    psql -d postgres -c "UPDATE pg_database SET datistemplate='true' WHERE datname='template_postgis';"

    psql -d template_postgis -f $POSTGIS_SQL_PATH/postgis.sql # Loading the PostGIS SQL routines
    psql -d template_postgis -f $POSTGIS_SQL_PATH/spatial_ref_sys.sql
    psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;" # Enabling users to alter spatial tables.
    psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"

Create a Spatial Database::
<<<<<<< local
    createdb -T template_postgis <dbname> 
    createdb -T template_postgis represent
=======
    createdb -T template_postgis -O <owner-username> -U <dbuser> <dbname> 
    createdb -T template_postgis -O django_user -U django_user represent
>>>>>>> other

    # Make sure it worked:
    psql -l 
    cd /home/web/representproject
    
    ### Be sure to set up the correct info in settings.py, Django 1.2
    DATABASES = {
        'default': {
            'ENGINE': 'django.contrib.gis.db.backends.postgis',
            'NAME': 'represent',
            'USER': 'django_user',
            'PASSWORD': '',
            'HOST': '',
            'PORT': '',
        }
    }
    
Edit settings.py and set DEBUG=False
    python manage.py syncdb


4. Install/Configure Apache & mod_wsgi
=======================================
NOTE: On Ubuntu, make sure ``universe`` is available (see /etc/apt/sources.list)
    aptitude install apache2 libapache2-mod-wsgi


Backup the original settings::
    cd /etc/apache2/
    tar -cvf ~/original_apache2_settings.tar
    gzip ~/original_apache2_settings.tar

NOTE: trying to remove the original apache settings didn't work for me,
      So,just editing the stuff in sites-available
# Remove the rest::
#     rm -rf apache2.conf conf.d/ httpd.conf magic mods-* sites-* ports.conf
# Include the custom apache config's (and the src code)::
#     ln -s /home/web/representproject/apache/apache2.conf .
# Remove the apche modules I don't need from mods-enabled (NOTE: authz_ stuff is required for Order Directives)::
#     cd /etc/apache2/mods-enabled
#     rm auth_basic.load authn_file.load 
#     rm autoindex.conf cgid.conf deflate.conf dir.conf env.load negotiation.load setenvif.load status.load autoindex.load 
#     rm cgid.load deflate.load dir.load negotiation.conf setenvif.conf status.conf

Install mod_wsgi::
    aptitude install libapache2-mod-wsgi

Make sure mod-wsgi stuff is there, if not add it::
    ln -s ../mods-available/wsgi.conf .
    ln -s ../mods-available/wsgi.load .

Add the python-eggs directory (which should be writeable by apache)::
    mkdir -m775 -p /var/www/.python-eggs
    sudo chown -R www-data:www-data /var/www/.python-eggs/

Restart Apache::
    invoke-rc.d apache2 restart

Make sure apache works::
    curl -I http://whatever.com

Set up Apache site files; NOTE: most of the work here involves making sure the .wsgi file includes the correct path::
    cd /etc/apache2/sites-enabled/
    ln -s /home/web/representproject/apache/sites-available/myrepresentatives-com .
    invoke-rc.d apache2 restart

IF the app is not pip-installable (and therefore in your virtualenv), you can put it in the project directory, 
BUT it will need to be on your PYTHONPATH in the .wsgi file.

Finally, Make sure Django's static media is available:
    cd /home/web/static
    ln -s /home/web/.virtualenvs/representproject/lib/python2.6/site-packages/django/contrib/admin/media .
    chown -R root:www-data media

Now, hit http://myrepresentative.com to see if everything works :)

Set Apache config's so they server on port 8000 rather than 80::
    change ports.conf and set <VirtualHost *:8000> in all virtual host files.

5. Install/Configure nginx
==========================

    aptitude install nginx
    cd /etc/nginx/

Remove all the default configs, and link to the custom config::
    rm -rf conf.d/ fastcgi_params koi-* nginx.conf sites-* win-utf
    ln -s /home/web/representproject/nginx/nginx.conf .

Restart nginx::
    invoke-rc.d nginx restart

Make sure nginx can see the site_media::
    cd /home/web/static/
    ln -s ../representproject/media/ site_media

Double-check that media is being served by nginx (server should say "nginx")::
    curl -I http://myrepresentatives.com/media/css/base.css
    curl -I http://myrepresentatives.com/site_media/css/screen.css

6. Load Data
============
Load any Shape files needed. (see represent/load.py)::
    python manage.py shell
    >>> from represent import load
    >>> load.run()

### NOTE: pgpool is not working :(
###6. Install Database connection middleware (pgpool2)
###===================================================
###NOTE: this information comes right from @jacobian's django deployment workshop:
###
###Install pgpool2. There's a bug in the current Ubuntu we'll need to work around
###with a custom ``/etc/default/pgpool2``::
###
###    aptitude install pgpool2
###    pkill pgpool
###    rm /etc/default/pgpool2
###    ln -s /home/web/django-deployment-workshop/pgpool/etc-default-pgpool /etc/default/pgpool2
###    rm /etc/pgpool.conf
###    ln -s /home/web/django-deployment-workshop/pgpool/pgpool-single-db.conf /etc/pgpool.conf
###
###Fix up ``pgpool.conf`` to point to the backend (``db1``). Start it and check::
###
###    vim /etc/pgpool.conf
###    invoke-rc.d pgpool2 start
###    psql -l
###
###Repeat on any other webservers.
###
###Modify Django to use pgpool::
###
###    DATABASE_HOST = ''
###
###Again, remember both hosts. Reload:
###
###    web1$ invoke-rc.d apache2 reload
###    web2$ invoke-rc.d apache2 reload
###
###Check it out.


7. Database Redundancy 
======================
(see jacob's notes)

8. Set up an MTA (for email delivery)
=====================================
    aptitude install postfix

Make sure postfix does not listen to outside connections::
    cd /etc/postfix
    vim master.cf

    Comment out the following line:
    smtp      inet  n       -       -       -       -       smtpd

    invoke-rc.d postfix restart

9. Set up a firewall (blocking everything from outside except 22, 80 and maybe 8000)
====================================================================================
TODO