Django + Heroku + Django_tutorials + S3

This is how I actually got all this stuff to work together. It'll attrophy over time but as of May 2013 it works. Maybe I'll keep it up to date.

-- [email protected]

On the shoulders...

Generally, this document is a a mashup of the following tutorials.

These links were super handy along the way

Read this for 5 minutes to save hours of trying to understand Django static files

We'll do the Django tutorial part first, then do the heroku steps necessary to keep it in sync.

first though, let's install some stuff we're going to need

brew install gnu-sed # this is because osx's sed is lame

Part 1 of the Django Tutorial

We keep and eye toward repeatability and Heroku.

DRY

Here we set up our project name. This is the same name as the project on heroku (ie horseday.heroku.com). You don't have to set it up this way, but it is convenient for smaller projects.

export PROJ=horseday # You probably want to use something other than 'horseday'

Create the local directory

cd ~/Projects/heroku # you might choose differently
mkdir $PROJ && cd $PROJ

Set up the .gitignore

cat <<EOF >.gitignore
venv
*pyc
EOF

Set up the git repo

git init
git add .
git commit -m "initial commit"

activate the local environment

virtualenv venv --distribute
source venv/bin/activate

install Django & friends

For me this takes about 60 seconds.

time pip install Django psycopg2 gunicorn dj-database-url django-storages boto

Start our project

django-admin.py startproject $PROJ .
git add . && git commit -m "Add the $PROJ project."

create a django poll app (from Tutorial #1)

python manage.py startapp polls
git add . && git commit -m "Add the poll app."

Add some imports to the top of settings.py

gsed -i '1aimport os' $PROJ/settings.py
git add . && git commit -m "Add imports to settings.py"

Bypass ALLOWED_HOSTS -- this is probably a bad idea for production

gsed -i "/^ALLOWED_HOSTS =/s/\[\]/[ '*' ]/" $PROJ/settings.py

Get the DEBUG variable from the environment in settings.py

gsed -i "/^DEBUG =/s/.*/DEBUG = 'True'==os.environ['DEBUG']/" $PROJ/settings.py

add polls & storages apps to INSTALLED_APPS in settings.py

gsed -i "s/.*INSTALLED_APPS.*/&\n    'polls',/" $PROJ/settings.py
gsed -i "s/.*INSTALLED_APPS.*/&\n    'storages',/" $PROJ/settings.py

Modify settings.py for our environment

cat <<EOF >> $PROJ/settings.py

# Parse database configuration from DATABASE_URL
import dj_database_url
DATABASES['default'] =  dj_database_url.config()
if not DEBUG:
    AWS_STORAGE_BUCKET_NAME = os.environ['AWS_STORAGE_BUCKET_NAME']
    AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
    AWS_SECRET_ACCESS_KEY = os.environ['AWS_SECRET_ACCESS_KEY']

    STORAGE = 'storages.backends.s3boto.S3BotoStorage'
    STATICFILES_STORAGE = STORAGE

    S3_URL = 'http://%s.s3.amazonaws.com/' % AWS_STORAGE_BUCKET_NAME
    MEDIA_ROOT = 'media'
    STATIC_ROOT = 'static'
    STATIC_URL = S3_URL + STATIC_ROOT + '/'
    MEDIA_URL = S3_URL + MEDIA_ROOT + '/'
    ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'    
EOF
git add . && git commit -m "settings.py modifications"

create polls/models.py

cat <<EOF > polls/models.py
import datetime
from django.db import models
from django.utils import timezone

class Poll(models.Model):
    question = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')
    def __unicode__(self):
        return self.question
    def was_published_recently(self):
        return self.pub_date >= timezone.now() - datetime.timedelta(days=1)

class Choice(models.Model):
    poll = models.ForeignKey(Poll)
    choice_text = models.CharField(max_length=200)
    votes = models.IntegerField(default=0)
    def __unicode__(self):
        return self.choice_text
EOF
git add . && git commit -m "create polls/models.py"

Enable admin by uncommenting "django.contrib.admin" in the INSTALLED_APPS setting.

sed -i "" "/'django.contrib.admin'/s/# //" $PROJ/settings.py
git add . && git commit -m "enable admin in settings.py"

Enable admin in urls.py

sed -i "" -e '/from.*admin/s/# //' -e '/admin.autodiscover/s/# //' -e "/url.*admin\/'/s/# //" $PROJ/urls.py
git add . && git commit -m "enable admin in urls.py"

Create the env directory for env setup scripts

mkdir env

Create the env/common script

The env/common script sets up the common environment variables

cat <<EOF > env/common
source venv/bin/activate
export PROJ=$PROJ
export PYTHONPATH=$PWD # so oneoff scripts can run
EOF
git add . && git commit -m "create the env/common script"

Create the env/local script

The env/local script sets up the local environment variables

cat <<EOF > env/local
export DATABASE_URL=postgres://\$PROJ@localhost:5432/\$PROJ
export DJANGO_SETTINGS_MODULE=\$PROJ.settings
cat <<HMM > ~/.$PROJ.hmm
runserver     run the server in the background
goadmin       start a browser in the admin site
HMM
alias hmm="sort ~/.$PROJ.hmm"
echo "hmm for help"
EOF
git add . && git commit -m "create env/local script"

Create the env/debug script

The env/debug script is used to set up the local development environment in debug mode.

cat <<EOF > env/debug
source env/common
source env/local
export DEBUG=True
function runserver { python manage.py runserver & }
function goadmin { open http://localhost:8000/admin/; }
EOF
git add . && git commit -m "create env/debug script"

Create the env/foreman script

The env/foreman script is used to set up the local development environment in debug mode using the same server as heroku.

cat <<EOF > env/foreman
source env/common
source env/local
export DEBUG=False
export AWS_STORAGE_BUCKET_NAME=\$PROJ
function runserver { foreman start -p 8000 & }
function goadmin { open http://localhost:8000/admin/; }
EOF
git add . && git commit -m "create env/foreman script"

activate our env/debug environment

source env/debug

Create the local database

psql -c "CREATE USER $PROJ;"
psql -c "CREATE DATABASE $PROJ;"
psql -c "GRANT ALL PRIVILEGES ON DATABASE $PROJ to $PROJ;"

verify

psql -c "\list"  # look for $PROJ in left column. Ctc means a user can Connect Create and create Temp
psql -c "\du" # shows users

sync the database locally

python manage.py syncdb # yes/YOURNAME/[email protected]/YOURPASS

show it

psql $PROJ -c "\dt" | grep polls

create a directory for oneoff scripts

mkdir oneoff

Create a oneoff script to create some data

cat <<EOF > oneoff/01.py
from polls.models import Poll, Choice
from django.utils import timezone
p = Poll(question="What's new?", pub_date=timezone.now())
p.quesiton = "What's up?"
p.save()
p.choice_set.create(choice_text='Not much', votes=0)
p.choice_set.create(choice_text='The sky', votes=1)
p.save()
EOF
git add . && git commit -m "create oneoff script to create data"

run it

python oneoff/01.py

check it out...

echo "from polls.models import Poll; Poll.objects.all()" | python manage.py shell

Create the Procfile for foreman

cat <<EOF > Procfile
web: gunicorn $PROJ.wsgi
EOF
git add . && git commit -m "Create Procfile"

test local debug server

source env/debug
runserver
goadmin
kill %1 # to kill the local debug server

Write the static files out to S3

source env/foreman
python manage.py collectstatic

test local foreman server

source env/foreman
runserver
goadmin
kill %1 # to kill the foreman server

The Heroku part of the Django Tutorial, part 1

Create the requirements.txt file

pip freeze > requirements.txt
git add . && git commit -m "create the requirements.txt"

Add lines to the end of the settings.py script for heroku https

cat <<EOF >> $PROJ/settings.py

# Honor the 'X-Forwarded-Proto' header for request.is_secure()
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
EOF
git add . && git commit -m "configure heroku https in settings.py"

Create the heroku project

heroku create $PROJ

Link the current directory to the heroku project

git remote add heroku [email protected]:$PROJ.git
# Note: move on if 'fatal: remote heroku already exists'

Deploy to heroku

git push heroku master

Set up the environment on the heroku side

heroku config:add \
    DEBUG=False \
    PYTHONPATH=/app \
    DJANGO_SETTINGS_MODULE=$PROJ.settings \
    AWS_STORAGE_BUCKET_NAME=$PROJ \
    AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
    AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \

Start heroku (free tier)

heroku ps:scale web=1

sync the database on heroku

heroku run python manage.py syncdb

run our data creation script on the heroku side

heroku run python oneoff/01.py

test heroku admin

open http://horseday.herokuapp.com/admin/

Commands to reset and start over

cd ~/Projects/heroku
rm -rf $PROJ
psql -c "drop database $PROJ" && psql -c "drop user $PROJ"
heroku destroy $PROJ #--confirm $PROJ # get rid of the 1st
exit # get rid of environment

misc commands

heroku logs # show me the logs
heroku run python manage.py shell # manage the heroku side from python shell
python manage.py sql polls # show sql for polls app

johnnylambada/heroku_django.md