【deploy Django on AWS EC2】

Solve a deployment problem

Problem：psycopg2 not found

This afternoon I have met a problem for my website, it is not accessible due to an error of psycopg2, which is a module for python to connect with Postgresql database. I would like to document this to public for anyone in future for a referrence.

After the problem occurred, I blamed AWS, that everything was running OK and it just suddenly failed. The fact is that I didn't change anything in between, which is annoying. Or maybe one of the system upgrade suggested by AWS was the reason, I remembered that several packages was installed, but it seemed the problem had not really instantly occurred. But anyhow, the problem has to be solved, after waiting for serveral hours, I started to debug it. The problem states that:

ImproperlyConfigured: Error loading psycopg2 module: No module named psycopg2

which apprently means that psycopg2 is not found by my web server.

background：system configuration

Here is the settings of my system.

apache (httpd)
Django 1.6
python 2.7
AWS EC2 with centOS 6.4
mod_wsgi 3.5
virtualenv 1.9

solution：recompiling mod_wsgi and reinstalling httpd

I google around and found something similar as mine. So I checked my path of virtualenv and everything, all seem in correct ways. Finally I decided to recompile mod_wsgi, from the hint projected by this post. Actually, when I first deploy my website, I have met similar problem, though not exactly as this one, and I have documented this post for later use. It was a surprise to return to this post. On that document, I even marked that section where this post is recorded as a "Caution" part. So I decided to follow the solution in that post, re-compliling mod_wsgi.

As I proceed, just found that mod_wsgi has a new version availible, and it mentions that the previous version of 3.4 has security issues, so why not just update it to the newest. So I download the 3.5 version, compile it under my virtualenv, so that it is compatible with python27 rather than python2.6.9 which is the defaul python of centOS 6.4 of EC2.

So is it solved? No. Not yet.

So I tried twice with system installed python2.7 but without virtualenv, and only virtualenv, but both failed to solve the problem. So I decided to re-install httpd as well.

sudo yum remove httpd

and httpd with two dependencies also were removed， one is httpd-devel, another is mod_wsgi 3.2 .

So I reinstall httpd and finally the problem solved. From this experience, I re-discovered that to document in detail is how much of importance. So I have to record how I deploy Django on AWS EC2.

Deploy Django on AWS EC2

I was much lighted by this post, this tutorial is based on it, while I add more details and explanations here as well.

AWS EC2

I chose centOS AMI, which is recommended by AWS. Another main choice is Ubuntu.

install python 27

Note that the default CentOS installation of Python is 2.6.9, we better upgrade it to 2.7. Before this, you might optionally update all packages first, which is sudo yum update, but I just skip it.

sudo yum install gcc python27 python27-devel

gcc is for compiling a lot modules, so better install it as soon as possible.
python27-devel is centOS compatible packages for compiling python related packages.

in case not for yum install on AWS EC2:

** for aliyun **

yum install python-devel.x86_64 openssl-devel

** for compiling python2.7 from source **

./configure —enable-shared (for postgresql to use)
make && make install

for problem related with shared lib referrence.

run commands as root user

Note that a series of commands need to be run with sudo previledge, so better do it once,

sudo su - root

now we can install software without prefix sudo everytime.

install httpd

yum install httpd httpd-devel

install virtualenv

curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.1.tar.gz
tar -xvf virtualenv-1.9.1.tar.gz
python27 virtualenv-1.9.1/virtualenv.py ~/venv-27
source ~/venv-27/bin/activate

Note:

curl and wget is used interchangeably.
for https connection failure, add this option: —no-check-certificate
for zlib problems of virtualenv, can first install
```
yum install zlib-devel
```
then reinstall python27
venv-27 is a folder that holds the virtual env packages, can be customized as you wish for its location and name

install mod_wsgi

install this if not yet, because it is a dependency of mod_wsgi

yum install httpd httpd-devel

and then

wget https://github.com/GrahamDumpleton/mod_wsgi/archive/3.5.tar.gz
tar -xvf 3.5.tar.gz
./configure  (note that right now should be in virtualenv)
make && make install  (note that now should be with root user privilege)

Note:

virtualenv can help compilation compatible with python27

configure httpd

either add a file to /etc/httpd/conf.d/

echo 'LoadModule wsgi_module modules/mod_wsgi.so' > /etc/httpd/conf.d/wsgi.conf

or just add another line to /etc/httpd/conf/httpd.conf.

LoadModule wsgi_module modules/mod_wsgi.so

and change the user which run httpd to ec2-user

User ec2-user
Group ec2-user

install pip

before installing pip, you might need this package, if not installed by default, for downloading from internet sources.

yum install openssl-devel
easy_install pip

for Postgresql users, install psycopg2

sudo yum install postgresql-devel
pip install psycopg2

install Django and other packages

(venv-27)$ pip install Django==1.6

or from a file

pip install -r requirements-python27.txt

for Mac user

pip sometimes not work due to xcode 5.1

ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install -r requirements-python27.txt

X11 problem

first install

xcode-select —install

postgresql app

install the app, and the database is at

 /Users/tony/Library/Application Support/Postgres/var-9.3

setup virtual host for httpd

add these lines in file httpd.conf at the bottom part, you need

sudo vi /etc/httpd/conf/httpd.conf

WSGIPythonPath /home/ec2-user/weblinwo:/home/ec2-user/venv-27/lib/python2.7/site-packages

<VirtualHost *:80>
	#    ServerAdmin [email protected]
	     ServerAdmin [email protected]
	#    DocumentRoot /www/docs/dummy-host.example.com
	     DocumentRoot /home/ec2-user/web
	
	#    ServerName dummy-host.example.com
	    ServerName ec2-123-123-123-123.xxxxxx.compute.amazonaws.com

     Timeout 60
    WSGIScriptAlias / /home/ec2-user/web/youwebapp/wsgi.py

    <Directory /home/ec2-user/web/youwebapp>
      <Files wsgi.py>
        Order allow,deny
        Allow from all
      </Files>
    </Directory>

    AliasMatch ^/([^/]*\.css) /home/ec2-user/static/css/$1

    #Alias /media/ /path/to/mysite.com/media/
    Alias /static/ /home/ec2-user/static/
    Alias /favicon.ico /path/to/favicon.ico

    <Directory /home/ec2-user/static>
      Allow from all
    </Directory>

    <Directory />
        AllowOverride None
        Options Indexes
    </Directory>

    <Directory /home/ec2-user/web/.git/>
        Deny From All
    </Directory>

    ErrorLog /home/ec2-user/httpdlogs/error_log
    CustomLog /home/ec2-user/httpdlogs/access_log combined
</VirtualHost>

start httpd server

sudo service httpd start

check httpd的pid

sudo cat /etc/httpd/run/httpd.pid

ps -A
ps aux

psql

after installed psql

database create user

createdb -T template0 db_mba_withyou_pg

database backup

pg_dump —username=Username —dbname=dbname —file=backupfile

referrence

consider use RDS

so it will be easily scaled

AWS Security setup

inbound

from outside to connect to server，ask for enter server
icmp setup finished, then you can ping from local machine to public dns of ec2

outbound

from server to transfer data to outside，or from server to connect to other sources icmp setup finished, then you can ping from ec2 to public websites

RDS, inbound settings，open 5432 port of custom tcp
ec2, outbound settings，also should open this port

mobilestack/deploy Django on AWS EC2.md