Ubuntu 8.10 Intrepid 32-bit server AMI taken from (http://alestic.com/): ami-7cfd1a15
Useful guides:
- http://articles.slicehost.com/2009/1/6/ubuntu-intrepid-ruby-on-rails
- http://devblog.michaelgalero.com/2008/11/27/setting-up-your-ubuntu-server-for-merb/
See this guide for details about securing an EC2 instance: http://developer.amazonwebservices.com/connect/entry!default.jspa?categoryID=100&externalID=1233&printable=true
Panda (the Merb application) will be run as the panda user on the system. To setup the user we first create them:
adduser panda
The password is set to panda123 (however we will disable password login in a moment).
Edit /etc/ssh/sshd_config and disable password login by ensuring these lines exist:
PasswordAuthentication no
Then restart ssh:
/etc/init.d/ssh restart
Now we'll make a copy of the EC2 script is run on startup and copies the key in your EC2 keychain to authorized_keys.
nano /etc/init.d/ec2-get-credentials-for-panda-user
The file should contain the following script:
#!/bin/bash
#
# ec2-get-credentials - Retrieve the ssh credentials and add to authorized_keys
#
# Based on /usr/local/sbin/ec2-get-credentials from Amazon's ami-20b65349
#
prog=$(basename $0)
logger="logger -t $prog"
public_key_url=http://169.254.169.254/1.0/meta-data/public-keys/0/openssh-key
public_key_file=/tmp/openssh_id.pub
public_key_ephemeral=/mnt/openssh_id.pub
authorized_keys=/home/panda/.ssh/authorized_keys
# Wait for the network to come up.
perl -MIO::Socket::INET -e '
until(new IO::Socket::INET("169.254.169.254:80")){print"Waiting for network...\n";sleep 1}
' | $logger
# Try to get the ssh public key from instance data.
curl --silent --fail -o $public_key_file $public_key_url
test -d /home/panda/.ssh || mkdir -p -m 700 /home/panda/.ssh
if [ $? -eq 0 -a -e $public_key_file ] ; then
if ! grep -s -q -f $public_key_file $authorized_keys
then
cat $public_key_file >> $authorized_keys
$logger "New ssh key added to $authorized_keys from $public_key_url"
fi
chmod 600 $authorized_keys
rm -f $public_key_file
# Try to get the ssh public key from ephemeral storage
elif [ -e $public_key_ephemeral ] ; then
if ! grep -s -q -f $public_key_ephemeral $authorized_keys
then
cat $public_key_ephemeral >> $authorized_keys
$logger "New ssh key added to $authorized_keys from $public_key_ephemeral"
fi
chmod 600 $authorized_keys
chmod 600 $public_key_ephemeral
fi
chown -R panda:panda /home/panda/.ssh
Then we set this script to run on startup:
chmod +x /etc/init.d/ec2-get-credentials-for-panda-user
cd /etc/rc2.d
ln -s ../init.d/ec2-get-credentials-for-panda-user S69ec2-get-credentials-for-panda-user
Double check this works by running /etc/init.d/ec2-get-credentials-for-panda-user and sshing in with the panda user.
Add src repo to the source so we can compile ffmpeg from source. The /etc/apt/source.list should look like so:
deb http://ec2-us-east-mirror.rightscale.com/ubuntu intrepid main restricted universe multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ intrepid main restricted universe multiverse
deb http://ec2-us-east-mirror.rightscale.com/ubuntu intrepid-updates main restricted universe multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ intrepid-updates main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu intrepid-security main restricted universe multiverse
Ruby 1.8 is already installed.
apt-get update
apt-get install irb1.8 libreadline-ruby1.8 rdoc1.8 ri1.8 ruby1.8-dev
ln -s /usr/bin/ruby1.8 /usr/bin/ruby
ln -s /usr/bin/ri1.8 /usr/bin/ri
ln -s /usr/bin/rdoc1.8 /usr/bin/rdoc
ln -s /usr/bin/irb1.8 /usr/bin/irb
wget http://rubyforge.org/frs/download.php/45905/rubygems-1.3.1.tgz
tar xzvf rubygems-1.3.1.tgz
cd rubygems-1.3.1
ruby setup.rb
ln -s /usr/bin/gem1.8 /usr/bin/gem
apt-get install mysql-client-5.0 mysql-common mysql-server-5.0 libmysqlclient15-dev
We'll also create the MySQL db panda as well:
mysqladmin -u root create panda
apt-get install git-core unzip xfsprogs
See: http://www.modrails.com/install.html
apt-get install apache2 apache2-prefork-dev
Using Passenger 2.1.1 (beta): http://blog.phusion.nl/2009/03/01/phusion-passenger-211-beta-released-thanks-sponsors/
wget http://phusion-passenger.googlecode.com/files/passenger-2.1.1.gem
gem install passenger-2.1.1.gem --no-rdoc --no-ri
passenger-install-apache2-module -a
Also get the upload progress module and compile as described here: http://drogomir.com/blog/2008/6/18/upload-progress-bar-with-mod_passenger-and-apache
Add the following config lines into /etc/apache2/httpd.conf:
LoadModule upload_progress_module /usr/lib/apache2/modules/mod_upload_progress.so
UploadProgressSharedMemorySize 1024000
LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.1.1/ext/apache2/mod_passenger.so
PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.1.1
PassengerRuby /usr/bin/ruby1.8
Then add the site config to /etc/apache2/sites-available/panda:
<VirtualHost *:80>
DocumentRoot /var/www/panda/current/public
<Location />
# enable tracking uploads in /
TrackUploads On
</Location>
<Location /progress>
# enable upload progress reports in /progress
ReportUploads On
</Location>
</VirtualHost>
Enable the site and disable the default site config:
rm /etc/apache2/sites-enabled/000-default
ln -s /etc/apache2/sites-available/panda /etc/apache2/sites-enabled/000-panda
We also must create the app's dir:
mkdir -p /var/www/panda /var/www/panda/releases /var/www/panda/shared /var/www/panda/shared/system /var/www/panda/shared/log /var/www/panda/shared/pids
chmod -R g+w /var/www/panda
chown -R panda:panda /var/www/panda
On Startup we will need to create a dir in /mnt for temporary video storage when encoding:
nano /etc/init.d/panda-create-tmp-store-dir
The script should contain the following:
#!/bin/bash
mkdir -p /mnt/panda/tmp
chown -R panda:panda /mnt/panda
Then enable it on startup:
chmod +x /etc/init.d/panda-create-tmp-store-dir
cd /etc/rc2.d
ln -s ../init.d/panda-create-tmp-store-dir S68panda-create-tmp-store-dir
...
Put this script into /usr/local/bin/panda_services (see Gist for revision history: http://gist.github.com/81885):
#!/bin/bash
# Startup script for a Panda encoder and notifier.
APPLICATION=$2
PORT=$3
MERB_ENV=$APPLICATION
MERB_RUNNER_FILE="bin/$APPLICATION.rb"
MERB_ROOT="/var/www/panda/current"
PID_FILE="$MERB_ROOT/log/merb.$APPLICATION.pid"
USER_AND_GROUP="panda"
RETVAL=0
# Go no further if merb app conf is gone
if [ ! -d "$MERB_ROOT/" ]; then
echo "FATAL: $MERB_ROOT does not exist"
exit 0
fi
start()
{
echo -n "Starting Panda encoder:"
cd $MERB_ROOT
command="merb -u $USER_AND_GROUP -G $USER_AND_GROUP -n $APPLICATION.$PORT -e $MERB_ENV -P $PID_FILE -p $PORT -r $MERB_RUNNER_FILE -d";
eval "$command";
timeout=10
while [ ! -f "$PID_FILE" ] ; do
sleep 1
timeout=$(( $timeout - 1 ))
if [ $timeout -le 0 ] ; then
echo " [ FAIL ]"
echo "FATAL: master process did not drop a correct pid file within 10 seconds"
exit 1
fi
done
echo " [ OK ]"
exit 0
}
stop()
{
kill -s INT `cat $PID_FILE`
RETVAL=$?
echo -n "Stopping Panda encoder:"
if [ "$RETVAL" -ne 0 ]
then
echo " [ FAIL ]"
else
echo " [ OK ]"
fi
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
sleep 2
start
;;
*)
echo "Usage: $0 (start|stop|restart) (encoder|notifier) <port>"
exit $RETVAL
esac
Make it executable:
chmod +x /usr/local/bin/panda_services
gem install god --no-ri --no-rdoc
We'll setup God to monitor Panda's encoder and notifier processes.
Edit your /etc/god/panda_services.god to look like so:
{:encoder => 5000, :notifier => 6000}.each do |app, port|
God.watch do |w|
w.name = "#{app}-#{port}"
w.interval = 20.seconds # default
w.start = "/usr/local/bin/panda_services start #{app} #{port}"
w.stop = "/usr/local/bin/panda_services stop #{app}"
w.restart = "/usr/local/bin/panda_services restart #{app}"
w.start_grace = 10.seconds
w.restart_grace = 10.seconds
w.pid_file = "/var/www/panda/current/log/merb.#{app}.pid"
w.behavior(:clean_pid_file)
w.start_if do |start|
start.condition(:process_running) do |c|
c.interval = 5.seconds
c.running = false
end
end
end
end
Setup God to start on boot (source of script) by creating /etc/init.d/god:
#!/bin/sh
### BEGIN INIT INFO
# Provides: god
# Required-Start: $all
# Required-Stop: $all
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: God
### END INIT INFO
NAME=god
DESC=god
set -e
# Make sure the binary and the config file are present before proceeding
test -x /usr/bin/god || exit 0
# Create this file and put in a variable called GOD_CONFIG, pointing to
# your God configuration file
test -f /etc/default/god && . /etc/default/god
[ $GOD_CONFIG ] || exit 0
. /lib/lsb/init-functions
RETVAL=0
case "$1" in
start)
echo -n "Starting $DESC: "
/usr/bin/god -c $GOD_CONFIG -P /var/run/god.pid -l /var/log/god.log
RETVAL=$?
echo "$NAME."
;;
stop)
echo -n "Stopping $DESC: "
kill `cat /var/run/god.pid`
RETVAL=$?
echo "$NAME."
;;
restart)
echo -n "Restarting $DESC: "
kill `cat /var/run/god.pid`
/usr/bin/god -c $GOD_CONFIG -P /var/run/god.pid -l /var/log/god.log
RETVAL=$?
echo "$NAME."
;;
status)
/usr/bin/god status
RETVAL=$?
;;
*)
echo "Usage: god {start|stop|restart|status}"
exit 1
;;
esac
exit $RETVAL
Make executable and add to startup:
chmod +x /etc/init.d/god
update-rc.d god defaults
Then set the location of the God config file in /etc/default/god:
GOD_CONFIG=/etc/god/panda_services.god
Script should read EC2 user data to determine Panda github repo and branch to use.
PULL PANDA TO /var/www/panda/current
AUTOMIGRATE
SETUP DEFAULT USER LOGIN
cd /var/www/panda/current
merb -e production -r "User"
Next we follow the steps in the Local Installation Guide
Once complete we should have a nicely configured AMI which users can boot up, attach and EBS volume to and do a simple cap deploy from their local box to get Panda up and running.