Using RVM, PostgreSQL, NGINX, Unicorn and Capistrano
Today I will cover how to setup a multi-application Ruby on Rails server with RVM, NGINX, Unicorn, Capistrano and PostgreSQL on a fresh virtual private server running Linux Ubuntu Server 14.04. I will be using Rails 4.2.0 new applications as examples.
My current company asked me to configure this setup, they're using the IBM's Softlayer as the main datacenter, so the server that I am playing with is a Softlayer. Nothing special about it, but in this case I have decided to don't use Chef or anything like that, but to make the entire server by hand, because it's funnier, right? Let's go.
####Creating the first user and SSH security configurations
By default I have only the root user and a plain text password SSH access. I need to fix that.
If you don't know what is it or how to create an SSH Key you should read: Generating SSH keys (GitHub) and How To Set Up SSH Keys (Ocean Digital).
But if you already have one let's move on.
My server IP address is 158.85.44.54 named (euler) and I defined in the company's cloud servers domain to point him with: euler.webgetinfo.net. Let's connect:
wizard:~ fschuindt$ ssh [email protected] The authenticity of host 'euler.webgetinfo.net (158.85.44.54)' can't be established. RSA key fingerprint is 18:81:09:56:55:5e:d8:a9:83:07:c6:ba:6e:46:6f:91. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'euler.webgetinfo.net,158.85.44.54' (RSA) to the list of known hosts. Password: Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-45-generic i686) * Documentation: https://help.ubuntu.com/ The programs included with the Ubuntu system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. root@euler:~#
The first step is create an Unix user. I don't want to keep using root (nobody wants). So I added the user 'deployer'.
root@euler:~# adduser deployer Adding user `deployer' ... Adding new group `deployer' (1000) ... Adding new user `deployer' (1000) with group `deployer' ... Creating home directory `/home/deployer' ... Copying files from `/etc/skel' ... Enter new UNIX password: Retype new UNIX password: passwd: password updated successfully Changing the user information for deployer Enter the new value, or press ENTER for the default Full Name []: Euler Deployer Room Number []: Work Phone []: Home Phone []: Other []: Is the information correct? [Y/n] Y
Now I will grant root privileges to my new user:
root@euler:~# visudo
Edit this section adding the user to the list like this:
# User privilege specification root ALL=(ALL:ALL) ALL deployer ALL=(ALL:ALL) ALL
Then save: Ctrl+O. And close: Ctrl+X.
It's good for now. I will switch to my user.
root@euler:~# exit logout Connection to euler.webgetinfo.net closed. wizard:~ fschuindt$ ssh [email protected] Password: Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-45-generic i686) * Documentation: https://help.ubuntu.com/ The programs included with the Ubuntu system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. deployer@euler:~$
After log in for the first time, is a good idea to update the system, so:
deployer@euler:~$ sudo apt-get update deployer@euler:~$ sudo apt-get upgrade
As I don't want to keep using SSH with password authentication, I uploaded my public key (from my local notebook) '~/.ssh/id_rsa.pub' to the server.
On euler.webgetinfo.net:
deployer@euler:~$ mkdir .ssh deployer@euler:~$ touch .ssh/authorized_keys
My notebook (wizard is my notebook's name):
wizard:~ fschuindt$ cat ~/.ssh/id_rsa.pub | ssh [email protected] 'cat >> .ssh/authorized_keys'
Now my public key string is recorded in the server '/home/deployer/.ssh/authorized_keys'. That's the place where the SSH server will check for SSH keys authentication with the user 'deployer'.
We should configure the SSH server to don't accept plain text password logins anymore, only SSH keys. Once my public key is located in the authorized_keys file, the OpenSSH will automatically uses it, but password authentications still allowed, we just need to edit the sshd_config file.
$ sudo nano /etc/ssh/sshd_config
Then find the section:
PermitRootLogin yes
Change to:
PermitRootLogin no
And the section:
ChallengeResponseAuthentication yes
To:
ChallengeResponseAuthentication no
Now the section:
# Change to no to disable tunnelled clear text passwords # PasswordAuthentication yes
Change to:
(Don't forget to uncomment!)
# Change to no to disable tunnelled clear text passwords PasswordAuthentication no
Save, close and finally:
$ sudo service ssh restart
Now you can only login in the SSH with the private key, so keep an backup of this key in a safe place.
I also disabled login as root. It's a much more safe configuration.
####Installing RVM, Ruby and Rails
The Ruby Version Manager (RVM) is a great way to get Ruby installed in a Unix like system. It's pretty and easy. To do it we need to install the curl and then uses it to install the RVM itself. Using the commands:
$ sudo apt-get install curl $ gpg --keyserver hkp://keys.gnupg.net --recv-keys D39DC0E3 $ \curl -sSL https://get.rvm.io | bash -s stable $ source /home/deployer/.rvm/scripts/rvm
That's the default way to get RVM full installed, you can read more at RVM Webpage.
I will pick the Ruby 2.2.0, which is pretty fine nowadays. Install it, set as default Ruby and also install the latest Rubygems:
$ rvm install 2.2.0 $ rvm use 2.2.0 --default $ rvm rubygems current
And of course, install the Ruby on Rails:
$ gem install rails --no-ri --no-rdoc
There are also some good packages to ensure installation in a Rails production environment:
$ sudo apt-get install build-essential bison openssl libreadline6 libreadline6-dev git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev autoconf libxslt-dev nodejs imagemagick libmagickwand-dev
That is enough by now.
####The PostgreSQL
We need to install and do some changes with PostgreSQL in order to make it work properly with a Rails application.
First we install the following packages:
$ sudo apt-get install postgresql postgresql-server-dev-9.3 libpq-dev
Next install the pg gem:
$ gem install pg -- --with-pg-config=/usr/bin/pg_config
Now I will create a postgres user. Many developers create one user per app, I prefer just one user for all. And again I will name it deployer:
$ sudo -u postgres psql # create user deployer with password 'YourPasswordHere';
Now grant good privileges to it:
# alter role deployer superuser createrole createdb replication;
After that PostgreSQL will run just fine. To quit the psql just:
# \q
####The NGINX
First, let's configure the bundler:
$ gem install bundler $ mkdir ~/.bundle $ nano ~/.bundle/config
Paste the following:
BUNDLE_PATH: vendor/bundle
Save and exit.
Now install and start NGINX:
$ sudo apt-get install nginx $ sudo service nginx start
####The applications and pre-deployment
For this post I will create two new Rails apps as example, and deploy them both using Capistrano. Capistrano synchronizes with a Git repository to make the deployment in the server, so it needs to have the app in a remote repository too.
Once it's a company's server, I will use the official Team Account in the company's Bitbucket as the main Git account in the server. The Capistrano will access the repositories from the server, so I have to allow my server's SSH key into the Team Account at the Bitbucket.
But first, I need to create a SSH Key Pair in my server. Again, if you don't know about SSH Keys or how to create them, read the tutorials linked in the section "Creating the first user and SSH security configurations". I will just run the ssh-keygen program in the server.
After it's created, I can see the id_rsa and id_rsa.pub files in my ~/.ssh folder.
deployer@euler:~$ cd .ssh/ deployer@euler:~/.ssh$ ls authorized_keys id_rsa id_rsa.pub
Nice. I will copy the id_rsa.pub content and add it as a new SSH Key in the company's Bitbucket Team Account with the label "deployer@euler". Now the server must be able to access the company's Bitbucket repositories.
Outside the server, in my notebook, I will create two applications:
wizard:Desktop fschuindt$ rails new app1 wizard:Desktop fschuindt$ rails new app2
And create two DNS records in the domain, pointing to the server's IP to get two URLs.
app1.webgetinfo.net.br -> 158.85.44.54 - 14400 app2.webgetinfo.net.br -> 158.85.44.54 - 14400
###The deployment
Let's configure the apps in order to deploy. Note that everything I will do now on must be done for each app you pretend to deploy. In this post I will only show the app1 configuration, you must do the same in the app2 or any other app you like, just changing the app's name when it's needed. Take care.
First step is add the NGINX configurations to app1 in the server:
deployer@euler:~$ sudo nano /etc/nginx/nginx.conf
Look inside the block 'http', in the end, after the line: 'include /etc/nginx/sites-enabled/*;', you should add this code for each app you want to deploy:
upstream app1 { server unix:/home/deployer/unicorn_sockets/unicorn.app1.sock fail_timeout=0; }
server {
listen 80;
server_name app1.webgetinfo.net;
root /home/deployer/app1/current/public;
location ^~ /assets/ {
gzip_static on;
expires max;
add_header Cache-Control public;
}
try_files $uri/index.html $uri @unicorn;
location @unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app1;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
Don't forget to edit where is needed! App name, DNS, etc.
Save and close.
If you want to add other app, like the app2, you just need to duplicate this inside the http block of the nginx.conf file.
Restart the NGINX:
deployer@euler:~$ sudo service nginx restart
Create the sockets directory:
(Only do this in the first app)
deployer@euler:~$ mkdir unicorn_sockets
And now create an view and a root url for the new apps:
(local notebook)
wizard:Desktop fschuindt$ cd app1/
wizard:app1 fschuindt$ rails g controller Pages index
create app/controllers/pages_controller.rb
route get 'pages/index'
invoke erb
create app/views/pages
create app/views/pages/index.html.erb
invoke test_unit
create test/controllers/pages_controller_test.rb
invoke helper
create app/helpers/pages_helper.rb
invoke test_unit
invoke assets
invoke coffee
create app/assets/javascripts/pages.coffee
invoke scss
create app/assets/stylesheets/pages.scss
Edit the "app/views/pages/index.html.erb" to show "app1" in the title instead of "Pages#index". In this way we will know that's is the app1 and not the app2.
Set it as the root url in your config/routes.rb:
root 'pages#index'Add to Gemfile:
gem 'pg'
gem 'unicorn'
gem 'capistrano'
gem 'rvm-capistrano', require: falseAlso move the sqlite3 gem to inside the "group :development, :test do" block. It ensures that we'll use PostgreSQL in production and SQLite in test and development.
And now bundle install:
wizard:app1 fschuindt$ bundle install
Create the config/unicorn.rb, config/unicorn_init.sh files and set the executable permission:
wizard:app1 fschuindt$ touch config/unicorn.rb wizard:app1 fschuindt$ touch config/unicorn_init.sh wizard:app1 fschuindt$ chmod +x config/unicorn_init.sh
Add to config/unicorn.rb: (Again, change the app name)
root = "/home/deployer/app1/current"
working_directory root
pid "#{root}/tmp/pids/unicorn.pid"
stderr_path "#{root}/log/unicorn.log"
stdout_path "#{root}/log/unicorn.log"
listen "/home/deployer/unicorn_sockets/unicorn.app1.sock"
worker_processes 2
timeout 30
# Force the bundler gemfile environment variable to
# reference the capistrano "current" symlink
before_exec do |_|
ENV["BUNDLE_GEMFILE"] = File.join(root, 'Gemfile')
endAnd add to the config/unicorn_init.sh:
(Again, change to fit your app needs)
#!/bin/sh
### BEGIN INIT INFO
# Provides: unicorn
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Manage unicorn server
# Description: Start, stop, restart unicorn server for a specific application.
### END INIT INFO
set -e
# Feel free to change any of the following variables for your app:
TIMEOUT=${TIMEOUT-60}
APP_ROOT=/home/deployer/app1/current
PID=$APP_ROOT/tmp/pids/unicorn.pid
CMD="cd $APP_ROOT; bundle exec unicorn -D -c $APP_ROOT/config/unicorn.rb -E production"
AS_USER=deployer
set -u
OLD_PIN="$PID.oldbin"
sig () {
test -s "$PID" && kill -$1 `cat $PID`
}
oldsig () {
test -s $OLD_PIN && kill -$1 `cat $OLD_PIN`
}
run () {
if [ "$(id -un)" = "$AS_USER" ]; then
eval $1
else
su -c "$1" - $AS_USER
fi
}
case "$1" in
start)
sig 0 && echo >&2 "Already running" && exit 0
run "$CMD"
;;
stop)
sig QUIT && exit 0
echo >&2 "Not running"
;;
force-stop)
sig TERM && exit 0
echo >&2 "Not running"
;;
restart|reload)
sig HUP && echo reloaded OK && exit 0
echo >&2 "Couldn't reload, starting '$CMD' instead"
run "$CMD"
;;
upgrade)
if sig USR2 && sleep 2 && sig 0 && oldsig QUIT
then
n=$TIMEOUT
while test -s $OLD_PIN && test $n -ge 0
do
printf '.' && sleep 1 && n=$(( $n - 1 ))
done
echo
if test $n -lt 0 && test -s $OLD_PIN
then
echo >&2 "$OLD_PIN still exists after $TIMEOUT seconds"
exit 1
fi
exit 0
fi
echo >&2 "Couldn't upgrade, starting '$CMD' instead"
run "$CMD"
;;
reopen-logs)
sig USR1
;;
*)
echo >&2 "Usage: $0 <start|stop|restart|upgrade|force-stop|reopen-logs>"
exit 1
;;
esacSetup Capistrano:
wizard:app1 fschuindt$ capify .
At this point I will create the app repository locally and in the Bitbucket. Once it's important to don't have the config/database.yml in your version control, let's copy it as example and add it to the .gitignore file. The same to the config/secrets.yml file. Rails encourage us to use environment variables to set the production secret_key_base, but I have experienced some issues with this approach together with Unicorn, so let's just ignore it in the Git and add it later by hand. If you are using OS X as your development environment is good to ignore the .DS_Store files too, so add to the .gitignore the following lines:
/config/database.yml /config/secrets.yml .DS_Store
Copy the database.yml and secrets.yml to be versioned as example only:
wizard:app1 fschuindt$ cp config/database.yml config/database.example.yml wizard:app1 fschuindt$ cp config/secrets.yml config/secrets.example.yml
Now the repository, using the remote url given by the Bitbucket:
wizard:app1 fschuindt$ git init wizard:app1 fschuindt$ git add . wizard:app1 fschuindt$ git commit -m "first commit" wizard:app1 fschuindt$ git remote add origin [email protected]:GetinfoDevTeam/app1.git
Add to config/deploy.rb, remove what was previously inside and replace for:
(There's some important variables such your server address, repository url, etc. Replace them if you are not using the recently created app1)
require "bundler/capistrano"
require "rvm/capistrano"
server "euler.webgetinfo.net", :web, :app, :db, primary: true
set :application, "app1"
set :user, "deployer"
set :port, 22
set :deploy_to, "/home/deployer/#{application}"
set :deploy_via, :remote_cache
set :use_sudo, false
set :scm, "git"
set :repository, "[email protected]:GetinfoDevTeam/app1.git"
set :branch, "master"
default_run_options[:pty] = true
ssh_options[:forward_agent] = true
after "deploy", "deploy:cleanup" # keep only the last 5 releases
namespace :deploy do
%w[start stop restart].each do |command|
desc "#{command} unicorn server"
task command, roles: :app, except: {no_release: true} do
run "/etc/init.d/unicorn_#{application} #{command}"
end
end
task :setup_config, roles: :app do
sudo "ln -nfs #{current_path}/config/nginx.conf /etc/nginx/sites-enabled/#{application}"
sudo "ln -nfs #{current_path}/config/unicorn_init.sh /etc/init.d/unicorn_#{application}"
run "mkdir -p #{shared_path}/config"
put File.read("config/database.example.yml"), "#{shared_path}/config/database.yml"
put File.read("config/secrets.example.yml"), "#{shared_path}/config/secrets.yml"
puts "Now edit the config files in #{shared_path}."
end
after "deploy:setup", "deploy:setup_config"
task :symlink_config, roles: :app do
run "ln -nfs #{shared_path}/config/database.yml #{release_path}/config/database.yml"
run "ln -nfs #{shared_path}/config/secrets.yml #{release_path}/config/secrets.yml"
end
after "deploy:finalize_update", "deploy:symlink_config"
desc "Make sure local git is in sync with remote."
task :check_revision, roles: :web do
unless `git rev-parse HEAD` == `git rev-parse origin/master`
puts "WARNING: HEAD is not the same as origin/master"
puts "Run `git push` to sync changes."
exit
end
end
before "deploy", "deploy:check_revision"
endNow to the Capfile, again remove what was previously inside and replace for:
load 'deploy' load 'deploy/assets' load 'config/deploy'
Commit and push:
wizard:app1 fschuindt$ git commit -am "deployment configurations" wizard:app1 fschuindt$ git push origin master
Start the deploy:
wizard:app1 fschuindt$ cap deploy:setup
Now is time to edit the /home/deployer/app1/shared/config/database.yml file on the server, and setup PostgreSQL credentials.
Like:
production:
adapter: postgresql
encoding: utf8
database: app1_production
username: deployer
password: your-password-here
host: localhostYou should also edit the /home/deployer/app1/shared/config/secrets.yml. You need to put your app secret production key.
In this case it's ok to have the raw secret_key_base in the file because it's not versioned (The same to the database password in the database.yml). Use the "rake secret" in your local computer to create a key, once you have it, edit your /home/deployer/app1/shared/config/secrets.yml production block like this:
production:
secret_key_base: your-generated-key-hereJust save and close.
Now create the production database on the server:
deployer@euler:~$ sudo -u postgres psql [sudo] password for deployer: psql (9.3.6) Type "help" for help. postgres=# create database app1_production with owner deployer encoding='UTF-8' lc_collate='en_US.utf8' lc_ctype='en_US.utf8' template template0; CREATE DATABASE postgres=# \q
Now go:
wizard:app1 fschuindt$ cap deploy:cold
After, do this in the server:
(The rm command is only needed in the first app setup)
deployer@euler:~$ sudo rm /etc/nginx/sites-enabled/default deployer@euler:~$ sudo service nginx restart * Restarting nginx nginx [ OK ]
deployer@euler:~$ sudo update-rc.d -f unicorn_app1 defaults Adding system startup for /etc/init.d/unicorn_app1 ... /etc/rc0.d/K20unicorn_app1 -> ../init.d/unicorn_app1 /etc/rc1.d/K20unicorn_app1 -> ../init.d/unicorn_app1 /etc/rc6.d/K20unicorn_app1 -> ../init.d/unicorn_app1 /etc/rc2.d/S20unicorn_app1 -> ../init.d/unicorn_app1 /etc/rc3.d/S20unicorn_app1 -> ../init.d/unicorn_app1 /etc/rc4.d/S20unicorn_app1 -> ../init.d/unicorn_app1 /etc/rc5.d/S20unicorn_app1 -> ../init.d/unicorn_app1
Back to the local computer:
wizard:app1 fschuindt$ cap deploy
It's done! Visit your app URL to check it. Now you can repeat these steps to deploy another app in the same server. You basically can deploy many apps as you like. Hope you guys enjoyed.
####Credits
I want (and must to) say thanks for a guy who I don't know, named James Dullaghan. He published this gist, which I mostly based me on. So thanks him for the code. And also his gist is a good place for looking clues if you're having troubles.
Also thanks to my friend Arthur Rocha who is an Rails enthusiast and helped me a lot revising the text.
That's it. I will keep updating this document for a while. And I pretend to release new ones about Rails/Linux servers administration. The next will be about some routines in this server like database backup, easy methods to configure files and keep track of the logs with no pain.
Check out my personal blog for new releases later on:
https://schuindtlog.wordpress.com/
April 8, 2015 - Fernando Schuindt
[email protected]