Revision-Controlled Worpdress with "One Click" Deplyoment using Git, Composer, Capistrano
Author: Oran Blackwell
Revised: May 06, 2014
This is a Work In Progress - All feedback is always welcome.
I am attempting to document everything in this document so don't be put off if it seems overly complex. Once the initial setup is done, there is minimal interaction required to deploy. I've tried to include how to handle errors etc.
[TOC]
- Version control with Git
- Dependency management with Composer
- Additional Gem dependency management with Bundler
- Automated deployments with Capistrano
- Environment variables with Dotenv
- Wordpress theme and plugin management with Wordpress Packagist
- Automate Everything!!!
- Improve workflow
- Easy team colloboration via Git repositories hosted with BitBucket
- Allow for complete staging/testing/QA environments as needed
- Easy and reversible deployment of everything, including:
- Wordpress core updates
- Plugin updates
- Theme updates
- Database entries and schemes
- Alterations to
chown
andchmod
of directories and files
- Eliminate the dreaded Cowboy-Coder
- Establish an audit trail to facilitate accountability - not blame ;)
- To harden up security
- Eliminate the need for server passwords via Public/Private SSH Keys
- Eradicate FTP deployments
- Eradicate file changes directly on servers
- Allow for modular improvements to process
- integration with Grunt for all its goodies like auto-refresh, compression, concatenation,
- To setup automated tests for everything with, possibly:
- To facilitate more accurate planning
- Full and reusable documentation
- Projects built (and therefor re-buildable) to specifications and within scope
- Other wonderful things
- Git
- PHP >= 5.3.2 (for Composer) Use something like XAMPP & make PHP accessible from PATH on Windows
- Ruby >= 1.9 (for Capistrano)
- Bundler (manages your Ruby gems/dependencies)
- Composer
- Required Gems:
- capistrano (> 3.1.0)
- capistrano-composer
See Requirements for more details.
$ php -v
$ ps aux | grep -i mysql
$ gem -v
$ ruby -v
$ composer --version
- Install:
$ curl -sS https://getcomposer.org/installer | php
- On remote server run:
$ mv composer.phar /usr/local/bin/composer
- Or add "composer" Alias - see elsewhere in this file
$ bundle -v
- Install:
$ sudo gem install bundler
Note: Dont forget that you may need to run $ sudo gem install <gem>
when adding new gems - You should only need to run this on your local machine
The script will create its own project folder so run from your project's root directory. <path-to-projects>/Sites/
$ composer create-project roots/bedrock example
Installing roots/bedrock (1.2.4)
- Installing roots/bedrock (1.2.4)
Downloading: 100%
Created project in example
Generate salts and append to .env file? [Y,n]? y
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
- Installing composer/installers (v1.0.12)
Downloading: 100%
- Installing fancyguy/webroot-installer (1.1.0)
Downloading: 100%
- Installing vlucas/phpdotenv (1.0.6)
Downloading: 100%
- Installing wordpress/wordpress (3.9)
Downloading: 100%
Generating autoload files
Note:
You MAY need to re-run $ composer update
to fully install some bits.
Note: Make sure you point it to <example>/web/
Set up new host - eg example.local
pointing to <path-to-projects>/Sites/example/web/
Note: Make sure you point it to <example>/current/web
DocumentRoot /var/www/vhosts/staging.<domain>.com/current/web
<Directory /var/www/vhosts/staging.<domain>.com/current/web>
ErrorLog "<path-to-projects>/Logs/example-error_log"
CustomLog "<path-to-projects>/Logs/example-access_log" common
Using above vhosts for http://.local/
WP_ENV=development
WP_HOME=http://<example>.local/
WP_SITEURL=http://<example>.local/wp/
http://<example>.local/
Should now resolve to some Wordpress page.
See below
Blah
In depth guide can be found here for Mac and Bitbucket if you have difficulties.
Check if your localhost has an SSH Key generated:
$ shh-add -l
The agent has no identities.
I didnt have any so I created a new one, using the default location.
Note: If prompted, enter an actual passphrase.
$ ssh-keygen -t rsa -C 'oranblackwell@<domain>.com'
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/oran/.ssh/id_rsa):
Created directory '/Users/oran/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/oran/.ssh/id_rsa.
Your public key has been saved in /Users/oran/.ssh/id_rsa.pub.
The key fingerprint is:
17:8e:df:fc:e7:29:e8:8c:51:6b:14:d8:2b:43:c9:67 oranblackwell@<domain>.com
The key's randomart image is:
+--[ RSA 2048]----+
| |
| . + |
| =.E |
| .oo.o |
| Soo+ |
| o=o. |
| ..o+ |
| =. o o|
| ..o o+.|
+-----------------+
Add the newly created key to your SSH Agent on you local machine to prevent you having to enter your password everytime:
$ ssh-add
Enter passphrase for /Users/oran/.ssh/id_rsa:
Identity added: /Users/oran/.ssh/id_rsa (/Users/oran/.ssh/id_rsa)
We need to add the public part of this key to Bitbucket and to each of the servers that we will be deploying to (see section below).
Note: the capital L
$ ssh-add -L
ssh-rsa AAAAB3..................................................b2V /Users/oran/.ssh/id_rsa
Note: You need to add this to ~/.ssh/authorized_keys
for the remote server's user which will be doing the deployment. I had intended this to be deploy
but it didnt work very well so I resorted to using root
. So, if you logged in as root
, run the following to switch to that user's account before the next step using:
$ su - deploy
In the correct user's account, add the output from $ ssh-add -L
as a new line to ~/.ssh/authorized_keys
Remember: This needs to be done on every server you want to use, you can use the same key for each server, but only one key per developer.
Tip: While on each server - Install Composer. (see elsewhere in document for instructions.)
@todo Setup a deploy user and group - I ran into difficulties when setting up users and groups for the deployment process. I reverted to using root. Please dont use root! See this article for an excellent guide.
NOTE If the following happens:
ssh-add -l
Could not open a connection to your authentication agent.
check if ssh-agent
is running with :
ps -e | grep [s]sh-agent
start with :
ssh-agent /bin/bash
In your Bitbucket account go to Avatar -> Manage Account -> SSH Keys
Blah
Important Note: Never add any sesitive data to a repository, including usernames and passwords. Usually this includes wp-config.php files but not in this particular setup as the database details have been moved to .env
files.
Note: I modified .gitignore
before performing my first add and commit as it can be tricky to untrack files once they are initially tracked - for now I just added:
# PHP Storm
.idea/*
.inputrc
# MAC OS Annoyances
.DS_STORE
Note: Add your remote origin using the git
protocol, not http
or ssh
. eg. [email protected]
$ git init
$ git add . # be cautious when using "."
$ git commit -am "Initial Commit."
$ git remote add origin [email protected]:oranblackwell/staging-<domain>.git
$ git push -u origin --all # pushes up the repo and its refs for the first time
The authenticity of host 'bitbucket.org (131.103.20.168)' can't be established.
RSA key fingerprint is 97:8c:1b:f2:6f:14:6b:5c:3b:ec:aa:46:46:74:7c:40.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'bitbucket.org,131.103.20.168' (RSA) to known hosts.
Counting objects: 35, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (30/30), done.
Writing objects: 100% (35/35), 16.29 KiB | 0 bytes/s, done.
Total 35 (delta 3), reused 0 (delta 0)
To [email protected]:oranblackwell/staging-<domain>.git
* [new branch] master -> master
Branch master set up to track remote branch master from origin.
$ git push -u origin --tags # pushes up any tags
Warning: Permanently added the RSA key for IP address '131.103.20.167' to known hosts.
Everything up-to-date
Important changes are:
set :repo_url, '[email protected]:oranblackwell/staging-<domain>.git'
set :deploy_to, "/var/www/vhosts/#{fetch(:application)}"
Note: Do not add .htaccess
yet.
blah blah
Update server and user details
# Extended Server Syntax
# ======================
server '23.253.218.207', user: 'root', roles: %w{web app db}
$ sudo bundle install
Fetching gem metadata from https://rubygems.org/.......
Installing rake 10.1.1
Installing i18n 0.6.9
Installing net-ssh 2.8.0
Installing net-scp 1.1.2
Installing tins 1.0.0
Installing term-ansicolor 1.3.0
Installing sshkit 1.3.0
Installing capistrano 3.1.0
Installing capistrano-composer 0.0.3
Using bundler 1.6.2
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
$
You will need to be able to execute the following command from you remote project directory, eg /var/www/vhosts/staging.<domain>.com/
See instructios here
Note: Annoyingly this WILL fail the first time because the .env
file doesnt exist within the appropriate folder, but the script should setup all the directories as required.
Note: This is always run from your local machine.
$ bundle exec cap staging deploy:check
INFO [5aaf7546] Running /usr/bin/env mkdir -p /tmp/staging.<domain>.com/ on 23.253.218.207
INFO [5aaf7546] Finished in 2.424 seconds with exit status 0 (successful).
INFO Uploading /tmp/staging.<domain>.com/git-ssh.sh 100.0%
INFO [91935aeb] Running /usr/bin/env chmod +x /tmp/staging.<domain>.com/git-ssh.sh on 23.253.218.207
INFO [91935aeb] Finished in 0.247 seconds with exit status 0 (successful).
INFO [0f9eb12b] Running /usr/bin/env mkdir -pv /var/www/vhosts/staging.<domain>.com/shared /var/www/vhosts/staging.<domain>.com/releases on 23.253.218.207
INFO [0f9eb12b] Finished in 0.242 seconds with exit status 0 (successful).
INFO [0754feb6] Running /usr/bin/env mkdir -pv /var/www/vhosts/staging.<domain>.com/shared/web/app/uploads on 23.253.218.207
INFO [0754feb6] Finished in 0.244 seconds with exit status 0 (successful).
INFO [a773f2c3] Running /usr/bin/env mkdir -pv /var/www/vhosts/staging.<domain>.com/shared /var/www/vhosts/staging.<domain>.com/shared on 23.253.218.207
INFO [a773f2c3] Finished in 0.236 seconds with exit status 0 (successful).
ERROR linked file /var/www/vhosts/staging.<domain>.com/shared/.env does not exist on 23.253.218.207
$
You should save a copy of this on your local machine as .env.<stage>
, eg. .env.staging
, but be sure to never add it to the repository.
Re-running the deploy:check
should finish and exit silently. (Remember: run from your local machine)
$ bundle exec cap staging deploy:check
INFO [a9f7639c] Running /usr/bin/env mkdir -p /tmp/staging.<domain>.com/ on 23.253.218.207
INFO [a9f7639c] Finished in 2.113 seconds with exit status 0 (successful).
INFO Uploading /tmp/staging.<domain>.com/git-ssh.sh 100.0%
INFO [039f95a6] Running /usr/bin/env chmod +x /tmp/staging.<domain>.com/git-ssh.sh on 23.253.218.207
INFO [039f95a6] Finished in 0.250 seconds with exit status 0 (successful).
INFO [5343ea46] Running /usr/bin/env mkdir -pv /var/www/vhosts/staging.<domain>.com/shared /var/www/vhosts/staging.<domain>.com/releases on 23.253.218.207
INFO [5343ea46] Finished in 0.251 seconds with exit status 0 (successful).
INFO [4e391b72] Running /usr/bin/env mkdir -pv /var/www/vhosts/staging.<domain>.com/shared/web/app/uploads on 23.253.218.207
INFO [4e391b72] Finished in 0.242 seconds with exit status 0 (successful).
INFO [fe9e743d] Running /usr/bin/env mkdir -pv /var/www/vhosts/staging.<domain>.com/shared on 23.253.218.207
INFO [fe9e743d] Finished in 0.249 seconds with exit status 0 (successful).
$
Hopefully this will finish without any failures and exit silently.
$ bundle exec cap staging deploy
INFO [928f53f7] Running /usr/bin/env mkdir -p /tmp/staging.<domain>.com/ on 23.253.218.207
INFO [928f53f7] Finished in 2.223 seconds with exit status 0 (successful).
INFO Uploading /tmp/staging.<domain>.com/git-ssh.sh 100.0%
INFO [b0c24462] Running /usr/bin/env chmod +x /tmp/staging.<domain>.com/git-ssh.sh on 23.253.218.207
INFO [b0c24462] Finished in 0.237 seconds with exit status 0 (successful).
INFO [f0e90dd3] Running /usr/bin/env mkdir -pv /var/www/vhosts/staging.<domain>.com/shared /var/www/vhosts/staging.<domain>.com/releases on 23.253.218.207
INFO [f0e90dd3] Finished in 0.251 seconds with exit status 0 (successful).
INFO [29128b98] Running /usr/bin/env mkdir -pv /var/www/vhosts/staging.<domain>.com/shared/web/app/uploads on 23.253.218.207
INFO [29128b98] Finished in 0.247 seconds with exit status 0 (successful).
INFO [e483154a] Running /usr/bin/env mkdir -pv /var/www/vhosts/staging.<domain>.com/shared on 23.253.218.207
INFO [e483154a] Finished in 0.266 seconds with exit status 0 (successful).
INFO The repository mirror is at /var/www/vhosts/staging.<domain>.com/repo
INFO [3796db84] Running /usr/bin/env git remote update on 23.253.218.207
INFO [3796db84] Finished in 1.345 seconds with exit status 0 (successful).
INFO [2e4a1f40] Running /usr/bin/env mkdir -p /var/www/vhosts/staging.<domain>.com/releases/20140501145739 on 23.253.218.207
INFO [2e4a1f40] Finished in 0.277 seconds with exit status 0 (successful).
INFO [da8cb911] Running /usr/bin/env git archive master | tar -x -C /var/www/vhosts/staging.<domain>.com/releases/20140501145739 on 23.253.218.207
INFO [da8cb911] Finished in 0.266 seconds with exit status 0 (successful).
INFO [f9a095ae] Running /usr/bin/env mkdir -pv /var/www/vhosts/staging.<domain>.com/releases/20140501145739 on 23.253.218.207
INFO [f9a095ae] Finished in 0.265 seconds with exit status 0 (successful).
INFO [fcc46f2f] Running /usr/bin/env ln -s /var/www/vhosts/staging.<domain>.com/shared/.env /var/www/vhosts/staging.<domain>.com/releases/20140501145739/.env on 23.253.218.207
INFO [fcc46f2f] Finished in 0.258 seconds with exit status 0 (successful).
INFO [bc5ae1fe] Running /usr/bin/env mkdir -pv /var/www/vhosts/staging.<domain>.com/releases/20140501145739/web/app on 23.253.218.207
INFO [bc5ae1fe] Finished in 0.266 seconds with exit status 0 (successful).
INFO [59629ca3] Running /usr/bin/env rm -rf /var/www/vhosts/staging.<domain>.com/releases/20140501145739/web/app/uploads on 23.253.218.207
INFO [59629ca3] Finished in 0.266 seconds with exit status 0 (successful).
INFO [34c31a23] Running /usr/bin/env ln -s /var/www/vhosts/staging.<domain>.com/shared/web/app/uploads /var/www/vhosts/staging.<domain>.com/releases/20140501145739/web/app/uploads on 23.253.218.207
INFO [34c31a23] Finished in 0.274 seconds with exit status 0 (successful).
INFO [e530ab1c] Running /usr/bin/env composer install --no-dev --no-scripts --quiet --optimize-autoloader on 23.253.218.207
INFO [e530ab1c] Finished in 4.937 seconds with exit status 0 (successful).
INFO [c5d921db] Running /usr/bin/env rm -rf /var/www/vhosts/staging.<domain>.com/current on 23.253.218.207
INFO [c5d921db] Finished in 0.256 seconds with exit status 0 (successful).
INFO [9ff5283e] Running /usr/bin/env ln -s /var/www/vhosts/staging.<domain>.com/releases/20140501145739 /var/www/vhosts/staging.<domain>.com/current on 23.253.218.207
INFO [9ff5283e] Finished in 0.351 seconds with exit status 0 (successful).
INFO [ac0b23f1] Running /usr/bin/env echo "Branch master (at 5a4c50f) deployed as release 20140501145739 by oran; " >> /var/www/vhosts/staging.<domain>.com/revisions.log on 23.253.218.207
INFO [ac0b23f1] Finished in 0.277 seconds with exit status 0 (successful).
$
$ bundle exec cap staging deploy
INFO [5f1ed629] Running /usr/bin/env mkdir -p /tmp/staging.<domain>.com/ on 23.253.218.207
[....]
INFO [af21e721] Running /usr/bin/env ln -s /var/www/vhosts/staging.<domain>.com/shared/web/app/uploads /var/www/vhosts/staging.<domain>.com/releases/20140501140613/web/app/uploads on 23.253.218.207
INFO [af21e721] Finished in 0.242 seconds with exit status 0 (successful).
INFO [86c720dc] Running /usr/bin/env composer install --no-dev --no-scripts --quiet --optimize-autoloader on 23.253.218.207
cap aborted!
composer stdout: Nothing written
composer stderr: Nothing written
/Library/Ruby/Gems/2.0.0/gems/sshkit-1.3.0/lib/sshkit/command.rb:94:in `exit_status='
/Library/Ruby/Gems/2.0.0/gems/sshkit-1.3.0/lib/sshkit/backends/netssh.rb:142:in `block (4 levels) in _execute'
[...]
/Library/Ruby/Gems/2.0.0/gems/sshkit-1.3.0/lib/sshkit/backends/netssh.rb:54:in `instance_exec'
/Library/Ruby/Gems/2.0.0/gems/sshkit-1.3.0/lib/sshkit/backends/netssh.rb:54:in `run'
/Library/Ruby/Gems/2.0.0/gems/sshkit-1.3.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
Tasks: TOP => composer:run
(See full trace by running task with --trace)
The deploy has failed with an error: #<SSHKit::Command::Failed: composer stdout: Nothing written
composer stderr: Nothing written
There are (at least) 2 possible causes for this.
You will need to be able to execute using command composer
.
See Composer
See deploy.rb Modifications
@todo I should add a check for this as a deploy recipe.
There is an issue with the remote server's net-ssh gem version. I think it's probably gonna happen on all Rackspace hosting :(
Add the following to /Gemfile
gem 'net-ssh', '~> 2.8.1', :git => "https://github.com/net-ssh/net-ssh"
Note: See Installing New Gems
Plugins and Themes should not be uploaded to <project>/web/wp/wp-content/
!
Where possible all plugins and themes should be managed through Composer and the composer.json
file.
Blah
Plugins and Themes should not be uploaded to <project>/web/wp/wp-content/
! They should be managed with Composer, generally through WP-Packagist which provides a mirror of the WordPress plugin and theme directories as a Composer repository.
There are 2 easy ways to handle adding a new package:
- Use the comand line which will automatically
$ composer require wpackagist-plugin/akismet
Whenever you add a new plugin or update the WP version, run composer update
to install your new packages.
Dont forget to commit these changes to git in order for them to be included in the deploy.
plugins
, and mu-plugins
are Git ignored by default since Composer manages them. If you want to add something to those folders that isn't managed by Composer, you need to update .gitignore
to whitelist them:
!web/app/plugins/plugin-name
Updating your WordPress version (or any plugin) is just a matter of changing the version number in the composer.json
file.
Then running composer update
should activate the new version.
You may need to update in a couple of places, in the [truncated] example below I have had to update version 3.8
in 3 places.
"repositories": [
{
"type": "package",
"package": {
"name": "wordpress/wordpress",
"version": "3.8",
"type": "webroot",
"dist": {
"type": "zip",
"url": "https://wordpress.org/wordpress-3.8.zip"
}
}
}
],
"require": {
"php": ">=5.3.2",
"wordpress/wordpress": "3.8"
}
]
Example output of uploading from Wordpress 3.8 to Wordpress 3.9
$ composer update
Loading composer repositories with package information
Updating dependencies (including require-dev)
- Removing wordpress/wordpress (3.8)
- Installing wordpress/wordpress (3.9)
Loading from cache
Writing lock file
Generating autoload files
Optional Gems:
- net-ssh (> 2.8.1) I needed this for GAA Server
- capistrano-grunt
- capistrano-npm (This didnt work for me, probably due to a mismatch of names somewhere - ie npm -> nodejs)
Note: You may need to run # sudo gem install <gem>
when adding new gems - You should only need to run this on your local machine though AFAIK.
Blah
When adding a new plugin via "require"
, you need to run composer update
.
"wpackagist-plugin/contact-form-7":"3.8",
"wpackagist-plugin/contact-form-7-to-database-extension": "2.7"
@todo I must be able to automate updating composer...
# Needed only if Net SSH is giving trouble
gem 'net-ssh', '~> 2.8.1', :git => "https://github.com/net-ssh/net-ssh"
Needed only if Net SSH is giving trouble (probably)
# maps composer command
SSHKit.config.command_map[:composer] = "/usr/local/bin/composer"
modify Capfile to autoload custom recipes.
create the config/deploy/recipes/
directory and files
Blah - add autoloading
Blah
see .bash_profile for solution to auto-starting ssh-agent
untested: - An alternative way to install your public key in the remote machine's authorized_keys:
cat ~/.ssh/id_rsa.pub | ssh USER@HOST "mkdir -p ~/.ssh; cat >> ~/.ssh/authorized_keys"
Convert the following into a task:
root@<domain>2:/var/www/vhosts/staging.<domain>.com# ssh-add -l
Could not open a connection to your authentication agent.
root@<domain>2:/var/www/vhosts/staging.<domain>.com# eval 'ssh-agent'
SSH_AUTH_SOCK=/tmp/ssh-CdMEHBe20991/agent.20991; export SSH_AUTH_SOCK;
SSH_AGENT_PID=20992; export SSH_AGENT_PID;
echo Agent pid 20992;
root@<domain>2:/var/www/vhosts/staging.<domain>.com# ssh-add -l
- @todo Update repo to point to something I control - add recipes, autoloads, additional plugins, wp-cli setup
- @todo Update installation script to prompt for environment variables as laid out in Deployment - Quick Start.md
- @todo Update composer.json to pull the latest WP version, not a fixed version