Deploying from Github to VPS using Travis CI

deploying-from-github-to-vps-using-travis-ci.md

From: https://beta.psy-dreamer.com/category/automation/deploying-from-github-to-vps-using-travis-ci

Recently, I spent around 14 to 16 hours learning all of the necessary steps to getting an existing repo set up with Travis CI to run unit tests, and then once successful, connect to a remote server that isn't a PaaS (in this case, Linode) and then proceeds to use Git hooks to do post deployment things.

Starting with your local machine and you have your project already checked out from Github.

Setting Up

• Assuming you have Ruby (at least 2.3.1) installed, run gem install travis. This installs the Travis CI command-line tools. We're going to use these tools to encrypt RSA keys that Travis will use to connect to your remote server.
• This tutorial also assumes that you have a working repo and a Travis-CI account set up.
• Create a .travis.yml file inside your project. Here is an example of how I have my file set up: .travis.yml. Keep in mind that this one is intended for a Laravel or other PHP framework project. Depending on the project, there are some things you may not need, or some additional options you might need that aren't in my file.
• Inside your repo, create a directory called .travis. You will store anything Travis-related inside this folder.

A barebones .travis.yml file:

language: node_js
node_js:
  - "8.7.0"


# line 9 resolved an issue where the Travis server would
# try to connect to my remote using ssh-dss, which then caused my
# remote server to puke and demand a password, causing the build
# to time out
dist: trusty
sudo: false

# This keeps the "add <your-host.tld> to known_hosts" prompt from popping up
addons:
  ssh_known_hosts:
  - <your-host.tld>:<portnumber> # if your server uses a different port for ssh

before_install:
- <openssl line generated by travis encrypt>
- eval "$(ssh-agent -s)"
- cp .travis/id_rsa ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- ssh-add ~/.ssh/id_rsa

deploy:
  - provider: script
    skip_cleanup: true
    script: <shell script/command that runs on deploy>
    on:
      branch: master

Creating SSH Keys

The next section assumes you have a separate user setup on your remote server for deploying. This user has to be part of the same group as the remote version of your project. This is the user you will use in your deploy.sh file to handle git push production master. Now, run cd .travis and create a .gitignore file inside. Add these two lines:

deploy_rsa.pub
deploy_rsa

Why? In the next step, we're going to create our ssh keys and this will ensure that we don't accidentally commit them.

• While inside your .travis folder, create a new ssh key by running ssh-keygen -t rsa -b 4096 -C '[email protected]' -f deploy_rsa Double-check the .travis folder and confirm that you have two files: deploy_rsa and deploy_rsa.pub
• Remember the Travis CI command-line tools we installed earlier? We're going to use them here. Inside your .travis folder, run travis encrypt-file deploy_rsa --add This will create a new file called deploy_rsa.enc. The --add flag will add a new section to your .travis.yml folder, which will look something like this:

before_install:
- openssl aes-256-cbc -K $encrypted_e92badce6860_key -iv $encrypted_e92badce6860_iv
  -in id_rsa.enc -out id_rsa -d

• The -in id_rsa.enc -out id_rsa -d line will need to be changed to -in .travis/id_rsa.enc -out .travis/id_rsa -d, matching our directory structure. The encrypt tool doesn't take into account which directory the files are located in. It always assumes that the id_rsa.enc file is inside the root of the project. Now, go ahead and add .travis/id_rsa.enc and .travis.yml to your repo.
• Now, we add our deploy key to our remote server. That can be accomplished by running ssh-copy-id -i .travis/deploy_rsa.pub <ssh-user>@<deploy-host> or ssh-copy-id -i .travis/deploy_rsa.pub <ssh-user>@<deploy-host>, add -p <port> if your ssh port is set to a different port than 22. You can also run this line first with the -n flag, which does a dry-run first before installing the key.

If you're working with the barebones .travis.yml file, you can skip this part. Otherwise, continue on. Inside the .travis.yml file, find the before_install section. Below the line - openssl ..., add the following lines:

- eval "$(ssh-agent -s)"
- cp .travis/id_rsa ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- ssh-add ~/.ssh/id_rsa

These lines set up the ssh-agent and the ssh keys on the Travis worker, we're doing this on before_install so that it is ready to go once the build completes.

The deploy.sh script

The deploy.sh script is the script that'll kick off the deployment once the build finishes successfully. For now, we'll have a barebones script set up to test our ssh connection:

#!/bin/bash
set -e
ssh <user>@<host.tld> [-p port] -v exit

• pro-tip: remember to set permissions on the deploy.sh before committing. chmod +x .travis/deploy.sh

Setting up the deployment process

Create a new directory on your live server, then run git init inside your new directory. After that, run git config receive.denyCurrentBranch updateInstead This prevents the "refusing to update checked out branch" error that crops up when you try to link two repos together.

Inside your .travis/deploy.sh script, replace the ssh line with:

git config --global push.default simple # we only want to push one branch — master
# specify the repo on the live server as a remote repo, and name it 'production'
# <user> here is the separate user you created for deploying
git remote add production ssh://<user>@<host[:port]>/<path-to-live-repo>
git push production master # push our updates

deploy section

The deploy section handles deployment configuration, and is normally used for 3rd party platforms (Additional reading: Travis CI Deployment). However, you can also use it for custom deployments and you do that by specifying the provider as script, and then pass the script to be run (or a shell command) as your script. You can further configure your deployment strategy by locking it down to one branch, in this case, we're using master as our live branch. We do this by using the on section.

deploy:
- provider: script
  skip_cleanup: true
  script: ".travis/deploy.sh"
  on:
    branch: master

Finish

Now, commit everything and push it to your Github account. Assuming that you already have your repo synced with Travis-CI, this should trigger a build. Provided everything has been set up right, your build should finish successfully.

Issues I ran into while writing this tutorial:

• Executable permissions weren't set on deploy.sh prior to committing
• (user error) Wrong ssh keys were provided for the user I was trying to configure
• If your ssh server is configured to use a different port from the standard port 22, this needs to be included in the known_hosts section of your travis.yml file, example:

  addons:
    ssh_known_hosts:
    - psy-dreamer.com:9922
    - 72.14.176.51:9922

• Wrong language or language version was set
  • In one case, I was dealing with a Laravel project and the PHP version needed was higher than 7. The Travis-CI worker defaults to PHP 5.6. I had to specify PHP 7.1.11 by using:

    language: php
    php:
    - 7.1.11

  • Another example was simply a case of me not reading the fucking manual. For language, I put html which Travis-CI doesn't recognize. When that happened, the build defaulted to Ruby and then it failed because it couldn't find a Rakefile.
  • The Travis-CI worker trying to access my server using ssh-dss — an out-of-date public key algorithm that isn't supported in later versions of OpenSSH. Needless to say, my server rejected it. You can enable ssh-dss support on your server, but it's not recommended. I fixed my builds by adding dist: trusty to my .travis.yml file. If I recall correctly, Travis will randomly pick either Ubuntu 12.04 or 14.04 for each build. This line tells Travis you want to use Trusty (14.04) instead of Precise (12.04).

More reading:

• Trusty as default Linux is coming
• Update on Trusty as default Linux

Hope you enjoyed this tutorial. The goal was to recreate and debug the steps I took to automate the deployment of Psy-dreamer.com. The initial set up took around 14 to 16 hours, most of that was trial-and-error with some derp moments thrown in.

Till next time!