Installing Jenkins from scratch

Jenkins.md

Installing Jenkins from scratch

This is how I install Jenkins on a bare system. I'm using Ubuntu 12.04 32bit

First Steps

System Config

Always good to update the system packages after a fresh install:

# apt-get update
# apt-get -y upgrade

Added SSH Security:

# echo "PasswordAuthentication no" >> /etc/ssh/sshd_config

Jenkins

Instructions adapted from https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+Ubuntu

Version I'm currently using: Jenkins ver. 1.522

# wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | apt-key add -
# echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list
# apt-get update
# apt-get -y install jenkins

Now for security we configure jenkins to listen only on localhost:

# vi /etc/default/jenkins

In the last line, add to JENKINS_ARGS: --httpListenAddress=127.0.0.1

nginx

Jenkins will be publicly accessible only through nginx.

Note: If you plan to access Jenkins through the public internet then it is important to use SSL with nginx. In addition to just build status, it is also possible to access through Jenkins your entire source code tree!

Instructions adapted from https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+Ubuntu and adjustments for SSL support from: https://gist.github.com/rdegges/913102/#comment-198697

# apt-get -y install nginx

Remove default configuration:

# rm /etc/nginx/sites-available/default

Upload SSL certificate files yoursite.com.crt and yoursite.com.key to someplace on the server, and install them to:

# mkdir -p /etc/nginx/ssl
# mv yoursite.com.crt /etc/nginx/ssl/yoursite.com.crt
# mv yoursite.com.key /etc/nginx/ssl/yoursite.com.key

Make sure that only root can read these! And make sure that the server is locked down to protect these: any one of your developers can use build jobs to run arbitrary commands as the jenkins user.

# chown root:root /etc/nginx/ssl/*
# chown root:root /etc/nginx
# chmod 600 /etc/nginx/ssl/*
# chmod 700 /etc/nginx/ssl

Configure nginx (replace yoursite.com):

# cat > /etc/nginx/sites-available/jenkins

ssl_certificate     /etc/nginx/ssl/yoursite.com.crt;
ssl_certificate_key /etc/nginx/ssl/yoursite.com.key;

server {
    listen 80;

    rewrite ^(.*) https://ci.yoursite.com/ permanent;
}

upstream jenkins {
    server 127.0.0.1:8080 fail_timeout=0;
}

server {
    listen 443 ssl;

    server_name ci.yoursite.com;

    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect http:// https://;

        proxy_pass http://jenkins;
    }
}

Apply the new config:

# ln -s /etc/nginx/sites-available/jenkins /etc/nginx/sites-enabled/
# service nginx restart

Jenkins should now be accessible at https://ci.yoursite.com

Update the default Jenkins Plugins

1. Go to https://ci.yoursite.com

2. Jenkins -> Manage Jenkins -> Manage Plugins

3. Update all the installed plugins with "Download now and install after restart"

Must-do Jenkins Configuration

Jenkins -> Manage Jenkins -> Configure System

# of executors: For small servers set this to 1

Jenkins Location

• Set Jenkins URL to https://ci.yoursite.com

• Set a System Admin e-mail address to something useful. (TODO: What exactly is this email used for?)

More Good Stuff that should be Added

Now that we have a bare-bones Jenkins install working, let's add more things that I use.

Github OAuth Plugin

This plugin uses your GitHub organization to authenticate users for Jenkins.

Log into the GitHub website and "Register new application":

• Application name: Set this to anything such as "Jenkins"

• HomePage URL: Set this to https://ci.yoursite.com

• Authorization callback URL: Set this to https://ci.yoursite.com/securityRealm/finishLogin

After creating the application, take note of its "Client ID" and "Client Secret".

Go back to Jenkins, and install the "Github OAuth Plugin".

Then go to: Jenkins -> Manage Jenkins -> Configure Global Security

Check Enable security

Under "Security Realm" choose "Github Authentication Plugin" and enter the "Client ID" and "Client Secret" of the GitHub application.

Under "Authorization" choose "Github Commiter Authorization Strategy":

• Admin User Names: Add GitHub usernames here

• Participant in Organization: You should set this

• Grant READ permissions to all Authenticated Users: Dangerous, don't use!

• Grant READ permissions for /github-webhook: Enable this

• Grant READ permissions for Anonymous Users: Dangerous, don't use!

Make sure to also enable "Prevent Cross Site Request Forgery exploits" and choose the "Default Crumb Issuer" (Do not enable "Enable proxy compatibility")

Git

# apt-get -y install git

To allow jenkins to pull the code from GitHub, we will use the approach "Machine users" described here: https://help.github.com/articles/managing-deploy-keys#machine-users

Create an SSH key for the jenkins system user:

# sudo -u jenkins -i
$ ssh-keygen -t rsa -C jenkins

Make sure not to set a passphrase for the key!

Create a new GitHub account and attach the public key (located in /var/lib/jenkins/.ssh/id_rsa.pub)

While you are still logged into GitHub, create a "Personal API Access Token". This will be needed later for the Jenkins GitHub Plugin (You can give the token the name "Jenkins GitHub Plugin").

Verify the connection to GitHub. This step is important in order to establish GitHub's RSA key fingerprint:

$ ssh [email protected]

When prompted with the RSA key fingerprint make sure to type "yes".

You should see the message: "You've successfully authenticated"

Now set the git name and email configuration for jenkins:

$ git config --global user.name jenkins
$ git config --global user.email noemail

This needs to be set because by default Jenkins will create a git tag for each build. If git isn't configured with a name and an email then it will give an error when trying to create a tag.

$ exit

Jenkins Git Plugin

https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin

Need this to enable Jenkins to check out git repositories.

Jenkins GitHub Plugin

https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Plugin

GitHub integration including automatic triggering of builds, and pull request commit status updating.

Jenkins -> Manage Jenkins -> Configure System

Find the section "GitHub Web Hook".

You must use the option "Let Jenkins auto-manage hook URLs" and supply the OAuth token of the GitHub machine user we created. Jenkins needs the OAuth token so that it can set the GitHub commit status: https://github.com/blog/1227-commit-status-api

PostgreSQL Database

For testing apps that need a database here is how I create a PostgreSQL database and user that can be used by jenkins:

# apt-get -y install postgresql
# sudo -u postgres -i

$ export DB_NAME=jenkinsdb
$ export DB_USER=jenkins
$ export DB_PASS=jenkins123

$ createuser --no-superuser --createdb --no-createrole $DB_USER
$ psql -c "ALTER USER $DB_USER WITH PASSWORD '$DB_PASS'"
$ createdb -O $DB_USER $DB_NAME

$ exit

Give easy database access to Jenkins builds by going to:

Jenkins -> Manage Jenkins -> Configure System

Find the section "Global properties"

Set some environment variables that build scripts can use:

JENKINS_DATABASE_NAME=jenkinsdb
JENKINS_DATABASE_USER=jenkins
JENKINS_DATABASE_PASSWORD=jenkins123

Jobs Philosophy

Idea taken from http://bruno.im/2012/oct/24/github-pull-requests-status-jenkins/ where it is described best:

For each repository, create two builds:

• One which builds your default branch (master) and sends you notifications of all sorts (email, irc, etc.)

• One which builds all branches (put ** in the branch specifier) and doesn't notify.

All builds will update [GitHub commit] statuses but with two builds with separate notifications handling you can keep a clean build for your master branch. You don't want to get an email each time a builds fails in an experimental branch. Also, with a single build Jenkins is unable to tell the build status of each branch. It'll just consider the last build as the status for your repository, so having two separate builds makes it easier to determine whether you have something critical to fix or not.

You can also create a custom view to only display master builds on the Jenkins homepage.

Ok. So for each project there will be 2 jobs: "myproject" and "myproject-unstable"

It is easiest to first create the "myproject-unstable" job, configure it, and make sure it is working by doing some commits in a test branch. Then you can create the "myproject" job by doing: Jenkins -> New Job -> Copy existing Job

Some notes for configuring a job:

• Set the GitHub project

• For the Git Repository URL use the format [email protected]:user/repo.git

• For the Git Branch Specifier:

  • For the main job use master (or the name of your main stable branch)

  • For the unstable job use ** to build all branches

  • Create additional jobs for more specific branches if needed

• There are lots of other Git options that might be useful

• In Build Triggers enable "Build when a change is pushed to GitHub"

• Make sure to add as a "Post-build Action" the item "Set build status on GitHub commit". Make sure to always keep this as the last Post-build Action. This relies on the OAuth token being set in the "GitHub Web Hook" config.

Useful plugins

SLOCCount Plugin

https://wiki.jenkins-ci.org/display/JENKINS/SLOCCount+Plugin

You need the command line tool installed:

# apt-get -y install sloccount