gistfile1.txt

[[PageOutline]]

= Installing and Integrating Trac and Gitolite =

This blog post details how I install Trac and Gitolite using:

 * Trac 1.0
 * Gitolite v3
 * Ubuntu 12.04 LTS
 * Python 2.7.3
 * Postgres 9.1 for Trac's database
 * Apache 2.2.22 (Ubuntu) with mod_wsgi 3.3

The resulting system will have Trac running at `http://hostname.com` and Git repositories available to clone from `[email protected]:repo-name`.  In this setup Trac and Gitolite are tightly integrated in the following ways:

 * Git post-receive hooks are installed to notify Trac of all commits, to all branches, on all repositories (allowing Trac to be updated from commit messages, etc)
 * Trac Admin Panels provide screens for management of Gitolite SSH keys, repositories, and permissions
 * Trac Repository Browser is integrated with Gitolite permissions, so users can only view a repository in Trac if they have the "R" permission on the corresponding Gitolite repo

Some of these integration features are provided by a plugin that I've released: http://trac-hacks.org/wiki/GitolitePlugin

== System Setup ==

First we install some packages:

{{{
#!bash
apt-get update
apt-get install build-essential emacs git-core rsync apache2 libapache2-mod-wsgi postgresql postfix
}}}

=== Gitolite ===

Next is Gitolite installation.  This blog post helped me get it working: ​http://www.bigfastblog.com/gitolite-installation-step-by-step -- this is just a distillation of that post, which provides more explanation of what's happening and some extra tips.

On your local machine, create an SSH keypair called "gitolite" and then `scp gitolite.pub` to the server's `/tmp/` directory.  Then:

{{{
#!bash
adduser --system --shell /bin/bash --gecos 'git version control' --group --disabled-password --home /home/git git
chown git:git /tmp/gitolite.pub && mv /tmp/gitolite.pub ~git/
su - git
mkdir bin
cd /tmp && git clone git://github.com/sitaramc/gitolite.git && cd
/tmp/gitolite/install -to /home/git/bin/
/home/gitolite/bin/gitolite setup -pk gitolite.pub
}}}

You should now have gitolite installed and working properly.  To double check, from your local machine, run:

{{{
#!bash
cd /tmp && git clone [email protected]:gitolite-admin.git
}}}

If this prompts you for a password, something's wrong.

=== Trac ===

I installed Trac in a virtualenv under a dedicated system user's homedir, with each Trac instance living in a subdirectory of `$HOME/sites/`.

{{{
#!bash
adduser --system --shell /bin/bash --gecos 'trac project management' --group --disabled-password --home /home/trac trac
su - trac
git clone git://github.com/pypa/virtualenv.git
mkdir bin
ln -s /home/trac/virtualenv/virtualenv.py ./bin/
mkdir web
./bin/virtualenv.py web/ve
echo "Trac==1.0
-e git+git://github.com/boldprogressives/trac-GitolitePlugin.git" > web/requirements.txt
web/ve/bin/pip install -r web/requirements.txt
mkdir sites
echo "import sys
sys.stdout = sys.stderr

import os
os.environ['TRAC_ENV_PARENT_DIR'] = '/home/trac/sites'

import trac.web.main

application = trac.web.main.dispatch_request
" > web/wsgi.py
}}}

I then set up an Apache virtual host to serve Trac with mod_wsgi at http://hostname.com like so:

{{{
#!bash
a2enmod wsgi
echo "<VirtualHost *:80>
  ServerName hostname.com

  WSGIDaemonProcess site-trac user=trac group=trac threads=5 python-path=/home/trac/web/ve/lib/python2.7/site-packages
  WSGIProcessGroup site-trac

  WSGIScriptAlias / /home/trac/web/wsgi.py

  WSGIPassAuthorization On

</VirtualHost>" > /tmp/apache.conf
mv /tmp/apache.conf /etc/apache2/sites-enabled/hostname.com
service apache2 reload
}}}

If you visit http://hostname.com in a browser, it should display an "Available Projects" landing page with no projects listed.

Let's create a throwaway project to check that everything's working; just press enter a few times at the prompts:

{{{
su - trac
./web/ve/bin/trac-admin ./sites/test initenv
}}}

Now if you reload the http://hostname.com homepage in your browser, a "My Project" link should show up.  Click it; Trac's homepage should appear, with no errors.

We'll keep the testing environment around for now, so that we can use it to test Git/Trac commit integration next.

== Integrating Trac and Gitolite: Round One ==

We're now going to set up the core integration between Trac and Gitolite, including:

 * Gitolite post-receive hooks to notify Trac of new commits
 * Trac components to update or close tickets when commit messages like "Fixes #1" are pushed to the git server
 * Trac Repository Browser for git repositories

=== Getting Permissions Right ===

The most involved bit here is ensuring that the filesystem permissions are all correct, and remain correct as files are edited and created by the software over time.  Trac and Git both need some ability to read files from one another's directories.  After this, the rest of the integration is pretty straightforward.  Here's an overview of what permissions are needed:

 * For Trac to display files and changesets in its Repository Browser, the user running the Trac process needs read access to the objects in the relevant Git repositories.
 * For Gitolite's post-receive hook to notify Trac of new commits, the user running the Gitolite process needs read access to the relevant Trac instance's `conf/trac.ini` file, and read and write access to the Trac instance's database.

The strategy we'll use is to add both the "trac" and "git" system users to a shared group, and ensure that all the files they need to share are owned by that group and group-readable.

As root:
{{{
groupadd infra
usermod -a -G infra trac
usermod -a -G infra git
}}}

Now we need to fix up the permissions in the Trac instance, and ensure that they remain fixed up even if Trac edits its own files (e.g. when you use the Admin Panels to edit your `trac.ini` configuration) --
{{{
su - trac
chown -R trac:infra sites
chmod g+r sites/test/conf/trac.ini
chmod -R g+w sites/test/db/
find /home/trac/sites/ -type d -exec chmod +s {} \;
}}}

We also need to fix up the permissions for the Gitolite repositories:
{{{
su - git
chown git:infra -R /home/git/repositories/
chmod -R g+rX /home/git/repositories/
find /home/git/repositories/ -type d -exec chmod +s {} \;
echo "21c21
<     UMASK                       =>  0077,
---
>     UMASK                       =>  0027,
" > /tmp/gitolite.rc.patch
patch /home/git/.gitolite.rc < /tmp/gitolite.rc.patch && rm /tmp/gitolite.rc.patch
}}}

=== Gitolite Post Receive Hook ===

As user "git", copy this script into `/home/git/post-receive-trac` and `chmod +x` it:

{{{
#!python
#! /usr/bin/python
# -*- coding: utf-8 -*-
#
# Copyright (c) 2011 Grzegorz Sobański
#
# Version: 2.0
#
# Git post receive script developed for mlabs
# - adds the commits to trac
# based on post-receive-email from git-contrib
#

import re
import os
import sys
from subprocess import Popen, PIPE, call

# config
TRAC_ENV = '/home/trac/sites/test/'
GIT_PATH = '/usr/bin/git'
TRAC_ADMIN = '/home/trac/web/ve/bin/trac-admin'
# if you are using gitolite or sth similar, you can get the repo name from environemt
REPO_NAME = os.getenv('GL_REPO')

# communication with git

def call_git(command, args, input=None):
    return Popen([GIT_PATH, command] + args, stdin=PIPE, stdout=PIPE).communicate(input)[0]


def get_new_commits(ref_updates):
    """ Gets a list uf updates from git running post-receive,
    we want the list of new commits to the repo, that are part
    of the push. Even if the are in more then one ref in the push.

    Basically, we are running:
    git rev-list new1 ^old1 new2 ^old2 ^everything_else

    It returns a list of commits"""

    all_refs = set(call_git('for-each-ref', ['--format=%(refname)']).splitlines())
    commands = []
    for old, new, ref in ref_updates:
        # branch delete, skip it
        if re.match('0*$', new):
            continue

        commands += [new]
        all_refs.discard(ref)

        if not re.match('0*$', old):
            # update
            commands += ["^%s" % old]
        # else: new - do nothing more

    for ref in all_refs:
        commands += ["^%s" % ref]

    new_commits = call_git('rev-list', ['--stdin', '--reverse'], '\n'.join(commands)).splitlines()
    return new_commits


def handle_trac(commits):
    if not (os.path.exists(TRAC_ENV) and os.path.isdir(TRAC_ENV)):
        print "Trac path (%s) is not a directory." % TRAC_ENV

    if len(commits) == 0:
        return

    args = [TRAC_ADMIN, TRAC_ENV, 'changeset', 'added', REPO_NAME] + commits 
    with open("/tmp/trac-gitolite.log", 'a') as fp:
        fp.write(' '.join(args))

    call(args)


# main
if __name__ == '__main__':
    # gather all commits, to call trac-admin only once
    lines = sys.stdin.readlines()
    updates = [line.split() for line in lines]
    commits = get_new_commits(updates)

    # call trac-admin
    handle_trac(commits)
}}}

Then set up post-receive hooks on all repos to execute that script:
{{{
su - git
echo "#!/bin/sh

/home/git/post-receive-trac" > /home/git/.gitolite/hooks/common/post-receive && chmod +x /home/git/.gitolite/hooks/common/post-receive
./bin/gitolite setup --hooks-only
}}}

Now activate the Trac components to hook into that post-receive hook, and to enable a Git version-control backend:
{{{
#!bash
su - trac
echo "[components]
[components]
tracopt.ticket.commit_updater.committicketreferencemacro = enabled
tracopt.ticket.commit_updater.committicketupdater = enabled
tracopt.versioncontrol.git.git_fs.csetpropertyrenderer = enabled
tracopt.versioncontrol.git.git_fs.gitconnector = enabled
tracopt.versioncontrol.git.git_fs.gitwebprojectsrepositoryprovider = enabled
" >> sites/test/conf/trac.ini
}}}

=== Making sure it works ===

To test it, we'll add Gitolite's "testing.git" repo to Trac:
{{{
su - trac
web/ve/bin/trac-admin sites/test repository add testing /home/git/repositories/testing.git git
}}}

Through the web, create a ticket #1 in the Trac "test" environment.  Then, on your local machine, clone the repo, add some code, and commit with `-m "testing -- fixes #1"` before pushing.

If all goes well, Trac's "Browser" tab will show your code and commits from the "testing" repo, and ticket #1 will now be closed.