Build Aurora 0.7.1-SNAPSHOT on CentOS 6.5

I created a CentOS 6.5 Droplet on Digital Ocean with 8GB RAM (you'll need at least 4 for the Java portion of the build), feel free to use any other CentOS 6.5 environment that you can have root on.

Prepare the Environment

First, Mesos 0.21.0+ requires subversion 1.8+ devel package which is not available by default by yum. Add a repo that has subversion-devel 1.8 available, i.e:

Add new repo /etc/yum.repos.d/wandisco-svn.repo, with:

[WandiscoSVN]
name=Wandisco SVN Repo
baseurl=http://opensource.wandisco.com/centos/6/svn-1.8/RPMS/$basearch/
enabled=1
gpgcheck=0

Now install a bunch of packages:

yum -y update
yum groupinstall -y "Development Tools"
yum install -y python-devel java-1.7.0-openjdk-devel zlib-devel libcurl-devel openssl-devel cyrus-sasl-devel cyrus-sasl-md5 apr-devel subversion-devel wget apr-util-devel

Install Maven (used to build Mesos):

cd ~
wget http://mirror.nexcess.net/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz
tar -zxf apache-maven-3.0.5-bin.tar.gz -C /opt/
ln -s /opt/apache-maven-3.0.5/bin/mvn /usr/bin/mvn

Install Gradle (used to build Aurora):

cd ~
wget https://services.gradle.org/distributions/gradle-2.2.1-all.zip   
unzip gradle-2.2.1-all.zip

Install Python 2.7 (required by Aurora):

There's various ways to install Python 2.7, the simplest way I found to install it and to make sure it was being used when building Mesos was to do the following (thanks to http://bicofino.io/blog/2014/01/16/installing-python-2-dot-7-6-on-centos-6-dot-5/#comment-1575167554):

yum -y install centos-release-SCL
yum -y install python27

# drop into a new shell where python 2.7 is configured for the environment
scl enable python27 bash

Get Aurora

Get the latest Aurora:

cd ~  
wget https://github.com/apache/incubator-aurora/archive/master.zip -O aurora-master.zip
unzip aurora-master.zip
AURORA_HOME=/root/incubator-aurora-master

Build Mesos

This step will take a while (15 - 20min, something like that):

cd ~
mesos_version=0.21.1
MESOS_BASEURL=https://archive.apache.org/dist/mesos
wget --progress=dot "$MESOS_BASEURL/$mesos_version/mesos-${mesos_version}.tar.gz"
tar zxvf mesos-${mesos_version}.tar.gz
cd mesos-$mesos_version
mkdir build
cd build
../configure
make

Copy the python egg files into a third_party directory in the Aurora directory:

mkdir $AURORA_HOME/third_party
find . -name '*.egg' -exec cp {} $AURORA_HOME/third_party \;

Build Aurora

Build the client and admin client:

cd $AURORA_HOME
./pants binary src/main/python/apache/aurora/client/cli:aurora
./pants binary src/main/python/apache/aurora/admin:aurora_admin

Build the executors:

./pants binary src/main/python/apache/aurora/executor/bin:gc_executor
./pants binary src/main/python/apache/aurora/executor/bin:thermos_executor
./pants binary src/main/python/apache/thermos/bin:thermos_runner

# package runner within executor
python2.7 << EOF
import contextlib
import zipfile
with contextlib.closing(zipfile.ZipFile('dist/thermos_executor.pex', 'a')) as zf:
  zf.writestr('apache/aurora/executor/resources/__init__.py', '')
  zf.write('dist/thermos_runner.pex', 'apache/aurora/executor/resources/thermos_runner.pex')
EOF

Build the observer:

./pants binary src/main/python/apache/thermos/observer/bin:thermos_observer

Build the scheduler:

/root/gradle-2.2.1/bin/gradle distTar

At this point you should have a bunch of stuff under the dist directory:

[root@aurora incubator-aurora-0.7.1-master]# ll dist/
total 64520
-rwxr-xr-x 1 root root  2254309 Feb 24 05:35 aurora_admin.pex
-rwxr-xr-x 1 root root  2303226 Feb 24 05:35 aurora.pex
drwxr-xr-x 3 root root     4096 Feb 24 06:16 classes
drwxr-xr-x 2 root root     4096 Feb 24 06:16 dependency-cache
drwxr-xr-x 2 root root     4096 Feb 24 06:16 distributions
-rwxr-xr-x 1 root root 28791088 Feb 24 06:10 gc_executor.pex
drwxr-xr-x 2 root root     4096 Feb 24 06:16 libs
drwxr-xr-x 3 root root     4096 Feb 24 06:16 resources
drwxr-xr-x 2 root root     4096 Feb 24 06:16 scripts
-rwxr-xr-x 1 root root 30535717 Feb 24 06:11 thermos_executor.pex
-rwxr-xr-x 1 root root  1556039 Feb 24 06:11 thermos_observer.pex
-rwxr-xr-x 1 root root   582026 Feb 24 06:10 thermos_runner.pex
drwxr-xr-x 4 root root     4096 Feb 24 06:16 tmp

You'll want dist/*.pex and dist/distributions/aurora-scheduler-0.7.1-SNAPSHOT.tar.

Run Aurora

You can look at the upstart conf files to start the scheduler and observer(https://github.com/apache/incubator-aurora/blob/master/examples/vagrant/upstart/aurora-thermos-observer.conf)

mindscratch/aurora.md