Ganeti On Arch

bloggin

Arch + Ganeti

As much as I am currently in love with LXC, FreeBSD can't be loaded in a container yet to my knowledge. However some people have worked out a chroot/emulation layer allowing the execution of non-native binaries  (http://wiki.debian.org/QemuUserEmulation) which is pretty cool.

This has led me back to vitalization.

I chose to solve the general problem (http://xkcd.com/974/) because I am a nerd and because ganeti 2.5 has been rumoured to support LXC. Since Version 2.2.0 it looks like.

So I looked in the AUR where there is one package, and it is marked as outdated.

So build from source.


Might as well start with this.

Create the ganeti backing store. If you have pysical volumes to create your backing on then this is pointless. Ganeti will complain if you give it less than 20G. LXC++

<pre>
dd if=/dev/zero of=ganeti_backing_store.img bs=4k count=25000000
</pre>

Prequisites to install with pacman:

<pre>
pacman -S drbd iproute pyopenssl python-pyinotify pycurl ctypes socat python-paramiko haskell python-simplejson python-pyparsing python2-pyparsing python2-pyinotify start-stop-daemon
</pre>



<pre>
wget 'http://ganeti.googlecode.com/files/ganeti-2.5.1.tar.gz'
tar -xvzf ganeti-2.5.1.tar.gz.

cd ganeti-2.5.1
</pre>

On arch the problem here is that we don't have the system python as python3. Which sorta sucks for building this.


Most of ganeti will support a 

<pre>

PYTHON=python2.7 ./configure --localstatedir=/var --sysconfdir=/etc
</pre>

however 'autotools/build-bash-completion' fails even this because it has a '#!/usr/bin/python' shebang hardcoded. Fix this with vim, changing it to '#!/usr/bin/python2.7'

Then, again
<pre>

PYTHON=python2.7 ./configure --localstatedir=/var --sysconfdir=/etc
</pre>

This will actually succeed.

<pre>
make
make install
</pre>

Fix PYTHONPATH
<pre>

ln -s /usr/local/lib/python2.7/site-packages/ganeti/ /usr/lib/python2.7/site-packages/ganeti

</pre>

Fix shebangs

<pre>
for i in `find /usr/local -mmin 8 -type f | xargs`; do head -n 1 $i | grep python && echo $i >> it; done
for i in `cat it`; do sed -i   's/#!\/usr\/bin\/python/#!\/usr\/bin\/python2.7/' $i;done
</pre>

Note the use of -mmin 8 in the find above, that was specific to me, you may need to adjust it depending on how smoothly this is going.

Create ganeti's needed dirs:

<pre>
mkidr /etc/ganeti
mkdir -p /srv/ganeti{os,export}
mkdir -p /var/{log,lib}/ganeti
</pre>

Pick a name for your cluster:

<pre>
echo 192.168.0.199 mycluster.lan mycluster  >> /etc/hosts
</pre>

If this is your first interaction with ganeti, try not to recoil at the use of /etc/hosts. Ganeti basically refuses to do anything without name service records for everything. Try not to worry too much about this. Also note that the name of the cluster is its own thing and can't be the name of the root node, it must have a unique ip and fqdn. 

Lets fire up some networking.

<pre>
modprobe bridge

vim /etc/network/br1

-------------------
DESCRIPTION='A static ethernet connection for interface eth1'    
INTERFACE='br1'                                                  
CONNECTION='bridge'                                              
BRIDGE_INTERFACES="eth1"                                         
IP='static'                                                      
ADDR='192.168.0.1'                                               
NETMASK='255.255.255.0'                                          
BROADCAST='192.168.0.255'                                        


vim /etc/rc.conf

-------------------
NETWORKS=(br1 br0)  
                    
NETWORK_PERSIST="no"

netcfg br1
</pre>

This creates a br1 interface and assigns it a static Ip address. If you want to get more fancy than this, I refer you to the arch wiki on netcfg.

Lets get some disk backing:
Make sure you created a ganeti_backing_store.img

<pre>
modprobe loop
losetup /dev/loop0 ganeti_backing_store.img
pvcreate /dev/loop0
</pre>
You can ignore any 'file descriptor leak' errors here, thats a problem with bash, not you, though it should get fixed

<pre>
vgcreate ganeti /dev/loop0
</pre>

Make sure kvm is ready to go:

<pre>
modprobe kvm 
modprobe kvm-intel
</pre>

At this point begins the period where you will attempt various incantations beginning with gnt-cluster init, they will fail, and you will have to start over.

The following I found useful.

<pre>
rm -fr /var/{log,lib}/ganeti/*
rm -fr /srv/ganeti
pkill ganeti
pkill gnt
mkdir -p /srv/ganeti{os,export}
</pre>

Create your ganeti cluster:

<pre>
gnt-cluster init --nic-parameters=link=br1 --master-netdev=br1 --enabled-hypervisor=kvm --vg-name ganeti  mycluster
</pre>