my_hetzner_xen_setup.md

Hetzner Primary IPv4 IP + IPv4/2x Subnet + "Non-Routed" IPv6/64 Subnet HowTo

... so I do not forget the next time I have to figure this stuff out. And perhaps to help other poor souls fiddling with v6/v4 xen setups in a Hetzner network environment.

Basic setup

You can basically follow along the Xen Project Beginners Guide.

The short version

Install Debian Wheezy via Hetzner's installimage on the rescue system, the only important part about partitioning is that you have an LVM volume group named vg0 with enough space for your guests' disks.

Stuff you will need:

apt-get install bridge-utils xen-linux-system xen-tools
dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen
update-grub

and to switch from xm to xl set TOOLSTACK=xl in /etc/default/xen, then reboot or do service xen stop && service xen start

Dom0

dom0:/etc/network/interfaces

auto lo
iface lo inet loopback

auto  eth0
iface eth0 inet static
  address   <PrimaryIP>
  broadcast <PrimaryIP_Broadcast>
  netmask   <PrimaryIP_Netmask>
  gateway   <PrimaryIP_Gateway>
  up route add -net <PrimaryIP_NET> netmask <PrimaryIP_Netmask> gw <PrimaryIP_Gateway> eth0

iface eth0 inet6 static
  address <IPv6_Prefix>::1
  netmask 128
  gateway <IPv6_Gateway>

auto br0
iface br0 inet static
  address       <PrimaryIP>
  netmask       255.255.255.255
  bridge_ports  none
  bridge_stp    off
  bridge_fd     0
  pre-up        brctl addbr br0
  up            ip -4 route add <Available_Subnet_v4_IP_Dom0>/32 dev br0
  up            ip -4 addr add <Available_Subnet_v4_IP_Dom0>/<Subnet_Mask> dev br0
  down          ip -4 route del <Available_Subnet_v4_IP_Dom0>/32 dev br0
  down          ip -4 addr del <Available_Subnet_v4_IP_Dom0>/<Subnet_Mask> dev br0

iface br0 inet6 static
  address <IPv6_Prefix>::1
  netmask 64

####In a nutshell:

• Leave primary IP config as autogenerated by Hetzner
• Add an IP Address out of the /64 Pool to eth0 with /128 netmask
• create a bridge with no bridged devices
• add an IP out of the assigned additional IP subnet with a /32 netmask (used by XenU Domains for routing)
• add a route for the additional subnet with the appropriate netmask
• add the same IPv6 Address to the bridge as for the eth0 device with netmask of /64

dom0:/etc/xen/xend-config.sxp

Default configuration can be used as of Xen 4.1.4-3+deb7u2

# -*- sh -*-
(vif-script vif-bridge)
(dom0-min-mem 512)
(enable-dom0-ballooning no)
(total_available_memory 0)
(dom0-cpus 0)
(vncpasswd '')

####In a nutshell:

• leave default values, optionally restrict dom0 resources (min-mem, ballooning, cpu)
• no network-script as we use a bridge configured via etc/network/interfaces
• vif-bridge script will pick up the first available bridge device.

dom0:/etc/sysctl.conf

net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.default.forwarding = 1
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.proxy_ndp = 1
net.ipv6.conf.all.proxy_ndp = 1

####In a nutshell:

• enable ipv4 forwarding for all existing and all future devices
• enable ipv6 forwarding for all existing and all future devices
• enable ipv6 ndp proxy forwarding for all existing and all future devices
• ndp proxy is needed to properly "announce" XenU domain's IPv6 addresses. Either manually ( ip -6 neigh add proxy <VM-IPv6-IP> dev eth0) or by using ndppd to "announce" all IPs from the /64 v6 subnet in bulk.

dom0:/etc/ndppd.conf (Optional, read anyway!)

This is only required when you use the "unrouted" IPv6 subnet as assigned by Hetzner and do not plan on registering every XenU VM IP manually via ip -6 neigh add proxy <VM-IPv6-IP> dev eth0. Instead of doing this you can run ndppd which will answer all ND requests regardless if they are actually connected to a VM or not.

route-ttl 30000
proxy eth0 {
   router yes
   timeout 500
   ttl 30000
   rule <IPv6_Prefix>::/64 {
    auto
  }
}

####Alternatives:

• manually register each VM IP using ip -6 neigh add proxy <VM-IPv6-IP> dev eth0
• change the vif-script to do the registering automatically on vm creation

XenU

dom0:/etc/xen/example-vm.cfg

kernel      = '/boot/vmlinuz-3.2.0-4-amd64'
ramdisk     = '/boot/initrd.img-3.2.0-4-amd64'
extra       = "earlyprintk=xen console=hvc0"
vcpus       = '4'
memory      = '512'
root        = '/dev/xvda2 ro'
disk        = [
                  'phy:/dev/vg0/vmtest1-disk,xvda2,w',
                  'phy:/dev/vg0/vmtest1-swap,xvda1,w',
              ]

name        = 'example-vm'

vif         = [ 'ip=<Available_Subnet_v4_IP_XenU> <Available_v6_IP> ,mac=<ETH0_MAC> ,bridge=br0' ]

on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'

In a nutshell

• Choose any available IPv4 (Available_Subnet_v4_IP_XenU) and IPv6 IPs (Available_v6_IP_XenU) from your subnets
• Configuring the IPv6 IP is optional here and only required if you plan on extending the vif-bridge script and implement automatic ip -6 neigh add proxy <VM-IPv6-IP> dev eth0 registering.
• "Clone" eth0 MAC Address
• Optional: add kernel parameters to work around virtual console issues.
• Pay attention to the kernel / ramdisk settings. These need to match your Xen0 kernel. Alternatively use xen-create-imageto bootstrap your VM and autogenerate the config.

###domU:/etc/network/interfaces

auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
 address <Available_Subnet_v4_IP_XenU>
 gateway <Available_Subnet_v4_IP_Dom0>
 netmask <Subnet_Netmask>
 broadcast <Subnet_Broadcast>

iface eth0 inet6 static
 address <Available_v6_IP_XenU> 
 gateway <IPv6_Gateway> 

In a nutshell

• configure IPv4 to use the previously (dom0:/etc/xen/example-vm.cfg) selected IPv4 Address, use netmask, broadcast and gateway as dictated by Hetzner
• configure IPv6 to use any available IP address from your /64 pool as defined in (dom0:/etc/xen/example-vm.cfg)
• Use the Xen0's v6 IP Address and make sure that this very address is assigned to eth0 with /128 and to br0 with /64. Alternatively use RADVD on Xen0 and fe80::1 as your Gateway here.
• use the Xen0's br0 v4 IP as your v4 gateway.

Please jump to dom0:/etc/ndppd.conf section above if you have not read it yet and have not decided yet how the NDP requests are to be handled. If not configured v6 might work for a while and stop being routed correctly out of the blue.

Once the VM starts up the vifX.Y device will be added automatically to the br0 bridge. It can be verified on Dom0 via:

root@xen2 ~ # brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.feffffffffff       no              vif2.0

Links & Sources

• ndppd proxy daemon: http://priv.nu/projects/ndppd/
• ND explained: http://www.ipsidixit.net/2010/03/24/239/
• Alternative ND proxy daemon: https://github.com/npd6/npd6 (seems to be more active that ndppd)

##TODO I am not very happy with the <FOO_This_And_That> variable naming scheme. I tried to keep the naming consistent across all configuration sections in this document though. If you have a better Idea - go ahead and fork off and/or let me know in the comments ;)