How to set up static nspawn IP address networking without a bridge

README.md

How to set up static nspawn IP address networking without a bridge

Note - this article assumes you have reasonable familiarity with systemd, networkd and nspawn/machinectl

What is this guide for

There seem to be a lot of references on the internet for nspawn networking with a bridge - but not on simply using the stock veth networking with static addresses. The default behaviour of nspawn (as of 24/03/20 and systemd:245 on Arch) is to:

• Make a virtual ethernet port on the host (defined by /lib/systemd/network/80-container-ve.network)
• Make a virtual ethernet port in the container (defined by /lib/systemd/network/80-container-host0.network)
• Based on those configurations, serve the host port a random IP from the local network ranges with a 28 bit subnet, and a DHCP server for the host port serves a random IP in the subnet to the container

And with that - internet "just works". Ain't that nice

What I was trying to do

Essentially exactly what the default behaviour was but with two key changes

• The host port should have a statically set IP - so that applications on the host expecting output from the container can bind to a specific IP range via their own configs
• Set the container port statically as well - so a forwarding application (like NGINX) knows where to send traffic

No bridges - just virtual ethernet ports. If this sounds like you - read on.

How to do it

First - systemd-networkd will evaluate matches for assinging configs to ports based on alphabetical order. So - we need to inject config before we hit 80-.

To keep things simple - make sure your container is poweroff'd with machinectl.

On the host - copy the /lib/systemd/network/80-container-ve.network into /etc/systemd/network/50-mycontainer.network. Crack it open - it should look like

[Match]
Name=ve-*
Driver=veth

[Network]
# Default to using a /28 prefix, giving up to 13 addresses per container.
Address=0.0.0.0/28
LinkLocalAddressing=yes
DHCPServer=yes
IPMasquerade=yes
LLDP=yes
EmitLLDP=customer-bridge

We need to change the Name in [Match] to be the interface we expect. This is either defined by container name in machinectl to fit in the 16 character limit - or defined by the container config itself with VirtualEthernetExtra to give it a special name.

We also change the Address to be a static IP - I picked something in the 10.0.0.0/8 space, then restricted it to a 28 bit subnet (ie: 10.34.60.49/28 -> 10.34.60.48 - 10.34.60.51). This will sort the host side out.

For the container - we will need to either do

• Very restricted DHCP
• Another static .network config

Neither are amazingly hard

For DHCP - add to the 50-mycontainer.network config

[DHCPServer]
PoolOffset=2
PoolSize=1

This gives the DHCP server 1 address to serve, at 10.34.60.48 + 2 => 10.34.60.50

Doing it this way means that you have all networking controlled by the host - the container is fully stock systemd-networkd.

However - we can also do what we did for the host in the container. Start your container, log into it as root and copy /lib/systemd/network/80-container-host0.network to somewhere sane. It should look like

[Match]
Virtualization=container
Name=host0

[Network]
DHCP=yes
LinkLocalAddressing=yes

[DHCP]
UseTimezone=yes

We need to remove DHCP and instead define an Address=10.34.60.50/28, plus a Gateway=10.34.60.49 if you want this to talk to the net/wider network. I still haven't figured out how to get systemd-resolved happy with this configuration however - workaround could be just to point it directly at an internet DNS.

In summary

This should give you a predictable pair of IPs for things to talk over! systemd as always has great documentation - so if the internet is still turning up nothing its best to sit with a good cup of tea and just read, reread and reread it again.

@lamafab : 192.100.100.x belongs to public IP-space. You might want to switch it to 192.168.100.x