Building Armbian using Ubuntu (jammy) in a systemd-nspawn container

These are some steps for building armbian using Ubuntu (jammy) in a systemd-nspawn container. This is an alternative for building armbian from a Linux system (as like docker containers).

However this method requires the PREFER_DOCKER=no build flag as docker engines cannot be run within a systemd-nspawn container.

install
Start the systemd-nspawn container
Building Armbian in systemd-nspawn container
loop devices in systemd-nspawn (workaround) requires editing the armbian-build scripts

ref:

pre-requisites

pre-requieites:

In the host Linux system:
- systemd
- systemd-nspawn
- chroot
requires:

In the host Linux system:
- debootstrap
  
  Setting up the container is by means of debootstrap
  apt-get install debootstrap For non-debian or ubuntu systems, the tar source packages can be download from https://packages.debian.org/bookworm/debootstrap placed in a directory and run from there

Install base system in chroot folder

Setup chroot folder

Make sure that the folder has enough capacity to keep like 20-30GB of files. This example folder is where an Ubuntu distribution / container lives. It would be referenced further steps, but that you can use any folder (or mount point).

sudo mkdir /opt/armbian-build

Debootstrap to setup an Ubuntu (Jammy 22.04) chroot distribution

sudo debootstrap --variant=buildd --arch=amd64 jammy /opt/armbian-build http://archive.ubuntu.com/ubuntu/

Setup and configure the chroot folder/container for systemd-nspawn container

Once done chroot into the chroot/container folder

sudo chroot /opt/armbian-build /bin/bash

Install an editor (note inside the chroot container)

root@host> apt-get install vim

add repositories in sources.list

Edit /etc/apt/sources.list (note inside the chroot container)
add security, updates suites and restricted, universe, multiverse repositories

deb http://archive.ubuntu.com/ubuntu jammy           main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu jammy-security  main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu jammy-updates   main restricted universe multiverse

note: preferably use a full sources.list from an existing Ubuntu (Jammy) image e.g. Current Ubuntu Jammy sources.list extracted from an image

Update apt indexes in container (note inside the chroot container)

apt-get update

Install systemd in container

Install systemd (this is a big one) (note inside the chroot container).

apt-get install systemd

update DNS servers for domain name lookups

Edit /etc/systemd/resolved.conf (note inside the chroot container).
Add the DNS servers that you are using in your host os. e.g.

[Resolve]
DNS=1.1.1.1 8.8.8.8

Admin setups in container

Install sudo, for the user running the build (note inside the chroot container).

apt-get install sudo

Install some useful tools (optional) (note inside the chroot container). This is your /usr/bin/ping and /usr/sbin/ip and friends.

apt-get install iputils-ping iproute2

Change root password (note this is within the chroot container)

passwd

Add a user to run the build (note inside the chroot container). The user's home folder would be /home/armbian in this example

adduser armbian
usermod -aG sudo armbian

exit the chroot container

exit

you should be back in the host shell

Start the systemd-nspawn container

Run systemd-nspawn from the host shell to start the container. (note from the host shell). Make this a shell script, and you can simply run the script. e.g.

sudo systemd-nspawn -b -D /opt/armbian-build

On running the above, you should see Ubuntu booting up. And lands you in a login prompt.

You should be able to login as root or the build user (e.g. armbian) with the password you setup in the container prior.

Verify some stuff / networks etc

e.g. login as root or the build user (e.g. armbian) and check networking (note inside the systemd-nspawn container). e.g.

ping 1.1.1.1
ping archive.ubuntu.com
ip link
ip address

It should be possible to access external networks normally from within the container. You should also see your existing network interfaces and ip addresses. In this setup, host networking is used. systemd-nspawn has other networking options, e.g. -n/--network-veth private networking, virtual ethernet interface etc. For those kindly refer to the man pages or related info. e.g.

If ping external address works, but that ping with domain names fail, check the prior step on editing /etc/systemd/resolved.conf. update DNS servers for domain name lookups

Exiting the systemd-nspawn container

It should be possible to shutdown the container from within. (note inside the systemd-nspawn container). e.g.

systemctl poweroff

That should shut it down and return you to the host shell/prompt

Building Armbian in systemd-nspawn container

Building Armbian in the systemd-nspawn container is similar to the documented build steps. However this method requires the PREFER_DOCKER=no build flag as docker engines cannot be run within a systemd-nspawn container.

building armbian from armbian docs

Note that all these steps are done from within the systemd-nspawn container. Preferably run these with the user created for the build instead of root.

pre-requisites

Per the Armbian docs:

1 sudo apt-get install git  
2 git clone --depth=1 --branch=main https://github.com/armbian/build  
3 cd build

running compile.sh in container *

compile.sh must be run with PREFER_DOCKER=no

./compile.sh PREFER_DOCKER=no

Errors / Limits

Currently systemd-nspawn do not support loop devices needed in the full_build_packages_rootfs_and_image() stage, it will fail there. Hence, building of the images would need to be manually done outside the container

loop devices in systemd-nspawn

Currently systemd-nspawn do not support loop devices needed in the compile.sh build. This section documents some workarounds to create loop devices in a systemd-nspawn container. Use this shell script to start systemd-nspawn

#!/usr/bin/bash
sudo systemd-nspawn -b --capability=CAP_MKNOD \
        --property=DeviceAllow="block-loop rwm" \
        --property=DeviceAllow="block-blkext rwm" \
        --property=DeviceAllow="/dev/loop-control rwm" \
        -D /opt/armbian-build

when the container is started up, in the shell within the container, use this shell script to create the loop devices

#!/usr/bin/bash
if ! test -e /dev/loop-control; then
  sudo mknod /dev/loop-control c 10 237
fi

for i in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do 
if ! test -e /dev/loop${i}; then
  sudo mknod /dev/loop${i} b 7 ${i}
fi
done

In current tests, the above did create the loop devices, but is inadequate to fully resolve the issue with loop devices in armbian build compile.sh.

loop devices in systemd-nspawn (workaround) requires editing the armbian-build scripts

#!/usr/bin/bash
sudo systemd-nspawn -b --capability=CAP_MKNOD \
        --property=DeviceAllow="block-loop rwm" \
        --property=DeviceAllow="block-blkext rwm" \
        --property=DeviceAllow="/dev/loop-control rwm" \
        -D /opt/armbian-build

In the container in the Armbian build folder:
lib/functions/image/loop.sh add the functions create_loop_device() and dolosetup()
lib/functions/image/loop.sh.diff

*** lib/functions/image/loop.sh	2024-01-16 14:22:26.657901025 +0800
--- lib/functions/image/loop.sh	2024-01-16 15:10:10.615468807 +0800
***************
*** 115,117 ****
--- 115,177 ----
  	fi
  	losetup -d "${1}"
  }
+ 
+ 
+ function create_loop_devices() {
+ 	if ! test -e /dev/loop-control; then
+ 	  display_alert "creating loop control device" "/dev/loop-control" 
+ 	  run_host_command_logged mknod /dev/loop-control c 10 237
+ 	fi
+ 	
+ 	local i
+ 	for i in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do 
+ 	if ! test -e /dev/loop${i}; then
+ 	  display_alert "creating loop device" "/dev/loop${i}" 
+ 	  run_host_command_logged mknod /dev/loop${i} b 7 ${i}
+ 	fi
+ 	done
+ }
+ 
+ function dolosetup(){
+ 	local loopdev=$1
+ 	local image_file=$2
+ 	if test -z "${image_file}"; then
+ 	  exit_with_error "image file not specified"
+ 	fi
+ 	if ! test -e $image_file; then
+ 	  exit_with_error "image file ${image_file} not found"
+ 	fi
+ 	
+ 	if test -z "${loopdev}"; then
+ 	  exit_with_error "loop device not specified"
+ 	fi
+ 	display_alert "loop device" "${loopdev}" "debug"
+ 	if ! test -e ${loopdev}; then
+ 	  exit_with_error "image file ${loopdev} not found"
+ 	fi
+ 
+ 	run_host_command_logged losetup -P ${loopdev} ${image_file}
+ 	#check that loop device is linked
+ 	local lochk=$(losetup -j ${image_file} |sed 's/: .*$//')
+ 	if test "${lochk}" != "${loopdev}" ; then
+ 	  exit_with_error "losetup unable to setup ${loopdev}"
+ 	fi
+ 	
+ 	# drop the first line, as this is our loopdev itself, but we only want the child partitions
+ 	local partitions=$(lsblk --raw --output "MAJ:MIN" --noheadings ${loopdev} | tail -n +2)
+ 	if echo ${partitions} | egrep -q "[[:digit:]]:[[:digit:]].*" ; then
+ 	  local counter=1
+ 	  for i in $partitions; do
+ 	    local maj=$(echo $i | cut -d: -f1)
+ 	    local min=$(echo $i | cut -d: -f2)
+ 	    display_alert "partition" "${loopdev}p${counter} b ${maj} ${min}" "debug"
+ 	    if [ ! -e "${loopdev}p${counter}" ]; then
+ 	      display_alert "create partition loop device" "${loopdev}p${counter} b ${maj} ${min}" 
+ 	      mknod ${loopdev}p${counter} b ${maj} ${min}
+ 	    fi
+ 	    counter=$((counter + 1))
+ 	  done
+ 	else
+ 	  exit_with_error "cannot get MAJ:MIN ${maj}:${min} from lsblk"
+ 	fi
+ }

lib/functions/image/partitioning.sh edit the function prepare_partitions()
lib/functions/image/partitioning.sh.diff

*** lib/functions/image/partitioning.sh	2024-01-16 14:22:14.625947725 +0800
--- lib/functions/image/partitioning.sh	2024-01-16 14:16:18.071333320 +0800
***************
*** 214,226 ****
  	exec {FD}> /var/lock/armbian-debootstrap-losetup
  	flock -x $FD
  
  	declare -g LOOP
  	LOOP=$(losetup -f) || exit_with_error "Unable to find free loop device"
  	display_alert "Allocated loop device" "LOOP=${LOOP}"
  
  	CHECK_LOOP_FOR_SIZE="no" check_loop_device "$LOOP" # initially loop is zero sized, ignore it.
  
! 	run_host_command_logged losetup "${LOOP}" "${SDCARD}".raw # @TODO: had a '-P- here, what was it?
  
  	# loop device was grabbed here, unlock
  	flock -u $FD
--- 215,229 ----
  	exec {FD}> /var/lock/armbian-debootstrap-losetup
  	flock -x $FD
  
+ 	create_loop_devices
  	declare -g LOOP
  	LOOP=$(losetup -f) || exit_with_error "Unable to find free loop device"
  	display_alert "Allocated loop device" "LOOP=${LOOP}"
  
  	CHECK_LOOP_FOR_SIZE="no" check_loop_device "$LOOP" # initially loop is zero sized, ignore it.
  
! 	#run_host_command_logged losetup "${LOOP}" "${SDCARD}".raw # @TODO: had a '-P- here, what was it?
! 	dolosetup "${LOOP}" "${SDCARD}".raw 
  
  	# loop device was grabbed here, unlock
  	flock -u $FD

the above 2 patches may be applied by changing to the directory lib/functions/image and running patch e.g.

> cd lib/functions/image
> patch -b < loop.sh.diff
patching file loop.sh

it is recommended to specify the -b option for patch, the original file would be backup to filename.orig

after patching it run compile.sh with PREFER_DOCKER=no

./compile.sh PREFER_DOCKER=no

Current tests runs compile.sh PREFER_DOCKER=no successfully to completion and produce the images. However, it does so by patching loop devices from outside the container, hence it isn't risk free to the host operating system. (use at your own risk)

The loop partition device(s) /dev/loopXpY, is left in /dev after the run, it is recommended to delete/remove the device node/file after each run. This would prevent it from picking up a partition that is mapped to something else on re-run.

Running this with compile.sh requires root access in the container, as losetup connects the image files to the loop devices to build the images.

Appendix

Current Ubuntu Jammy sources.list extracted from an image

# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://archive.ubuntu.com/ubuntu/ jammy main restricted
# deb-src http://archive.ubuntu.com/ubuntu/ jammy main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://archive.ubuntu.com/ubuntu/ jammy-updates main restricted
# deb-src http://archive.ubuntu.com/ubuntu/ jammy-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ jammy universe
# deb-src http://archive.ubuntu.com/ubuntu/ jammy universe
deb http://archive.ubuntu.com/ubuntu/ jammy-updates universe
# deb-src http://archive.ubuntu.com/ubuntu/ jammy-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://archive.ubuntu.com/ubuntu/ jammy multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ jammy multiverse
deb http://archive.ubuntu.com/ubuntu/ jammy-updates multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ jammy-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ jammy-backports main restricted universe multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ jammy-backports main restricted universe multiverse

deb http://security.ubuntu.com/ubuntu/ jammy-security main restricted
# deb-src http://security.ubuntu.com/ubuntu/ jammy-security main restricted
deb http://security.ubuntu.com/ubuntu/ jammy-security universe
# deb-src http://security.ubuntu.com/ubuntu/ jammy-security universe
deb http://security.ubuntu.com/ubuntu/ jammy-security multiverse
# deb-src http://security.ubuntu.com/ubuntu/ jammy-security multiverse

ag88/armbbnspawn.md

Building Armbian using Ubuntu (jammy) in a systemd-nspawn container

ref:

pre-requisites

Install base system in chroot folder

Setup chroot folder

Debootstrap to setup an Ubuntu (Jammy 22.04) chroot distribution

Setup and configure the chroot folder/container for systemd-nspawn container

add repositories in sources.list

Install systemd in container

update DNS servers for domain name lookups

Admin setups in container

Start the systemd-nspawn container

Verify some stuff / networks etc

Exiting the systemd-nspawn container

Building Armbian in systemd-nspawn container

pre-requisites

running compile.sh in container *

Errors / Limits

loop devices in systemd-nspawn

loop devices in systemd-nspawn (workaround) requires editing the armbian-build scripts

Appendix

Current Ubuntu Jammy sources.list extracted from an image