balenaOS installation in a libvirt / qemu / kvm virtual machine

notes.md

balenaOS in a libvirt / qemu / kvm virtual machine

Successfully tested with Ubuntu 20.04 as the host OS in 2 quite different "environments":

• Intel NUC machine (bare metal) on my desk.
• Google Cloud Compute Engine instance:
  • Instance type: 'n2-standard-2', 8GB RAM, 2 vCPUs (Intel Cascade Lake), 10GB disk
  • Nested virtualization enabled as per docs:
    • https://cloud.google.com/compute/docs/instances/nested-virtualization/overview
    • https://cloud.google.com/compute/docs/instances/nested-virtualization/enabling

To save you time: AWS does not support nested virtualizaton unless using expensive *.metal instance types.

Host OS system setup

Based on guide: https://ubuntu.com/server/docs/virtualization-libvirt

sudo apt-get update && sudo apt-get upgrade -y
sudo apt-get install -y qemu-kvm libvirt-daemon-system virtinst unzip
sudo adduser "${USER}" libvirt
sudo reboot

Download and configure an intel-nuc balenaOS image

You can download a pre-configured image from the balenaCloud web dashboard, but here I used the balena CLI to download an unconfigured image and then configure it.

OS_VERSION="v2.83.18+rev1.dev"
IMG="${HOME}/Downloads/nuc-${OS_VERSION}.img"

cd "${HOME}"
mkdir -p "${HOME}/Downloads"
balena os download intel-nuc --version "${OS_VERSION}" -o "${IMG}"
balena login
balena os configure "${IMG}" --version "${OS_VERSION}" --fleet test-nuc --dev --config-network ethernet

Install balenaOS with virt-install

NAME='balena-test'
ARCH='--cpu host --vcpus=2'
IMG="${HOME}/Downloads/nuc-${OS_VERSION}.img"
RAM='2048'
VNC='vnc,password=foobar78,listen=0.0.0.0,port=55900'
STORAGE='size=4'  # or e.g. STORAGE='path=/some/partition/mount/point'

virt-install --debug \
  --name "${NAME}" \
  --description "${NAME}" \
  --os-type=ubuntu20.04 \
  ${ARCH} \
  --ram=${RAM} \
  --graphics ${VNC} \
  --disk "${STORAGE},bus=sata,boot.order=1" \
  --disk "${IMG},bus=sata,boot.order=2" \
  --boot uefi

I understand that the command above executes the NUC flasher image attached to the second disk. The flasher image then flashes (installs) balenaOS to the first disk, and then powers off the virtual machine. I did not really "see" it happening, but I suppose that if I had connected a VNC session quickly enough, I would have seen some console messages regarding the flashing process.

List and execute the balenaOS virtual machine with virsh

$ virsh list --all
 Id   Name          State
------------------------------
 -    balena-test   shut off

$ virsh start balena-test
Domain balena-test started

$ virsh list --all
 Id   Name          State
-----------------------------
 3    balena-test   running

A minute or so later, I could see the balenaOS "virtual device" listed in the balenaCloud web dashboard for my 'test-nuc' fleet, and I was able to ssh to balenaOS.

Related @toochevere's script:
https://github.com/balena-io-playground/devenv-toolkit/blob/3c5a3dc0dbff691368a04774d557dedb257593e0/qemu-device-create