Create Raspberry Pi OS Virtual Machine with GUI

create_raspi_vm.md

VM Running Raspberry Pi OS Desktop or Lite with GUI

Here's a script that creates and runs a virtual machine running Raspberry Pi OS Desktop or Lite with a GUI. It is based on cGandom's guide on setting up a Raspberry Pi OS Lite VM without a GUI. Quoting that guide:
"This isn't full-blown hardware emulation of the Raspberry Pi 4, but more about creating a virtual environment for the OS."

The main improvements made by this script over that guide are:

• working GUI (tested with Raspberry Pi OS Desktop, as well as frame-buffer and X-Server on Raspberry Pi OS lite)
• integration with virt-manager, a GUI VM manager application
• full automation, no user interaction required (assuming that the below prerequesites are installed)

Tested on:

• Hosts: x86-64 Ubuntu 23.10 & Ubuntu 24.04, but the script below was designed to be relatively distro-agnostic
• Guests OSs: arm-64 Raspberry Pi OS Desktop Bookworm, Raspberry Pi OS Lite Bookworm
• Guest Linux kernels: 6.1.34

Prerequisites

• virt-manager
• qemu tools
• wget
• C compiler for arm64-cross-compilation

On ubuntu, the following commands should install everything you need, though I haven't tested that on a freshly installed machine:

sudo apt update

# C compilation tools
sudo apt install -y build-essential
sudo apt install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
sudo apt install -y flex bison

# install virt-manager & qemu tools
sudo apt install -y qemu-kvm libvirt-clients bridge-utils libvirt-daemon-system qemubuilder qemu-system-gui qemu-system-arm qemu-utils qemu-system-data qemu-system
sudo apt install -y virt-manager virtinst
sudo apt install -y virtiofsd # isn't always available
sudo systemctl enable --now libvirtd
sudo systemctl start libvirtd
sudo usermod -aG kvm $USER
sudo usermod -aG libvirt $USER

Explanation

Overview of this Script's Outcomes

• In the directory configured in the $WORK_DIR variable, this script creates the following files:
  • raspi_vm_disk.img: a disk image file for the VM with Raspberry Pi OS installed, with user and password preconfigured and SSH enabled
  • linux-kernel: a folder containing compiled linux and kernel modules and a kernel image which is also used by the virtual machine
• In virt-manager, registers a virtual machine by the name specified in the $VM_NAME variable.
• Starts the newly created virtual machine.

Technical Details of How this Script Works

Read cGandom's guide first.

These are the steps performed by the script below, which is well commented and readable:

1. download a Raspberry Pi OS image
2. resize the image to make more space for software-installations inside the VM guest OS
3. mount the image's partitions to make some initial configurations, namely login details and SSH
4. download, configure and compile the linux kernel
   • configurations comprise enabling kernel modules necessary for graphics
   • kernel is cross-compiled for arm64 processors, so that this script can be run on an x86-64 computer
5. mount the root filesystem partition of the Raspberry Pi OS image and install the kernel modules there
6. register a virtual machine on virt-manager from an XML definition
7. run the virtual machine
8. open virt-manager to view the VM's output
9. wait for the boot, it is normal for the VM window to display 'Guest has not initialized the display (yet).' while booting, but after a minute the console log-in or GUI Desktop or installation wizard should display, depending on the guest VM variant and configuration. In the virt-manager VM window, under the menu View > Consoles > Serial 1 you can switch to the text console while the graphics aren't running yet.
10. to take advantage of the resized disk image, you may need to run the following command from within the guest VM:

sudo raspi-config nonint do_expand_rootfs

create_raspi_vm.sh

#!/bin/bash

## Creates and runs a virtual machine running Raspberry Pi OS Desktop or Lite with a GUI.
## Read it's documentation at https://gist.github.com/emendir/922d6914a1705ed2e8e4e96db726c422
## Developed based on https://gist.github.com/cGandom/23764ad5517c8ec1d7cd904b923ad863

VM_NAME="RasPi" # name of the virtual machine registered in virt-manager

# Directory in which the VM's disk and linux kernel images will be stored
WORK_DIR=/opt/VirtualMachines/$VM_NAME

## Generate the password hash using the following command: 
# openssl passwd -6
PASSWORD_HASH='$6$FBH3uTF54pUUDpcR$IR5cMxHRhPN.DsXCUFJpRgmWL2iAZKuTaakYq7i4Y/Qz02B8ngloRrnM2K1IPkFYPCn.f/cDm7DpJa5pSglsg0'  # this value is for the password 'password'
USERNAME='pi'


# Download URL of the Linux kernel used
KERNEL_URL=https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.1.34.tar.xz

# Download URL of the Raspberry Pi OS image to use (the following options have been tested with the above kernel)
# RPOS_IMAGE_URL=https://downloads.raspberrypi.com/raspios_lite_arm64/images/raspios_lite_arm64-2024-07-04/2024-07-04-raspios-bookworm-arm64-lite.img.xz
RPOS_IMAGE_URL=https://downloads.raspberrypi.com/raspios_full_arm64/images/raspios_full_arm64-2024-07-04/2024-07-04-raspios-bookworm-arm64-full.img.xz

# Flags to force rebuilding kernel or Raspberry Pi OS images
# RECREATE_RPOS_IMAGE implies REBUILD_KERNEL
RECREATE_RPOS_IMAGE=false   # CAREFULL: will delete any existing image
REBUILD_KERNEL=false         # will reuse existing kernel build files if found, delete $WORK_DIR/linux-kernel to start from scratch

# amount by which to resize the the VM's Raspberry Pi OS disk image
# (requires manually running `raspi-config nonint do_expand_rootfs` to apply)
RPOS_IMAGE_EXTRA_SPACE=4G

# location of the VM's Raspberry Pi OS disk image
RPOS_IMAGE=$WORK_DIR/raspi_vm_disk.img


# Setup mounting directories to edit the Raspberry Pi OS image
RPI_BOOT_MNT=/mnt/rpi_boot
RPI_ROOT_MNT=/mnt/rpi_root
if ! [ -d $RPI_BOOT_MNT ];then
  sudo mkdir $RPI_BOOT_MNT
fi
if ! [ -d $RPI_ROOT_MNT ]; then
  sudo mkdir $RPI_ROOT_MNT
fi

if ! [ -e $WORK_DIR ];then
  mkdir -p $WORK_DIR
fi
cd $WORK_DIR || exit 1

## Setup disk image for VM with Raspberry Pi OS pre-installed
RECREATED_RPOS_IMAGE=false
if $RECREATE_RPOS_IMAGE || ! [ -e $RPOS_IMAGE ];then
  RECREATED_RPOS_IMAGE=true
  
  # delete any old images
  if [ -e $RPOS_IMAGE ];then
    sudo rm $RPOS_IMAGE
  fi
  
  
  ## Download and resize Raspberry Pi OS image
  wget $RPOS_IMAGE_URL -O $RPOS_IMAGE.xz
  xz -d $RPOS_IMAGE.xz
  qemu-img resize -f raw $RPOS_IMAGE +$RPOS_IMAGE_EXTRA_SPACE
fi

# ensure we can work with the image now
# we'll change its permissions again before running the VM
sudo chown $USER:kvm $RPOS_IMAGE


# calculate image partition details for the boot and root partitions in the Raspberry Pi OS image
sector_size=$(fdisk -l $RPOS_IMAGE | grep 'Sector size' | awk '{print $4}')
start_sector_boot=$(fdisk -l $RPOS_IMAGE | grep -m1 "^${RPOS_IMAGE}1" | awk '{print $2}')
start_sector_root=$(fdisk -l $RPOS_IMAGE | grep -m1 "^${RPOS_IMAGE}2" | awk '{print $2}')
start_sector_bytes_boot=$((sector_size * start_sector_boot))
start_sector_bytes_root=$((sector_size * start_sector_root))


if $RECREATED_RPOS_IMAGE;then
  ## Preconfigure username & password, enable SSH
  # mount the first partition of the Raspberry Pi OS image (the boot partition)
  sudo mount -o loop,offset=$start_sector_bytes_boot $RPOS_IMAGE $RPI_BOOT_MNT
  echo "${USERNAME}:${PASSWORD_HASH}"| sudo tee $RPI_BOOT_MNT/userconf
  sudo touch $RPI_BOOT_MNT/ssh # enable SSH
  sudo umount $RPI_BOOT_MNT
fi



## Build Linux Kernel
KERNEL_IMAGE=$WORK_DIR/linux-kernel/arch/arm64/boot/Image

REBUILT_KERNEL=false
if $REBUILD_KERNEL || $RECREATED_RPOS_IMAGE || ! [ -e $KERNEL_IMAGE ];then
  REBUILT_KERNEL=true
  # download linux kernel if necessary
  if ! [ -e $KERNEL_IMAGE ];then
    sudo rm -r linux-* # remove old files
    wget $KERNEL_URL -O linux-kernel.tar.xz
    tar xvJf linux-kernel.tar.xz
    rm linux-kernel.tar.xz
    mv linux-* linux-kernel # rename linux-VERSION to known name
  fi
  
  cd linux-kernel || exit 1
  # create a .config file
  ARCH=arm64 CROSS_COMPILE=/bin/aarch64-linux-gnu- make defconfig
  
  # Use the kvm_guest config as the base defconfig, which is suitable for qemu
  ARCH=arm64 CROSS_COMPILE=/bin/aarch64-linux-gnu- make kvm_guest.config
  
  ## manual kernel configuration DON'T DO THIS ON A MACHINE OF DIFFERENT ARCHITECTURE
  # make menu config
  sed -i 's/# CONFIG_DRM_QXL is not set/CONFIG_DRM_QXL=m/' .config
  sed -i 's/# CONFIG_FB_SIMPLE is not set/CONFIG_FB_SIMPLE=y/' .config
  sed -i 's/# CONFIG_FB_VIRTUAL is not set/CONFIG_FB_VIRTUAL=y/' .config

  # Build the kernel
  ARCH=arm64 CROSS_COMPILE=/bin/aarch64-linux-gnu- make -j8

  ## Install kernel modules into the guest filesystem
  # mount the second partition of the Raspberry Pi OS image (the root partition)
  sudo mount -o loop,offset=$start_sector_bytes_root $RPOS_IMAGE $RPI_ROOT_MNT
  # Install kernel modules into the guest filesystem
  ARCH=arm64 CROSS_COMPILE=/bin/aarch64-linux-gnu- sudo make modules_install INSTALL_MOD_PATH=$RPI_ROOT_MNT
  sudo umount $RPI_ROOT_MNT
  
  cd .. || exit 1 # return to parent directory
fi


sudo chown libvirt-qemu:kvm $RPOS_IMAGE



## Run unregistered VM directly using qemu (CLI only)
# qemu-system-aarch64 -machine virt -cpu cortex-a72 -smp 6 -m 4G \
#     -kernel $KERNEL_IMAGE -append "root=/dev/vda2 rootfstype=ext4 rw panic=0 console=ttyAMA0" \
#     -drive format=raw,file=$RPOS_IMAGE,if=none,id=hd0,cache=writeback \
#     -device virtio-blk,drive=hd0,bootindex=0 \
#     -netdev user,id=mynet,hostfwd=tcp::2222-:22 \
#     -device virtio-net-pci,netdev=mynet \
#     -monitor telnet:127.0.0.1:5555,server,nowait

## Create registered VM for virt-manager (with graphics!)
echo "
<domain type='qemu'>
  <name>$VM_NAME</name>
  <memory unit='GiB'>4</memory>
  <vcpu placement='static'>6</vcpu>
  <os>
    <type arch='aarch64' machine='virt'>hvm</type>
    <kernel>$KERNEL_IMAGE</kernel>
    <cmdline>root=/dev/vda2 rootfstype=ext4 rw panic=0 console=ttyAMA0</cmdline>
  </os>
  <cpu mode='custom' match='exact'>
    <model>cortex-a72</model>
  </cpu>
  <devices>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='$RPOS_IMAGE'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <interface type='network'>
      <source network='default'/>
      <model type='virtio'/>
    </interface>
    <filesystem type='mount' accessmode='mapped'>
      <source dir='$QBM_SOURCE'/>
      <target dir='$SHARED_FS_TAG'/>
    </filesystem>
    <serial type='pty'>
      <source path='/dev/pts/4'/>
      <target type='system-serial' port='0'>
        <model name='pl011'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </controller>
    <input type='mouse' bus='usb'/>
    <input type='keyboard' bus='usb'/>
    <graphics type='spice' autoport='yes'>
      <listen type='address'/>
      <image compression='off'/>
      <gl enable='no'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='virtio' heads='1' primary='yes'/>
    </video>
  </devices>
</domain>

" > raspi_vm.config
virsh define raspi_vm.config
virsh start "$VM_NAME"



BLACK='\033[0;30m'
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
MAGENTA='\033[0;35m'
CYAN='\033[0;36m'
WHITE='\033[0;37m'
NC='\033[0m' # No Color
echo -e "$YELLOW
It is normal for the VM window to display while booting:
'Guest has not initialized the display (yet).'

Wait for a minute, after which the console log-in should display.
In the virt-manager VM window, under the menu _View > Consoles > Serial 1_ you can switch to the text console while the graphics aren't running yet.

$BLUE
To make your VM's root filesystem take up the full disk space, run:
$GREEN
sudo raspi-config nonint do_expand_rootfs
$NC"

Just curious if you have any insight to fix this issue?

./create_raspi_vm.sh error: Failed to define domain from raspi_vm.config error: missing target information for device

error: failed to get domain 'RasPi'

That error message comes from the virsh VM-management tool, but the root cause could be any previous part of the script failing, so look for error messages in the entire script's output and try to solve any errors you find.

And of course, make sure you've installed all the prerequisites.
I've updated the prerequisites list after finding some dependencies that were missing.