How to install Arch Linux ARM or Debian ARM in QEMU full system emulator

README.md

How to install a GNU/Linux ARM emulator

Prerequisites:

• A PC with a Linux distribution - Arch Linux amd64 used here

Dependencies (for Arch Linux amd64 but very easy to get on most distros):

• edk2-armvirt (or a prebuilt firmware binary from the Unofficial EDK2 nightly build project)
• qemu-arch-extra (provides the command qemu-system-aarch64)
• wget (or any downloader)
• gvim (or any editor)
• openssh (or QEMU console)

1. Download the Arch Linux ARM generic tarball and create an image, replacing 60G with your desired maximum size.

wget http://os.archlinuxarm.org/os/ArchLinuxARM-aarch64-latest.tar.gz
qemu-img create -f qcow2 arch-aarch64.qcow2 60G

Tip

Step 1 equivalent for Debian guest:

wget https://cdimage.debian.org/cdimage/release/current/arm64/iso-dvd/debian-12.7.0-arm64-DVD-1.iso
wget https://cdimage.debian.org/cdimage/release/current/arm64/iso-dvd/SHA256SUMS
shasum -a 256 --ignore-missing -c SHA256SUMS
qemu-img create -f qcow2 debian-aarch64.qcow2 60G

Users of Debian guest should now skip to step 7.

2. Become root, connect the image to nbd and partition it with fdisk.

sudo modprobe nbd
sudo qemu-nbd --connect=/dev/nbd0 arch-aarch64.qcow2
sudo fdisk /dev/nbd0

• then g (to create a new GPT partition table)
• then n (to create a new partition), then Enter twice, then +400M and Enter
• then t (to change the type), then 1 for EFI System Partition
• then n and Enter three times, then w to write changes and exit

3. Format the partitions of the image, mount them, and extract the Arch Linux ARM tarball to them.

sudo mkfs.vfat /dev/nbd0p1
sudo mkfs.ext4 /dev/nbd0p2
sudo mkdir rootfs
sudo mount /dev/nbd0p2 rootfs
sudo mkdir rootfs/boot
sudo mount /dev/nbd0p1 rootfs/boot 
sudo bsdtar -xpf ArchLinuxARM-aarch64-latest.tar.gz -C rootfs

4. Edit fstab.

• You will need both partitions' UUIDs - the UUID of the vfat partition in /dev/nbd0p1 looks like UUID="XXXX-XXXX" and the UUID of the ext4 partition in /dev/nbd0p2 looks like UUID="XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX":

sudo blkid

• Then, edit rootfs/etc/fstab:

sudo vim rootfs/etc/fstab

• Paste the following, replacing each instance of X with the corresponding digit of the UUID of the corresponding partition, /dev/nbd0p1 and /dev/nbd0p2 respectively, then save the file:

/dev/disk/by-uuid/XXXX-XXXX                            /boot vfat defaults 0 0
/dev/disk/by-uuid/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX /     ext4 defaults 0 0

5. Create startup.nsh, which is read by the UEFI firmware to initially boot.

• Edit rootfs/boot/startup.nsh:

sudo vim rootfs/boot/startup.nsh

• Paste the following, replacing each instance of X with the corresponding digit of the UUID of the /dev/nbd0p2 partition, then save the file:

Image root=UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX rw initrd=\initramfs-linux.img

6. Unmount the partitions, sync, disconnect the image from nbd, and exit the root shell.

sudo umount -R rootfs
sudo sync
sudo qemu-nbd --disconnect /dev/nbd0
sudo rmmod nbd

7. Create flash images for the UEFI firmware and variables

Note

if you downloaded the RELEASEAARCH64_QEMU_EFI.fd instead of using the one from your distro's package, use that here in place of the QEMU_CODE.fd. AARCH64_QEMU_EFI.fds (which are installed into "flash0.img" here) can very slowly become slightly outdated over time. After using one with an Arch Linux ARM emulator for several years, then installing a new emulator from scratch and comparing its behavior with the old one, I've noticed very slight, subtle differences in behavior between them in the pre-boot stage before guest OS code runs. I would say the newer one's behavior seems slightly more polished and desirable.

truncate -s 64M flash0.img
truncate -s 64M flash1.img
dd if=/usr/share/edk2-armvirt/aarch64/QEMU_CODE.fd of=flash0.img conv=notrunc

8. Launch QEMU, removing or adding anything you see fit.

qemu-system-aarch64 -M virt -m 8192 -cpu cortex-a72 -smp 8 \
      -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash0.img \
      -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash1.img \
      -drive if=none,file=arch-aarch64.qcow2,format=qcow2,id=hd0 \
      -device virtio-scsi-pci,id=scsi0 \
      -device scsi-hd,bus=scsi0.0,drive=hd0,bootindex=1 \
      -nic user,model=virtio-net-pci,hostfwd=tcp::2222-:22 \
      -monitor none -display none -vga none

Tip

Step 8 equivalent for Debian guest:

qemu-system-aarch64 -M virt -m 8192 -cpu cortex-a72 -smp 8 \
     -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash0.img \
     -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash1.img \
     -drive if=none,file=debian-aarch64.qcow2,format=qcow2,id=hd0 \
     -cdrom debian-12.7.0-arm64-DVD-1.iso \
     -device virtio-scsi-pci,id=scsi0 \
     -device scsi-hd,bus=scsi0.0,drive=hd0,bootindex=2 \
     -nic user,model=virtio-net-pci,hostfwd=tcp::2222-:22 \
     -nographic

9. Upon successful first boot, initialize Arch Linux ARM and install a new bootloader.

• Log in as alarm, password alarm:

ssh -p 2222 alarm@localhost

• Become root, password root:

su

• Initialize the pacman keyring, update the system and install efibootmgr, replacing each instance of X with the corresponding digit of the UUID of the /dev/nbd0p2 partition from earlier (which is now /dev/sda2), then shut down:

pacman-key --init
pacman-key --populate archlinuxarm
pacman -Syu
pacman -S efibootmgr
efibootmgr --disk /dev/sda --part 1 --create --label "Arch Linux ARM" --loader /Image --verbose \
           --unicode 'root=UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX rw initrd=\initramfs-linux.img'
poweroff

Tip

Step 9 equivalent for Debian guest:

• At the EDK II Shell, type this command

FS1:\efi\boot\grubaa64.efi

Users of Debian guest can choose "Install" and use the guided installer.

10. Launch QEMU again, exactly as in step 8.

qemu-system-aarch64 -M virt -m 8192 -cpu cortex-a72 -smp 8 \
      -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash0.img \
      -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash1.img \
      -drive if=none,file=arch-aarch64.qcow2,format=qcow2,id=hd0 \
      -device virtio-scsi-pci,id=scsi0 \
      -device scsi-hd,bus=scsi0.0,drive=hd0,bootindex=1 \
      -nic user,model=virtio-net-pci,hostfwd=tcp::2222-:22 \
      -monitor none -display none -vga none

Tip

Step 10 equivalent for Debian guest:

qemu-system-aarch64 -M virt -m 8192 -cpu cortex-a72 -smp 8 \
     -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash0.img \
     -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash1.img \
     -drive if=none,file=debian-aarch64.qcow2,format=qcow2,id=hd0 \
     -device virtio-scsi-pci,id=scsi0 \
     -device scsi-hd,bus=scsi0.0,drive=hd0,bootindex=2 \
     -nic user,model=virtio-net-pci,hostfwd=tcp::2222-:22 \
     -nographic

11. Proceed with configuring Arch Linux ARM as normal (time, locales, users, software, configuration), using the Arch Linux Wiki as a guide.

Note

If your host has a simple network configuration, you can replace the slower -nic user argument with an efficient -netdev tap argumnet if you want using my minimal tap0 guide.

@robertkirkman No worries, the guide was clear and really useful, and I was gratefully surprised that everything booted first try. Having the manually resize the ESP was a very minor thing compared to the value I got from it.

I'm glad to hear that!

I'm not sure whether you'd need this at all, but this QEMU has some very useful OpenGL acceleration capabilities that I never really explained here before either, and at least for me, there is a quirk with it I can explain that causes it to seem stuck in a permanent multiseat mode. That is to say, It might seem like you can't interact your keyboard with the QEMU window itself, and have to use ssh. But the real truth is that you can if you add certain types of QEMU arguments - and for me, it's only worked with a second keyboard/mouse plugged into the host!

Here are my current QEMU arguments for an Emulator with virgl OpenGL passthrough to real GPU and multiseat input with a USB mouse and keyboard.

#!/bin/bash
qemu-system-aarch64 -M virt \
                    -m 16384 \
                    -cpu cortex-a72 \
                    -smp 32 \
                    -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash0.img \
                    -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash1.img \
                    -drive if=none,file=arch-aarch64.qcow2,format=qcow2,id=hd0 \
                    -device virtio-scsi-pci,id=scsi0 \
                    -device scsi-hd,bus=scsi0.0,drive=hd0,bootindex=1 \
                    -nic user,model=virtio-net-pci,hostfwd=tcp::2222-:22,hostfwd=tcp::5555-:5555 \
                    -device virtio-gpu-gl-pci,xres=1280,yres=1024 \
                    -display sdl,gl=on \
                    -device qemu-xhci,id=xhci \
                    -device usb-host,bus=xhci.0,hostdevice=/dev/bus/usb/003/009 \
                    -device usb-host,bus=xhci.0,hostdevice=/dev/bus/usb/003/010

When I change the USB port that the keyboard or mouse is plugged into, or sometimes update the host OS which can also change how those numbers are assigned, I use the lsusb -tvv command on the host to find their new device file paths, by finding the paths associated with the names of the USB devices that match their manufacturers.

First, you would make sure the virglrenderer package is installed in your host, then the -device virtio-gpu-gl-pci argument combined with the -display sdl,gl=on argument should allow the guest to spawn a DRM TTY within the QEMU SDL window which you should then be able to launch Xorg on with OpenGL acceleration using the virgl Mesa driver.

Unfortunately, it seems like there is a hardware-cursor-rendering-related issue with QEMU and Xorg operating together in this mode. To work around it, I use andykitchen/swcursor in the guest, OR the Xorg builtin software cursor in the guest:

/etc/X11/xorg/conf.d/20-swcursor.conf

Section "Device"
	Identifier "card0"
	Driver "modesetting"
	Option "SWcursor" "on"
EndSection

Of course, this is all very useless for you if you only need the ssh connection and no GUI, but I always wanted to add it as an update here since I feel that it is a very underrated configuration that I've never seen anyone else actually using.

Nice! I don't usually need high-performance graphics on VMs, but I remember trying to get OpenGL acceleration to work about a year or two ago but didn't manage to get it to work due to my old NVIDIA GPU (oof). Nowadays I also have an AMD APU around, so I will keep that in mind if I need it again :)

I have successfully installed an app that I needed within this for a long time but only yesterday finally got to work within this QEMU, redroid, so I am documenting the steps here. I haven't got GPU acceleration working in this so its graphics are a lot slower than the Arch Linux ARM emulator itself, but it works with software rendering.

Prerequisites

• Everything I posted here prior to this point, including the hostfwd=tcp::5555-:5555 setting to forward the adb port through QEMU for scrcpy

Build necessary kernel

sudo pacman -S base-devel git
git clone https://github.com/archlinuxarm/PKGBUILDs.git
cd PKGBUILDs/core/linux-aarch64/
git checkout 593aef97b9a94a0d0a79bb996cb5dd28ad039fe5
git apply -v << 'EOF'
--- a/core/linux-aarch64/PKGBUILD
+++ b/core/linux-aarch64/PKGBUILD
@@ -28,7 +28,7 @@ md5sums=('b9828ed78dae306e3d90643cd5cdb8f1'
          'f65db321ef86dcccf3d27d26023c6438'
          '7b08a199a97e3e2288e5c03d8e8ded2d'
          'c9d4e392555b77034e24e9f87c5ff0b3'
-         '22283f60f34dddb1d4b6748482b6c86c'
+         'SKIP'
          '7c97cf141750ad810235b1ad06eb9f75'
          '61c5ff73c136ed07a7aadbf58db3d96a'
          '584777ae88bce2c5659960151b64c7d8'
--- a/core/linux-aarch64/config
+++ b/core/linux-aarch64/config
@@ -8701,11 +8701,13 @@ CONFIG_DMA_ENGINE_RAID=y
 #
 CONFIG_SYNC_FILE=y
 # CONFIG_SW_SYNC is not set
-# CONFIG_UDMABUF is not set
+CONFIG_UDMABUF=y
 # CONFIG_DMABUF_MOVE_NOTIFY is not set
 # CONFIG_DMABUF_DEBUG is not set
 # CONFIG_DMABUF_SELFTESTS is not set
-# CONFIG_DMABUF_HEAPS is not set
+CONFIG_DMABUF_HEAPS=y
+CONFIG_DMABUF_HEAPS_SYSTEM=y
+CONFIG_DMABUF_HEAPS_CMA=y
 # CONFIG_DMABUF_SYSFS_STATS is not set
 # end of DMABUF options
 
@@ -10565,7 +10567,10 @@ CONFIG_RAS=y
 #
 # Android
 #
-# CONFIG_ANDROID_BINDER_IPC is not set
+CONFIG_ANDROID_BINDER_IPC=y
+CONFIG_ANDROID_BINDERFS=y
+CONFIG_ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder"
+CONFIG_ANDROID_BINDER_IPC_SELFTEST=y
 # end of Android
 
 CONFIG_LIBNVDIMM=y
EOF
# Emulator can compile the kernel for itself quickly and not too slow!
echo 'MAKEFLAGS="-j32"' | sudo tee -a /etc/makepkg.conf
makepkg -s
sudo pacman -U linux-aarch64-6.9.6-1-aarch64.pkg.tar.xz linux-aarch64-headers-6.9.6-1-aarch64.pkg.tar.xz
sudo reboot

Install Docker

sudo pacman -S docker
sudo systemctl enable --now docker

Install and run redroid

sudo docker run -itd --privileged -v ~/redroid-data:/data -p 5555:5555 -p 8022:8022 --name redroid redroid/redroid:14.0.0_64only-latest

Important

There is one step during redroid's boot that sometimes takes an extremely long time within the emulator. The longest I had to wait was 7 hours and then it eventually booted fully. You can use dmesg -w within QEMU to monitor redroid's boot progress (and compare it with the kernel log of a native redroid instance outside the emulator to see what point you're at)

Install scrcpy on the host amd64 OS and connect to the emulated redroid

Note

These commands can be run outside the emulator

sudo pacman -S scrcpy
adb connect localhost
scrcpy -s localhost:5555