Boot LXC ready-to-go arm64 images in QEMU on amd64

README.md

⚠️ Quickly put together in April 2023, you're reading a gist, not a community wiki. It's fundamentally hacky, heavily summarized, and might be out-of-date, make sure you're familiar with the concepts before attempting this!

So, for some reason, the arm64 images used in lxc don't run on amd64 qemu by default.

The images are hosted here: https://us.lxd.images.canonical.com/images/
We will go for an ubuntu jammy arm64 image: https://us.lxd.images.canonical.com/images/ubuntu/jammy/arm64/default/20230413_07:43/disk.qcow2
There are plenty to choose from, just make sure to pick the qcow2 file
If you have lxc you can also run: lxc image export images:ubuntu/jammy/arm64 --vm images_ubuntu_arm64

It's a good thing to make a backup of this file as we're going to change it right away.

Using the qemu tools, mount the image to be able to modify the GRUB config:

modprobe nbd max_part=8
qemu-nbd -c /dev/nbd0 disk.qcow2
fdisk /dev/nbd0 -l
blkid /dev/nbd0*

blkid will show 2 partitions, copy the uuid for the "rootfs" partition and past it somewhere. For distributions other than ubuntu and debian, it might be a little different but don't get confused: do not modify grub.cfg in EFI/BOOT as afaik it is skipped.
Now we mount the partition to be able to edit it:

mount /dev/nbd0p1 /mnt/somepoint/
# for ubuntu
vim /mnt/somepoint/EFI/ubuntu/grub.cfg
# for debian
vim /mnt/somepoint/EFI/debian/grub.cfg

The file contains some wrong bits, I haven't investigated a solution, but now that we have the uuid, we can just write the boot commands:
Replace the content with the following:

set root=(hd0,gpt2)
linux /boot/<vmlinuz filename> root=UUID=<uuid> init=/bin/bash
initrd /boot/<initrd filename>
boot

Note the init=/bin/bash above, this is temporary and will allow us to change the root password on first boot.

Unmount: umount /dev/nbd0p1 and qemu-nbd -d /dev/nbd0p1

To boot this image, I am using this relatively basic qemu command:

qemu-system-aarch64 -m 4096 -smp 6 -cpu cortex-a76 -M virt -nographic -boot menu=on \
  -drive format=qcow2,file=disk.qcow2 -bios /usr/share/AAVMF/AAVMF_CODE.fd

The bios is important. The path above is for Debian/Ubuntu (part of package qemu-efi-aarch64) and might be different on other distros.

This should theoretically drop you in a root shell.
If not, connect the image with qemu-nbd and mount the first partition again, comment everything in EFI/debian/grub.cfg and add console on a newline at the end. Run again and you should end up in grub shell and be able to look around. Some help here for finding the right path. Note that you can run the commands given above for grub.cfg manually to help you find out where it failed. Good luck!

Go ahead and do some stuff:

mount -o rw,remount /
passwd
passwd ubuntu
usermod -aG sudo ubuntu

Set passwords and groups etc.

Now is a good time to remove the init:

mount /dev/vda1 /somemountpoint
vim /somemountpoint/EFI/debian/grub.cfg

Check using mount first to doublecheck it's the right path, eg. /dev/vda2 should be mounted on /.
Just remove the init= parameter.
At this point I don't think there is a clean way to shutdown. You can either ctrl+A X or ctrl+D.

Rerun the command and you should see the familiar ubuntu/debian tty, try your new passwords!

As the qemu command is very basic, you won't have any virtual device connected. I advise to now run our working arm64 vm in virt-manager.
Here are the steps in virt-manager:

• Create a new vm
• Select the option to use an en existing disk
• Select aarch64 architecture
• Select the disk used in the command above
• If you can, I suggest adding some memory and virtual cpus
• ⚠️ Before finishing select the option to personalize the vm before starting:
  • In the firmware section, select "AAVMF" for aarch64 architecture
  • (Optional) In the processor tab, select cortex-a76, I have no idea if that actually improves performance