Run Win11 on ARM in QEMU

README.md

Guide updated to use the official installation .iso from Microsoft, which finally became available in Dec. 2024! For the VHDX disk image based guide check an earlier version of this gist.

When following this guide on a host not capable of aarch64 virtualization, replace -cpu host -accel kvm with -cpu max.

1. Download the installation .iso from https://www.microsoft.com/en-us/software-download/windows11arm64. Win11_24H2_EnglishInternational_Arm64.iso is known to work.
2. Download the the latest virtio-win.iso from https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/latest-virtio/.
3. Create a disk image for the Windows installation: qemu-img create -f qcow2 win11.qcow2 25G
4. Start the installation with generic (ramfb for display, usb-storage for .isos) as well as virtio (virtio-gpu-pci, virtio-block) devices: qemu-system-aarch64 -M virt -cpu host -accel kvm -m 4G -smp 4 -device ramfb -device virtio-gpu-pci -bios /usr/share/qemu/qemu-uefi-aarch64.bin -device qemu-xhci -device usb-kbd -device usb-tablet -drive file=Win11_24H2_EnglishInternational_Arm64.iso,media=cdrom,if=none,id=inst -device usb-storage,drive=inst -drive file=virtio-win.iso,media=cdrom,if=none,id=iso -device usb-storage,drive=iso -drive file=win11.qcow2,if=virtio -nic user,model=virtio-net-pci,mac=2A:50:A7:4E:D9:C4. At least since build 25931, it's necessary to provide a network connection to finish the initial setup. The default QEMU MAC address will result in a false UUID match with some AutoPilot enrolled devices, forcing an "Intellek" domain login. To avoid that, use a different MAC address, e.g. one from https://duckduckgo.com/?q=random+mac+addr&ia=answer. The one from the QEMU man page is meanwhile bound to NVIDIA (argh!). To have a higher resolution with ramfb, enter the OVMF setup by pressing F10 during boot, change the preferred resolution to 1024x768 and reset.
5. In the setup dialog appears, proceed until the keyboard layout is configured but not further.
6. Press Shift-F10 to open cmd.exe and run regedit. Navigate to HKEY_LOCAL_MACHINE/SYSTEM/Setup, create a key (aka. folder) called LabConfig and inside that two DWORDs BypassTPMCheck and BypassSecureBoot and set both of them to 1. Alternatively you can also use a SB-enabled OVMF as bios and provide a TPM, but I haven't tried that (and IMO especially the SB requirement is absolute bullshit anyway).
7. Proceed to the drive selection. Be aware that if you read the license agreement, you'll probably want to throw your computer out of the window.
8. Load the virtio-blk driver by going to the E:\viostor\w11\ARM64 folder and selecting the driver that appears in the dialog (don't be confused: the driver title includes SCSI, but it's for virtio-blk).
9. Continue with the installation until the OOBE welcome wizard. It'll take ages (>30min here!) at "Installing 42%" for some reason.
10. Complete the setup until it asks for network. Use the "Install driver" button to install the virtio-gpu driver first (!) by navigating to E:\viogpudo\w11\ARM64 and letting it install the driver. After that completes, use the button and navigate to E:\NetKVM\w11\ARM64 and let it install the driver, followed by a reboot. You can shut down the VM at that point.
11. Now the generic devices can be omitted: qemu-system-aarch64 -M virt -cpu host -accel kvm -m 4G -smp 4 -device virtio-gpu-pci -bios /usr/share/qemu/qemu-uefi-aarch64.bin -device qemu-xhci -device usb-kbd -device usb-tablet -drive file=win11.qcow2,if=virtio -nic user,model=virtio-net-pci,mac=2A:50:A7:4E:D9:C4. The first boot will take a while with a disabled display, be patient (if it does not work, add -device ramfb and use it for troubleshooting),
12. Complete the setup as usual. To avoid using a Microsoft account, use Shift-F10 and run oobe\bypassnro.

openQA setup.md

openQA expects a resolution of 1024x768, so change the display resolution in settings. With some older virtio driver builds this was only possible to change using the registry, but at least with 0.1.266 the display configuration in settings works!

To avoid having to add new needles, switch the desktop wallpaper to the one used by the existing win11 tests and change the terminal in the developer settings to "Windows Console Host". Also set a fixed background picture for the lock and sign in screens.

Windows on Arm has a different model for serial ports, the usual methods to access them no longer work. However, FTDI ported the old serial port bus to Arm and it works with the usb-serial device! Download the VCP driver from https://ftdichip.com/drivers/vcp-drivers/ and extract it somewhere, then right click on the FTDIBUS.inf file and select "Install" and repeat the same for FTDIPORT.inf. In the device manager, open the properties for the (only) COM port and assign COM1 in the advanced settings. To use it with openQA, pass -chardev ringbuf,id=usbserial,logfile=serial0,logappend=on -device usb-serial,chardev=usbserial. This uses a bit of a hack as it writes into the same file as the default serial port (pl011, not usable as COM port), but there's no other way as QEMU does not allow using a chardev from multiple devices. Native support for usb-serial in os-autoinst would be the right way. usb-serial also conflicts with -no-migratable, so QEMU_DISABLE_SNAPSHOTS=1 needs to be passed.

Not specific to Arm: To optimize the qcow2 size, start the VM with discard=unmap on the qcow2 block dev. In a powershell, run Optimize-Volume -Verbose -DriveLetter C -ReTrim and shut down. Run qemu-img convert -pc -O qcow2 for_cleanup.qcow2 Win11ArmBuild25931.qcow2 to discard the unmapped space and perform recompression.

Based on a win11 x86_64 (UEFI) test:

openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 3546891 _GROUP=38 ARCH=aarch64 HDD_1=win11_22598.qcow2 QEMUMACHINE=virt QEMUVGA= QEMU_VIDEO_DEVICE=virtio-gpu-pci UEFI_PFLASH_CODE=/usr/share/qemu/aavmf-aarch64-code.bin UEFI_PFLASH_VARS=/usr/share/qemu/aavmf-aarch64-vars.bin WORKER_CLASS=qemu_aarch64 QEMU_DISABLE_SNAPSHOTS=1 "QEMU_APPEND=chardev ringbuf,id=usbserial,logfile=serial0,logappend=on -device usb-serial,chardev=usbserial"

WSL2.md

It's also possible to run WSL2 inside the Windows VM. However, that uses Hyper-V and thus requires hardware virtualization. Nested virtualization is not widely supported by hardware yet and is also not implemented in Linux or QEMU, so software emulation is needed.

Just -M virt,virtualization=on -cpu max will not work though: Windows no longer boots if the hypervisor is enabled. With WinDbg attached it complains about unexpected register values which indicate that it expects the CPU to have EL3.

Using https://github.com/ARM-software/arm-trusted-firmware in EL3 (following https://trustedfirmware-a.readthedocs.io/en/latest/plat/qemu.html) does not work though, it leads to Windows failing to initialize interrupts. Presumably some of the ATF initialization confuses the Windows kernel.

There's a way around ATF though: QEMU's -kernel option allows to have a CPU with EL3 but boot directly into EL2. To be able to use OVMF with that option (instead of -bios), it has to be built as Linux kernel image. A prebuilt QEMU_EFI.fd is available at https://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/5312/QEMU-KERNEL-AARCH64/RELEASE_GCC/QEMU_EFI.fd.

For QEMU < 8.2.0 you either need to use -cpu neoverse-n1 or apply https://lists.nongnu.org/archive/html/qemu-devel/2023-09/msg03039.html, to avoid that accesses to HDFGRTR_EL2 trap to EL3 which does not exist -> stuck.

diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index 720f22531a..24fa169060 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -761,6 +761,10 @@ static void do_cpu_reset(void *opaque)
                     if (cpu_isar_feature(aa64_hcx, cpu)) {
                         env->cp15.scr_el3 |= SCR_HXEN;
                     }
+                    if (cpu_isar_feature(aa64_fgt, cpu)) {
+                        env->cp15.scr_el3 |= SCR_FGTEN;
+                    }
+
                     /* AArch64 kernels never boot in secure mode */
                     assert(!info->secure_boot);
                     /* This hook is only supported for AArch32 currently:

QEMU < 8.1 is missing also SCR_HXEN and probably others, so make sure to use at least 8.1.0.

Full QEMU command with WSL2/Hyper-V support (tested with Build 25931):

qemu-system-aarch64 -M virt,virtualization=on,gic-version=3,secure=on -cpu neoverse-n1 -m 4G -smp 4 -device virtio-gpu-pci -kernel QEMU_EFI.fd -device qemu-xhci -device usb-kbd -device usb-tablet -drive file =Win11ArmBuild25931.qcow2,if=none,id=boot -device virtio-scsi -device scsi-hd,drive=boot,serial=hd0 -nic user,model=virtio-net-pci,mac=52:54:98:76:54:32 -vnc 0