Investigating what OSX-KVM
's pre-packaged SSDT files contained, I ran into an
issue with the default USB controllers' PCI addresses.
The USB controllers I was passing to QEMU did not match the
default VM definition & SSDT.
To more easily see the XNU kernel debugging output:
- I added a serial console device to the VM via this
libvirt
domain XML snippet:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<!-- ... Original VM definition ... -->
<devices>
<!-- ... other devices ... -->
<serial type='pty'>
<target type='isa-serial' port='0'>
<model name='isa-serial'/>
</target>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<!-- ... rest of VM definition ... -->
</devices>
</domain>
- I added the following XNU
boot-args
:debug=0x10A -v serial=3 msgbuf=1048576 serialbaud=115200
-v
enables verbose bootingserial=3
- Sends debug output over serialmsgbuf=10485760
- Sets kern.msgbuf to10485760
bytes (10 MiB
) to allow a much larger kernel-log.serialbaud=115200
- Sets the serial TTY's baud rate (e.g. to connect withscreen /dev/pts/NN 115200
)- The
debug=0x10A
bit mask enablesDB_LOG_PI_SCRN
,DB_SLOG
, andDB_KPRT
which respectively:- Disables graphical panic screen
- Sends some diagnostics to syslog
- Sends kernel's debugging
kprintf
to the serial console port - Note: A bitmask can be calculated by a logical OR operation on those values. For example:
0x100 | 0x2 | 0x8 = 0x10A
- To add this, I had to:
- First mount the
OpenCore.qcow2
disk withqemu-nbd
qemu-nbd
needsnbd
kernel module loaded:modprobe nbd max_part=8
- Then mount the inner
vfat
EFI partition (containing OpenCore bootloader,kext
s and ACPI SSDTs) - Edit the
config.plist
file inside - Unmount the EFI partition & disconnect the
qcow2
file from/dev/nbd0
- First mount the
- After changing
boot-args
, the kernel consolekprintf
debug output can be viewed in:- The
virt-manager
window under:View -> Consoles -> Serial 1
- By directly connecting to the
/dev/pts/NN
device with GNU screen:screen /dev/pts/5
. - Note: The device number can change, so pay attention that you're connecting to the right one.
This can be seen in
virt-manager
's "Details
" screen by checking theSerial 1
device under "Source path:
"
- The
$ sudo qemu-nbd --discard=unmap --connect=/dev/nbd0 ~/src/pub/OSX-KVM/OpenCore/OpenCore.qcow2
$ sudo sgdisk --print /dev/nbd0
Disk /dev/nbd0: 786432 sectors, 384.0 MiB
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 3D196FDE-9679-43B2-B0EA-52513C02D436
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 786398
Partitions will be aligned on 32-sector boundaries
Total free space is 6075 sectors (3.0 MiB)
Number Start (sector) End (sector) Size Code Name
1 2048 300000 145.5 MiB EF00 primary
2 302048 784384 235.5 MiB 8300 primary
$ sudo blkid /dev/nbd0p1
/dev/nbd0p1: SEC_TYPE="msdos" LABEL_FATBOOT="EFI" LABEL="EFI" UUID="BB7E-D63C" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="primary" PARTUUID="d79d1e93-e748-4fef-8de6-fd4e37e3ec5b"
$ mkdir /tmp/oc
$ sudo mount -t vfat /dev/nbd0p1 /tmp/oc
$ vim /tmp/oc/EFI/OC/config.plist
## Search for: boot-args with: /boot-args
## Add the kernel debug boot args:
## debug=0x10A -v serial=3 msgbuf=10485760 serialbaud=115200 keepsyms=1 amfi_get_out_of_my_way=1 tlbto_us=0 vti=9
$ sudo umount /tmp/oc
$ sudo qemu-nbd --disconnect /dev/nbd0
/dev/nbd0 disconnected
For reference, the original boot-args
defined as default by OSX-KVM
were:
<key>boot-args</key>
<string>-v keepsyms=1 amfi_get_out_of_my_way=1 tlbto_us=0 vti=9</string>
After editing, I ended up with the following boot-args
under the <key>NVRAM</key><dict><key>Add</key>... <key>boot-args</key> ...
section in config.plist
:
<key>boot-args</key>
<string>debug=0x10A -v serial=3 msgbuf=10485760 serialbaud=115200 keepsyms=1 amfi_get_out_of_my_way=1 tlbto_us=0 vti=9</string>
Note: For more details on XNU kernel debugging, see here
After booting the kernel, I saw some ACPI table errors in the console log:
ACPI Exception: AE_NOT_FOUND, (SSDT: QEMUUSB) while loading table (20160930/tbxfload-319)
So, for some reason macOS wasn't finding the "QEMUUSB
" device at the expected address.
It turns out that the original SSDT in OSX-KVM had defined the PCI address at a different location.
The original binary SSDT-EHCI.aml
file inside OpenCore.qcow2
image, under EFI/OC/ACPI/
contained an ACPI table called QEMUUSB
.
That QEMUUSB
table defined the PCI addresses for the USB controllers, as expected to be attached by QEMU.
However, the problem was that these addresses were not matching what QEMU was giving the devices automatically.
To demystify what's going on, we can use Intel's iasl
tool (included in the acpica
Arch Linux package).
We can decompile the .aml
file and view the source code to find what devices and PCI addresses were defined:
$ cp /tmp/oc/EFI/OC/ACPI/SSDT-EHCI.aml /tmp/
$ iasl /tmp/SSDT-EHCI.aml
Intel ACPI Component Architecture
ASL+ Optimizing Compiler/Disassembler version 20240927
Copyright (c) 2000 - 2023 Intel Corporation
File appears to be binary: found 58 non-ASCII characters, disassembling
Binary file appears to be a valid ACPI table, disassembling
Input file /tmp/SSDT-EHCI.aml, Length 0xA3 (163) bytes
ACPI: SSDT 0x0000000000000000 0000A3 (v01 KGP QEMUUSB 00000000 INTL 20190509)
Pass 1 parse of [SSDT]
Pass 2 parse of [SSDT]
Parsing Deferred Opcodes (Methods/Buffers/Packages/Regions)
Parsing completed
Disassembly completed
ASL Output: /tmp/SSDT-EHCI.dsl - 1232 bytes
### Inspecting the original SSDT for QEMUUSB
$ cat /tmp/SSDT-EHCI.dsl
/*
* Intel ACPI Component Architecture
* AML/ASL+ Disassembler version 20240927 (64-bit version)
* Copyright (c) 2000 - 2023 Intel Corporation
*
* Disassembling to symbolic ASL+ operators
*
* Disassembly of /tmp/SSDT-EHCI.aml
*
* Original Table Header:
* Signature "SSDT"
* Length 0x000000A3 (163)
* Revision 0x01
* Checksum 0xBF
* OEM ID "KGP"
* OEM Table ID "QEMUUSB"
* OEM Revision 0x00000000 (0)
* Compiler ID "INTL"
* Compiler Version 0x20190509 (538510601)
*/
DefinitionBlock ("", "SSDT", 1, "KGP", "QEMUUSB", 0x00000000)
{
External (_SB_.PCI0, DeviceObj)
External (_SB_.PCI0.S38_, DeviceObj)
Scope (\_SB.PCI0)
{
Device (EH01)
{
Name (_ADR, 0x00070007) // _ADR: Address
}
Scope (S38)
{
Name (_STA, Zero) // _STA: Status
}
Device (UHC1)
{
Name (_ADR, 0x00070000) // _ADR: Address
}
Device (UHC2)
{
Name (_ADR, 0x00070001) // _ADR: Address
}
Device (UHC3)
{
Name (_ADR, 0x00070002) // _ADR: Address
}
}
}
So, we can see it defines 4 devices:
EH01
ataddr=0x7.7
(_ADR
0x00070007
=slot: 7, function: 7
) - A USB2.0 controllerUHC1
ataddr=0x7.0
(_ADR
0x00070000
=slot: 7, function: 0
) - 1st USB1.x controllerUHC2
ataddr=0x7.1
(_ADR
0x00070001
=slot: 7, function: 1
) - 2nd USB1.x controllerUHC3
ataddr=0x7.2
(_ADR
0x00070002
=slot: 7, function: 2
) - 3rd USB1.x controller
In ACPI source language, the _ADR
defines the address in hex.
Each word breaks down into 2 parts: slot
and function
(in QEMU / libvirt
terminology).
As it turns out, this definition is wrong for the QEMU EHCI and UHCI devices I
passed, because they each occupied a single slot. (multifunction=on
was not set).
When I tried to add those devices and set addr=
parameters in QEMU using those function
numbers,
it gave me this error:
qemu-system-x86_64: -device ich9-usb-uhci2,id=uhci2,bus=pcie.0,addr=0x7.0x1: PCI: single function device can't be populated in function 7.1
Evidently the ich9-usb-
uhci*
& ehci*
devices are all "single function" devices,
unless the device at function 0x0
has the multifunction=on
parameter set.
So, unless we change our QEMU device addresses to match the default SSDT, we must fix the SSDT so it matches what devices and possible PCI addresses QEMU will give the VM.
That means in our example each of those USB controllers would need to occupy a
single slot, without any sub-function
numbers attached to them.
Alternatively, we could choose to set multifunction=on
for the first device
and match sub-device slot numbers to the original SSDT-EHCI.aml
.
In this guide, I will choose to edit the SSDT to demonstrate how it's done.
To fix this, we can:
- Edit the
SSDT-EHCI.dsl
file - Recompile it with
iasl
into a newSSDT-EHCI.aml
- Place it inside the
OpenCore.qcow2
image in the proper directory, using the same steps we used to mount the image and partition before.
Knowing that QEMU q35
machine has certain reserved & default PCI addresses,
we can edit the QEMUUSB
ACPI table to fix those PCI addresses it defines:
/*
* Intel ACPI Component Architecture
* AML/ASL+ Disassembler version 20240927 (64-bit version)
* Copyright (c) 2000 - 2023 Intel Corporation
*
* Disassembling to symbolic ASL+ operators
*
* Disassembly of /tmp/SSDT-EHCI.aml
*
* Original Table Header:
* Signature "SSDT"
* Length 0x000000A3 (163)
* Revision 0x01
* Checksum 0xBF
* OEM ID "KGP"
* OEM Table ID "QEMUUSB"
* OEM Revision 0x00000000 (0)
* Compiler ID "INTL"
* Compiler Version 0x20190509 (538510601)
*/
DefinitionBlock ("", "SSDT", 1, "KGP", "QEMUUSB", 0x00000000)
{
External (_SB_.PCI0, DeviceObj)
External (_SB_.PCI0.S38_, DeviceObj)
Scope (\_SB.PCI0)
{
Device (EH01)
{
Name (_ADR, 0x001a0000) // _ADR: Address bus=pcie.0 addr=0x1a.0
}
Scope (S38)
{
Name (_STA, Zero) // _STA: Status
}
Device (UHC1)
{
Name (_ADR, 0x00070000) // _ADR: Address bus=pcie.0 addr=0x07.0
}
Device (UHC2)
{
Name (_ADR, 0x00080000) // _ADR: Address bus=pcie.0 addr=0x8.0
}
Device (UHC3)
{
Name (_ADR, 0x00090000) // _ADR: Address bus=pcie.0 addr=0x9.0
}
}
}
In the edited version of SSDT-EHCI.dsl
above, I've matched the
ehci
USB 2.0 controller to QEMU's default PCI address location of 0x1a.0
.
This is an arbitrary choice, but can sometimes help in the case when not
specifying a default address for QEMU CLI -device
arguments.
However, it can often be best to avoid ambiguity, so declarative CLI params can
help to avoid SSDT mismatch. In the examples below, we will choose to specify
the QEMU device addresses with 'addr
'. Finally, we will choose to avoid
passing multifunction=on
, so all slot
numbers are zero '.0
'.
Next, I've changed all the addresses for uhci
USB 1.1 devices to different
unused QEMU PCI slot numbers: 0x7.0
, 0x8.0
, and 0x9.0
.
To add these devices to the VM, I passed QEMU CLI args:
## Ensure ACPI SSDT matches this
# USB2.0 EH01 in ACPI SSDT _ADR = 0x001a0000
-device 'ich9-usb-ehci1,id=ehci,bus=pcie.0,addr=0x1a.0'
# USB1.1 UHC1 in ACPI SSDT _ADR = 0x00070000
-device 'ich9-usb-uhci1,id=uhci1,bus=pcie.0,addr=0x7.0'
# USB1.1 UHC2 in ACPI SSDT _ADR = 0x00080000
-device 'ich9-usb-uhci2,id=uhci2,bus=pcie.0,addr=0x8.0'
# USB1.1 UHC3 in ACPI SSDT _ADR = 0x00090000
-device 'ich9-usb-uhci3,id=uhci3,bus=pcie.0,addr=0x9.0'
## Finally, attach the usb-kbd and usb-tablet to first USB2.0 device's EHCI bus with 'ehci.0'
-device 'usb-kbd,bus=ehci.0,id=input0,port=2'
-device 'usb-tablet,bus,ehci.0,id=input1,port=3'
Translated into libvirt
XML format:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<!-- ... Original VM definition ... -->
<qemu:commandline>
<!-- ... other args ... -->
<qemu:arg value='-device'/>
<qemu:arg value='ich9-usb-ehci1,id=ehci,bus=pcie.0,addr=0x1a.0'/>
<qemu:arg value='-device'/>
<qemu:arg value='ich9-usb-uhci1,id=uhci1,bus=pcie.0,addr=0x7.0'/>
<qemu:arg value='-device'/>
<qemu:arg value='ich9-usb-uhci2,id=uhci2,bus=pcie.0,addr=0x8.0'/>
<qemu:arg value='-device'/>
<qemu:arg value='ich9-usb-uhci3,id=uhci3,bus=pcie.0,addr=0x9.0'/>
<qemu:arg value='-device'/>
<qemu:arg value='usb-kbd,bus=ehci.0,id=input0,port=2'/>
<qemu:arg value='-device'/>
<qemu:arg value='usb-tablet,bus,ehci.0,id=input1,port=3'/>
</qemu:commandline>
<!-- ... rest of VM definition ... -->
</domain>
After making these changes, I recompiled the SSDT and placed it into the OpenCore.qcow2
image:
$ iasl /tmp/SSDT-EHCI.dsl
Intel ACPI Component Architecture
ASL+ Optimizing Compiler/Disassembler version 20240927
Copyright (c) 2000 - 2023 Intel Corporation
ASL Input: /tmp/SSDT-EHCI.dsl - 1322 bytes 11 keywords 0 source lines
AML Output: /tmp/SSDT-EHCI.aml - 163 bytes 0 opcodes 11 named objects
Compilation successful. 0 Errors, 0 Warnings, 0 Remarks, 0 Optimizations
$ cp /tmp/SSDT-EHCI.aml /tmp/oc/EFI/OC/ACPI/SSDT-EHCI.aml
$ sudo umount /tmp/oc
$ sudo qemu-nbd --disconnect /dev/nbd0
/dev/nbd0 disconnected
After booting the VM again, the QEMUUSB
ACPI table is found!
The mouse and keyboard worked, and the following was output in the kernel debug console:
ACPI: SDT 0x0000000079E4000 0000A3 (v01 KGP QEMUUSB 0000000 INTL 2024097)
Now the VM has USB controllers with addresses matching the SSDT-EHCI.aml
.
I hope this helps demystify some of these things.