Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save trinitronx/2c1544d35a5b0502620d843b2ff1c1d5 to your computer and use it in GitHub Desktop.
Save trinitronx/2c1544d35a5b0502620d843b2ff1c1d5 to your computer and use it in GitHub Desktop.
Diagnosing and fixing mismatched SSDT USB controller PCI addresses in OSX-KVM

Investigating what OSX-KVM's pre-packaged SSDT files contained, I ran into an issue with the default USB controllers' PCI addresses. The USB controllers I was passing to QEMU did not match the default VM definition & SSDT.

To more easily see the XNU kernel debugging output:

  • I added a serial console device to the VM via this libvirt domain XML snippet:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <!-- ... Original VM definition ... -->
  <devices>
  <!-- ... other devices ... -->

    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>

  <!-- ... rest of VM definition ... -->
  </devices>
</domain>
  • I added the following XNU boot-args: debug=0x10A -v serial=3 msgbuf=1048576 serialbaud=115200
    • -v enables verbose booting
    • serial=3 - Sends debug output over serial
    • msgbuf=10485760 - Sets kern.msgbuf to 10485760 bytes (10 MiB) to allow a much larger kernel-log.
    • serialbaud=115200 - Sets the serial TTY's baud rate (e.g. to connect with screen /dev/pts/NN 115200)
    • The debug=0x10A bit mask enables DB_LOG_PI_SCRN, DB_SLOG, and DB_KPRT which respectively:
      • Disables graphical panic screen
      • Sends some diagnostics to syslog
      • Sends kernel's debugging kprintf to the serial console port
      • Note: A bitmask can be calculated by a logical OR operation on those values. For example: 0x100 | 0x2 | 0x8 = 0x10A
      • To add this, I had to:
        • First mount the OpenCore.qcow2 disk with qemu-nbd
          • qemu-nbd needs nbd kernel module loaded: modprobe nbd max_part=8
        • Then mount the inner vfat EFI partition (containing OpenCore bootloader, kexts and ACPI SSDTs)
        • Edit the config.plist file inside
        • Unmount the EFI partition & disconnect the qcow2 file from /dev/nbd0
  • After changing boot-args, the kernel console kprintf debug output can be viewed in:
    • The virt-manager window under: View -> Consoles -> Serial 1
    • By directly connecting to the /dev/pts/NN device with GNU screen: screen /dev/pts/5.
    • Note: The device number can change, so pay attention that you're connecting to the right one. This can be seen in virt-manager's "Details" screen by checking the Serial 1 device under "Source path:"

CLI Steps to enable XNU debugging

$ sudo qemu-nbd --discard=unmap --connect=/dev/nbd0  ~/src/pub/OSX-KVM/OpenCore/OpenCore.qcow2

$ sudo sgdisk --print  /dev/nbd0
Disk /dev/nbd0: 786432 sectors, 384.0 MiB
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 3D196FDE-9679-43B2-B0EA-52513C02D436
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 786398
Partitions will be aligned on 32-sector boundaries
Total free space is 6075 sectors (3.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          300000   145.5 MiB   EF00  primary
   2          302048          784384   235.5 MiB   8300  primary

$ sudo blkid  /dev/nbd0p1
/dev/nbd0p1: SEC_TYPE="msdos" LABEL_FATBOOT="EFI" LABEL="EFI" UUID="BB7E-D63C" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="primary" PARTUUID="d79d1e93-e748-4fef-8de6-fd4e37e3ec5b"

$ mkdir /tmp/oc
$ sudo mount -t vfat  /dev/nbd0p1 /tmp/oc

$ vim /tmp/oc/EFI/OC/config.plist

## Search for: boot-args with: /boot-args
## Add the kernel debug boot args:
##   debug=0x10A -v serial=3 msgbuf=10485760 serialbaud=115200 keepsyms=1 amfi_get_out_of_my_way=1 tlbto_us=0 vti=9

$ sudo umount /tmp/oc
$ sudo qemu-nbd --disconnect /dev/nbd0
/dev/nbd0 disconnected

For reference, the original boot-args defined as default by OSX-KVM were:

<key>boot-args</key>
<string>-v keepsyms=1 amfi_get_out_of_my_way=1 tlbto_us=0 vti=9</string>

After editing, I ended up with the following boot-args under the <key>NVRAM</key><dict><key>Add</key>... <key>boot-args</key> ... section in config.plist:

<key>boot-args</key>
<string>debug=0x10A -v serial=3 msgbuf=10485760 serialbaud=115200 keepsyms=1 amfi_get_out_of_my_way=1 tlbto_us=0 vti=9</string>

Note: For more details on XNU kernel debugging, see here

Booting the VM -> Kernel ACPI errors

After booting the kernel, I saw some ACPI table errors in the console log:

ACPI Exception: AE_NOT_FOUND, (SSDT: QEMUUSB) while loading table (20160930/tbxfload-319)

So, for some reason macOS wasn't finding the "QEMUUSB" device at the expected address. It turns out that the original SSDT in OSX-KVM had defined the PCI address at a different location.

The original binary SSDT-EHCI.aml file inside OpenCore.qcow2 image, under EFI/OC/ACPI/ contained an ACPI table called QEMUUSB. That QEMUUSB table defined the PCI addresses for the USB controllers, as expected to be attached by QEMU. However, the problem was that these addresses were not matching what QEMU was giving the devices automatically.

To demystify what's going on, we can use Intel's iasl tool (included in the acpica Arch Linux package).

We can decompile the .aml file and view the source code to find what devices and PCI addresses were defined:

$ cp   /tmp/oc/EFI/OC/ACPI/SSDT-EHCI.aml /tmp/
$ iasl  /tmp/SSDT-EHCI.aml

Intel ACPI Component Architecture
ASL+ Optimizing Compiler/Disassembler version 20240927
Copyright (c) 2000 - 2023 Intel Corporation

File appears to be binary: found 58 non-ASCII characters, disassembling
Binary file appears to be a valid ACPI table, disassembling
Input file /tmp/SSDT-EHCI.aml, Length 0xA3 (163) bytes
ACPI: SSDT 0x0000000000000000 0000A3 (v01 KGP    QEMUUSB  00000000 INTL 20190509)
Pass 1 parse of [SSDT]
Pass 2 parse of [SSDT]
Parsing Deferred Opcodes (Methods/Buffers/Packages/Regions)

Parsing completed
Disassembly completed
ASL Output:    /tmp/SSDT-EHCI.dsl - 1232 bytes


### Inspecting the original SSDT for QEMUUSB

$ cat /tmp/SSDT-EHCI.dsl
/*
 * Intel ACPI Component Architecture
 * AML/ASL+ Disassembler version 20240927 (64-bit version)
 * Copyright (c) 2000 - 2023 Intel Corporation
 * 
 * Disassembling to symbolic ASL+ operators
 *
 * Disassembly of /tmp/SSDT-EHCI.aml
 *
 * Original Table Header:
 *     Signature        "SSDT"
 *     Length           0x000000A3 (163)
 *     Revision         0x01
 *     Checksum         0xBF
 *     OEM ID           "KGP"
 *     OEM Table ID     "QEMUUSB"
 *     OEM Revision     0x00000000 (0)
 *     Compiler ID      "INTL"
 *     Compiler Version 0x20190509 (538510601)
 */
DefinitionBlock ("", "SSDT", 1, "KGP", "QEMUUSB", 0x00000000)
{
    External (_SB_.PCI0, DeviceObj)
    External (_SB_.PCI0.S38_, DeviceObj)

    Scope (\_SB.PCI0)
    {
        Device (EH01)
        {
            Name (_ADR, 0x00070007)  // _ADR: Address
        }

        Scope (S38)
        {
            Name (_STA, Zero)  // _STA: Status
        }

        Device (UHC1)
        {
            Name (_ADR, 0x00070000)  // _ADR: Address
        }

        Device (UHC2)
        {
            Name (_ADR, 0x00070001)  // _ADR: Address
        }

        Device (UHC3)
        {
            Name (_ADR, 0x00070002)  // _ADR: Address
        }
    }
}

So, we can see it defines 4 devices:

  • EH01 at addr=0x7.7 (_ADR 0x00070007 = slot: 7, function: 7) - A USB2.0 controller
  • UHC1 at addr=0x7.0 (_ADR 0x00070000 = slot: 7, function: 0) - 1st USB1.x controller
  • UHC2 at addr=0x7.1 (_ADR 0x00070001 = slot: 7, function: 1) - 2nd USB1.x controller
  • UHC3 at addr=0x7.2 (_ADR 0x00070002 = slot: 7, function: 2) - 3rd USB1.x controller

In ACPI source language, the _ADR defines the address in hex. Each word breaks down into 2 parts: slot and function (in QEMU / libvirt terminology).

As it turns out, this definition is wrong for the QEMU EHCI and UHCI devices I passed, because they each occupied a single slot. (multifunction=on was not set). When I tried to add those devices and set addr= parameters in QEMU using those function numbers, it gave me this error:

qemu-system-x86_64: -device ich9-usb-uhci2,id=uhci2,bus=pcie.0,addr=0x7.0x1: PCI: single function device can't be populated in function 7.1 

Evidently the ich9-usb- uhci* & ehci* devices are all "single function" devices, unless the device at function 0x0 has the multifunction=on parameter set.

So, unless we change our QEMU device addresses to match the default SSDT, we must fix the SSDT so it matches what devices and possible PCI addresses QEMU will give the VM.

That means in our example each of those USB controllers would need to occupy a single slot, without any sub-function numbers attached to them.

Alternatively, we could choose to set multifunction=on for the first device and match sub-device slot numbers to the original SSDT-EHCI.aml.

In this guide, I will choose to edit the SSDT to demonstrate how it's done.

To fix this, we can:

  • Edit the SSDT-EHCI.dsl file
  • Recompile it with iasl into a new SSDT-EHCI.aml
  • Place it inside the OpenCore.qcow2 image in the proper directory, using the same steps we used to mount the image and partition before.

Fixing SSDT-EHCI

Knowing that QEMU q35 machine has certain reserved & default PCI addresses, we can edit the QEMUUSB ACPI table to fix those PCI addresses it defines:

/*
 * Intel ACPI Component Architecture
 * AML/ASL+ Disassembler version 20240927 (64-bit version)
 * Copyright (c) 2000 - 2023 Intel Corporation
 * 
 * Disassembling to symbolic ASL+ operators
 *
 * Disassembly of /tmp/SSDT-EHCI.aml
 *
 * Original Table Header:
 *     Signature        "SSDT"
 *     Length           0x000000A3 (163)
 *     Revision         0x01
 *     Checksum         0xBF
 *     OEM ID           "KGP"
 *     OEM Table ID     "QEMUUSB"
 *     OEM Revision     0x00000000 (0)
 *     Compiler ID      "INTL"
 *     Compiler Version 0x20190509 (538510601)
 */
DefinitionBlock ("", "SSDT", 1, "KGP", "QEMUUSB", 0x00000000)
{
    External (_SB_.PCI0, DeviceObj)
    External (_SB_.PCI0.S38_, DeviceObj)

    Scope (\_SB.PCI0)
    {
        Device (EH01)
        {
            Name (_ADR, 0x001a0000)  // _ADR: Address bus=pcie.0 addr=0x1a.0
        }

        Scope (S38)
        {
            Name (_STA, Zero)  // _STA: Status
        }

        Device (UHC1)
        {
            Name (_ADR, 0x00070000)  // _ADR: Address bus=pcie.0 addr=0x07.0
        }

        Device (UHC2)
        {
            Name (_ADR, 0x00080000)  // _ADR: Address bus=pcie.0 addr=0x8.0
        }

        Device (UHC3)
        {
            Name (_ADR, 0x00090000)  // _ADR: Address bus=pcie.0 addr=0x9.0
        }
    }
}

In the edited version of SSDT-EHCI.dsl above, I've matched the ehci USB 2.0 controller to QEMU's default PCI address location of 0x1a.0. This is an arbitrary choice, but can sometimes help in the case when not specifying a default address for QEMU CLI -device arguments. However, it can often be best to avoid ambiguity, so declarative CLI params can help to avoid SSDT mismatch. In the examples below, we will choose to specify the QEMU device addresses with 'addr'. Finally, we will choose to avoid passing multifunction=on, so all slot numbers are zero '.0'.

Next, I've changed all the addresses for uhci USB 1.1 devices to different unused QEMU PCI slot numbers: 0x7.0, 0x8.0, and 0x9.0.

To add these devices to the VM, I passed QEMU CLI args:

## Ensure ACPI SSDT matches this
# USB2.0 EH01 in ACPI SSDT _ADR = 0x001a0000
-device 'ich9-usb-ehci1,id=ehci,bus=pcie.0,addr=0x1a.0'

# USB1.1 UHC1 in ACPI SSDT _ADR = 0x00070000
-device 'ich9-usb-uhci1,id=uhci1,bus=pcie.0,addr=0x7.0'
# USB1.1 UHC2 in ACPI SSDT _ADR = 0x00080000
-device 'ich9-usb-uhci2,id=uhci2,bus=pcie.0,addr=0x8.0'
# USB1.1 UHC3 in ACPI SSDT _ADR = 0x00090000
-device 'ich9-usb-uhci3,id=uhci3,bus=pcie.0,addr=0x9.0'

## Finally, attach the usb-kbd and usb-tablet to first USB2.0 device's EHCI bus with 'ehci.0'
-device 'usb-kbd,bus=ehci.0,id=input0,port=2'
-device 'usb-tablet,bus,ehci.0,id=input1,port=3'

Translated into libvirt XML format:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <!-- ... Original VM definition ... -->

  <qemu:commandline>

  <!-- ... other args ... -->

    <qemu:arg value='-device'/>
    <qemu:arg value='ich9-usb-ehci1,id=ehci,bus=pcie.0,addr=0x1a.0'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='ich9-usb-uhci1,id=uhci1,bus=pcie.0,addr=0x7.0'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='ich9-usb-uhci2,id=uhci2,bus=pcie.0,addr=0x8.0'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='ich9-usb-uhci3,id=uhci3,bus=pcie.0,addr=0x9.0'/>

    <qemu:arg value='-device'/>
    <qemu:arg value='usb-kbd,bus=ehci.0,id=input0,port=2'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='usb-tablet,bus,ehci.0,id=input1,port=3'/>
  </qemu:commandline>

  <!-- ... rest of VM definition ... -->
</domain>

After making these changes, I recompiled the SSDT and placed it into the OpenCore.qcow2 image:

$ iasl /tmp/SSDT-EHCI.dsl

Intel ACPI Component Architecture
ASL+ Optimizing Compiler/Disassembler version 20240927
Copyright (c) 2000 - 2023 Intel Corporation

ASL Input:     /tmp/SSDT-EHCI.dsl -    1322 bytes     11 keywords      0 source lines
AML Output:    /tmp/SSDT-EHCI.aml -     163 bytes      0 opcodes      11 named objects

Compilation successful. 0 Errors, 0 Warnings, 0 Remarks, 0 Optimizations

$ cp /tmp/SSDT-EHCI.aml  /tmp/oc/EFI/OC/ACPI/SSDT-EHCI.aml
$ sudo umount /tmp/oc
$ sudo qemu-nbd --disconnect /dev/nbd0
/dev/nbd0 disconnected

After booting the VM again, the QEMUUSB ACPI table is found!

The mouse and keyboard worked, and the following was output in the kernel debug console:

ACPI: SDT 0x0000000079E4000 0000A3 (v01 KGP    QEMUUSB  0000000 INTL 2024097)

Now the VM has USB controllers with addresses matching the SSDT-EHCI.aml.

I hope this helps demystify some of these things.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment