Running Third Party Kernel Extensions on Virtualization Framework macOS Guest VMs

ThirdPartyKexts.md

Running Third Party Kernel Extensions on Virtualization Framework macOS Guest VMs

As of macOS 12 (Monterey), Apple's Virtualization framework has nice support for macOS guest virtual machines, but with severe limitations: For example you can't install a macOS guest on Intel Macs, install guests with newer versions of macOS than the host, copy and paste between the host and the guest, or install third party kernel extensions in the guest. As usual for Apple, the functionality they do support is nicely implemented, but they've left out so much that the result is only marginally useful -- at least compared to third-party implementations available on Intel Macs, like VMware Fusion and Parallels.

I've been working on the last of these limitations, and have found a workaround. It's rather complex, but should be very useful for those developing kernel extensions for macOS on Apple Silicon, or (more likely) porting existing Intel kexts to Apple Silicon. Developing kernel extensions on bare metal is a pain -- not least because you might end up damaging your nice new Apple Silicon Mac. But doing it in a virtual machine isolates the potential damage to the VM itself -- at worst you'll only need to trash it and create a replacement.

Parts of my workaround are interesting in themselves, even if you don't use them to run third party kernel extensions on macOS guest VMs. For example I've found a way to patch the VM's kernel cache (containing the kernel and built-in kernel extensions), which should be very useful in reverse engineering macOS.

Most of the discoveries here are ones I've made on my own, by trial and error. But I had a very helpful starting point -- NyanSatan's Virtual-iBoot-Fun project.

Prerequisites

• An Apple Silicon Mac running macOS 12 or higher. Mine is a 2020 Mac Mini (Macmini9,1).

• A decent virtual machine host that uses Apple's Virtualization framework. I use UTM.

• One or more decent disassemblers. I use Ghidra and Hopper Disassembler. I also installed Nick Botticelli's ghidra-iboot plugin

• Tihmstar's img4tool.

• A decent hex editor. I use Hex Fiend. I change its default edit mode to "Overwrite" and its line number format to "Hexadecimal".

• A binary diff tool. I use VBinDiff.

• A calculator with support for radix modes. Apple's Calculator in Programmer mode will do.

Background

My workaround patches three iBoot modules and the VM's kernel cache.

The iBoot modules load early in the macOS (and IOS) boot sequence, before the kernel. There are at least three variants of iBoot, used in three diffent "stages" -- Stage 0 (as I call it), Stage 1 and Stage 2. On bare metal, Stage 0 is implemented in hardware (in Apple's "Secure Enclave"). Stage 1 (aka LLB) and Stage 2 (iBoot properly so called) are implemented in software, though only the Stage 2 iBoot exists (inside iBoot.img4) in the macOS file system.

By design, the Secure Enclave Processor (SEP) is inaccessible to ordinary mortals. And on bare metal the LLB and iBoot modules are encrypted (using a key held in the SEP). But for a VM, the modules for all three stages are unencrypted. Stage 0 is implemented (on the host) in /System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin. Stage 1 is embedded (in img4 format) in an AuxiliaryStorage file (on the host) associated with the DMG file (with an img extension) that stores the VM's image. Stage 2 (inside iBoot.img4) exists in the VM's file system.

None of the iBoot variants call any external functions, and none have any symbols. So they're difficult to decipher. But the fact that the Virtualization framework uses them without encryption is a golden opportunity to reverse engineer them, and understand them better.

It's apparently common knowledge that the iBoot modules all contain a function to check the "digests" (DGST) of the many img4 images that govern the macOS boot process (including LLB, iBoot.img4 and the kernel cache itself). Each "digest" is a hash, used to ensure that none of these images has changed since they were "signed" by Apple. The source code for a similar function (image4_validate_property_callback()) is available here.

The images in question are those to be loaded at the next stage. So (among other things) Stage 0 iBoot checks the digest of the Stage 1 iBoot img4 image, Stage 1 iBoot checks the digest of the Stage 2 iBoot img4 file, and Stage 2 iBoot checks the digest of the kernelcache file. Aside from the file system itself, nothing checks the integrity of the stage 0 iBoot file (AVPBooter.vmapple2.bin). So all you need to stop these integrity checks is to defeat the file system's protection and patch this function (call it image4_validate_property_callback()) at every "stage".

As NyanSatan mentions, the standard approach is to patch out image4_validate_property_callback() completely -- to replace it with code that "returns 0". I find this causes problems, so my patch is more surgical -- I allow image4_validate_property_callback() to perform all its normal checks, and only patch its return value (to '0') just before it returns.

Some time ago I discovered that failed calls to _validate_acm_context() in the VM's AppleVPBootPolicy.kext stop the VM from creating an auxiliary kext cache. Since third-party kexts are stored here, this prevents them from loading in a Virtualization framework VM. The failed calls originate from two other functions in AppleVPBootPolicy.kext -- _command_create_linked_manifest() and _command_update_local_policy_for_kcos(). My workaround is to patch out the calls to _validate_acm_context() from both of these functions. This is a more "surgical" approach than patching out _validate_acm_context() itself, which is called from many additional functions in AppleVPBootPolicy.kext.

But this isn't enough by itself. An auxiliary kext cache does get created (you can see it using kmutil inspect). But macOS still complains that your third-party kext needs to be rebuilt. To fix this you need to go back to the Stage 2 iBoot module (inside iBoot.img4). One of its purposes is to fill the "device tree" with all the appropriate devices. But it (or the variant available in a Virtualization framework VM) refuses to allow any "AuxKC" entries to be added to chosen/memory-map in a VM, even when the auxiliary kext cache is present. Fixing this requires finding the function that checks whether a given boot object (originally in img4 format) should be processed, and patching out the call to it. Its first parameter is a four-character code -- for example illb, ibot, krnl or auxk. validate_boot_object() (my name for this function) indirectly calls image4_validate_property_callback(). So if you patch out the former you don't need to patch the latter. At some point I'll describe how I found validate_boot_object().

To display the contents of chosen/memory-map, run the following command in a Terminal prompt:

ioreg -p IODeviceTree -n "memory-map" -w 0 -r -t

Settings Changes (Host and Guest)

Some settings changes are required on the host by How to Defang macOS System Protections, which allows changes to AVPBooter.vmapple2.bin.

• In System Settings, under Software Update, disable "Download new updates when available". Otherwise your host can become unbootable.

• Boot into Recovery Mode, run Terminal and do the following:

csrutil disable
csrutil authenticated-root disable

You also, of course, need to create a macOS guest VM. Make sure it uses the Virtualization framework. In UTM this is accomplished by choosing "Virtualize" and then "macOS 12+".

Terminal will be used heavily below. So, to avoid lots of annoying prompts to give it permission to access files, it's best to give the Terminal app "full disk access" on the guest (in Privacy and Security in System Settings).

Then boot into Recovery Mode on the guest. On macOS 12 you'll need to use my hack to do this.

• Run the Startup Security Utility. Choose Reduced Security and "Allow user management of kernel extensions from identified developers".

• Run Terminal and do the following:

• csrutil disable, then y to "Allow booting unsigned operating systems and any kernel extensions".

Special Considerations Once You've Made the Changes Described Here

Below you'll create (and boot from) an APFS snapshot on your host computer to accomodate changes to the Stage 0 iBoot module (AVMBooter.vmapple2.bin). You need to keep your host computer in this state as long as you're using third party kernel extensions in your macOS guest VM. (Without the patched AVPBooter.vmapple2.bin it will simply refuse to start.) You also need to keep csrutil's "authenticated-root" disabled, and keep "Download new updates when available" disabled under Software Update. But don't use Software Update to update macOS on your host while it's booted from this custom snapshot. I don't know that this will cause trouble, but I expect it might.

Use the following command to revert your snapshot (and AVPBooter.vmapple2.bin), then reboot your computer:

sudo bless --mount / --last-sealed-snapshot

Release version macOS 13 and 14 guest VMs can be upgraded to newer versions of macOS. But before you do so you should revert this document's changes to the Stage 2 iBoot module (iBoot.img4) and the kernel cache (kernelcache): Boot into Recovery Mode, run the Startup Security Utility, and choose Full Security. Afterwards you'll need to go through all the following steps again, from scratch. Among other things you'll need to create new iBoot.img4.org and kernelcache.org files, and possibly also a new LLB.img4.org file.

macOS 12 guests can't be upgraded. The Virtualization process (com.apple.Virtualization.VirtualMachine) crashes and your VM becomes unbootable. This is an Apple bug. I've seen it myself, and have seen it documented here. The same is true for beta-version macOS 14 guests, though the consequences are less severe.

In this case your only option is to create a new guest VM directly from an IPSW file, then work through the following steps.

Finding the Modules to be Patched

iBoot Stage 0 (AVPBooter.vmapple2.bin)

This one's on the host, and is easy to find. Copy it and rename the copy to AVPBooter.vmapple2.bin.org.

/System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin

iBoot Stage 1 (LLB)

This one's also on the host, but is harder to find. By default, VMs created by UTM are stored in the following directory:

~/Library/Containers/com.utmapp.UTM/Data/Documents

Each VM is stored in a "package" with the extension .utm. Inside the package, in its Data subdirectory, is a file named AuxiliaryStorage. LLB is embedded here, in img4 format. It's followed immediately by an img4 format image of the "logo". You'll need to get copies of both.

If you've upgraded your guest VM to a newer version of macOS, there will be two copies of the LLB image in the AuxiliaryStorage file, each followed by a "logo" image. Only the second set will be active. So that's the one you'll need to work on. And if you've already patched a previous version of macOS in your guest VM (following the instructions in this document), the second LLB image may still contain your earlier patch. If so, you won't need to repatch it.

The first LLB image is always at offset 0x24000. The second, if it exists, should be at offset 0x224000. To be sure you've found all the LLB images, and their correct locations, search (in your hex editor) on "illb" (the four character code for the LLB image).

• Open AuxiliaryStorage in a hex editor. I use Hex Fiend, and will tailor my steps to it specifically.

• Jump to offset 0x224000. If an LLB image exists at this location, its first two bytes should be 3083. If not, jump to offset 0x24000.

• The img4 image is in DER format (used by ASN1), so the first five bytes are 3083 followed by a three digit hexadecimal number in big endian format (for example 037FD1). This number is the length of the image, exclusive of the length of its header. So the total length is 0x37FD6.

• Select the first five bytes of the image, then choose "Extend Selection". In this case, you'd extend it by 0x37FD1 bytes. Scroll down to the end of the selection (without disturbing it) and check that it's in the correct location (just before the "logo" image). If it is, CMD-C to copy the image, CMD-N to open a new window, CMD-V to paste in its contents, and save the file as LLB.img4.org.

• The header for the "logo" image should be 3082 followed by a two digit hexdecimal number (for example 36DA). Select the first four bytes, extend the selection by 0x36DA bytes and scroll down to its end. The following bytes should be a bunch of NULLs (00). CMD-C to copy the image, CMD-N to open another new window, CMD-V to paste in its contents, and save this second new file as logo.img4.org.

iBoot Stage 2 (iBoot.img4) and the Kernel Cache (kernelcache)

These modules are stored in files on the VM. So run the VM and do the following in it, at a Terminal prompt:

• Run kmutil inspect, and observe where the boot kernel cache exists in the VM's file system. The following is an example, which will get used in the following steps. The path's exact contents will differ from case to case.

/System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache

• Note the long hexadecimal number just before /System/Library/Caches -- in this case FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83. This is the "Next Stage Image4 Hash (nsih)", observable in the output of sudo bputil -d.

• Copy this file and rename it to kernelcache.org.

• cd / and sudo find . -name iBoot.img4 -exec ls -al \{\} \;. There will be at least two hits. Choose the one whose path contains the Next Stage Image4 Hash. Copy it and rename the copy to iBoot.img4.org. You might find the original iBoot.img4 here, for example:

/System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4

Patching These Modules

iBoot Stage 0 (AVPBooter.vmapple2.bin.org)

For this I use Ghidra with the ghidra-iboot plugin. Copy AVPBooter.vmapple2.bin.org to AVPBooter.vmapple2.bin. Then run Ghidra and create a new project. Then:

• Run Ghidra's CodeBrowser and import AVPBooter.vmapple2.bin.

• Choose Search : Program Text : All Fields. Type "0x4447" ('DG' of DGST) in "Search for", then choose "Search All". You should find two hits, both in the same function. Click on one to move the cursor to its location.

• Once again choose Search : Program Text : All Fields, and type "retab" in "Search For". Then choose "Next". Now you should see the code that runs just before the function returns. It will look something like as follows. In this example, you want to change the instruction at address 0x00102a0c (mov x0,x20) to mov x0,#0x0. (AARCH64 machine code uses the X0 register to store a return value.)

001029f4 a8 83 5a f8     ldur       x8,[x29, #local_68]
001029f8 29 f9 37 d0     adrp       x9,0x70028000
001029fc 1f 20 03 d5     nop
00102a00 29 a1 40 f9     ldr        x9,[x9, #0x140]=>DAT_70028140
00102a04 3f 01 08 eb     cmp        x9,x8
00102a08 21 06 00 54     b.ne       LAB_00102acc
00102a0c e0 03 14 aa     mov        x0,x20
00102a10 fd 7b 4c a9     ldp        x29=>local_10,x30,[sp, #0xc0]
00102a14 f4 4f 4b a9     ldp        x20,x19,[sp, #local_20]
00102a18 f6 57 4a a9     ldp        x22,x21,[sp, #local_30]
00102a1c f8 5f 49 a9     ldp        x24,x23,[sp, #local_40]
00102a20 fa 67 48 a9     ldp        x26,x25,[sp, #local_50]
00102a24 fc 6f 47 a9     ldp        x28,x27,[sp, #local_60]
00102a28 ff 43 03 91     add        sp,sp,#0xd0
00102a2c ff 0f 5f d6     retab

• Right click on the mov x0,0x20 instruction and choose "Patch Instruction". Then change x0,0x20 to x0,#0x0.

• In Ghidra's CodeBrowser, choose File : Export Program. Then choose Format : Raw Bytes, and overwrite AVPBooter.vmapple2.bin.

• Use vbindiff AVPBooter.vmapple2.bin.org AVPBooter.vmapple2.bin to check your results. There should be just one change, to a four-byte value -- the length of one AARCH64 instruction.

iBoot Stage 1 (LLB.img4)

All the other modules that need patching are "wrapped" in img4 format. So to get at their actual content you need to use img4tool to unpack them. Then you'll patch them and rewrap them in new img4 format files.

• img4tool -e -p LLB.im4p.org LLB.img4.org

• img4tool -e -m LLB.im4m.org LLB.img4.org

• img4tool -e -o LLB.bin.org LLB.im4p.org

The iBoot Stage 1 module should now be in LLB.bin.org. Copy it to LLB.bin and patch it according to the instructions for the iBoot Stage 0 binary (AVPBooter.vmapple2.bin) above. If you've upgraded macOS in your guest VM from a version that you already patched, LLB.bin.org (and LLB.bin) may still contain your previous patch. In this case you don't need to repatch it, and may skip ahead to iBoot Stage 2.

• Run img4tool LLB.im4p.org, which will produce output something like what follows. Use the information from it to run the next command. The value for "desc" differs from one version of macOS to another. Note that you do not want to use compression, even though img4tool supports it. img4tool uses Apple's libcompression.dylib to implement its compression (and decompression). But, even though it's the same type ("bvx2"), it's incompatible with the decompression used by the iBoot binaries, which don't have access to any external modules.

img4tool version: 0.199-ed194718f9d6a035a432f2fdfe9fc639b72cba6c-RELEASE
Compiled with plist: YES
IM4P: ---------
type: illb
desc: iBoot-8422.141.2
size: 0x000369df

Compression: bvx2
Uncompressed size: 0x0006dc90
IM4P does not contain KBAG values

• img4tool -c LLB.im4p -t illb -d "iBoot-8422.141.2" LLB.bin

• img4tool -c LLB.img4 -p LLB.im4p -m LLB.im4m.org

The patched iBoot Stage 1 module should now be in LLB.img4.

iBoot Stage 2 (iBoot.img4)

Once again you'll need to unwrap the iBoot Stage 2 binary, patch it, and then rewrap it in img4 format. But this time the process is more complicated: iBoot.im4p.org contains a PAYP structure, tacked on to its end, that you'll need to append to iBoot.im4p by hand.

• img4tool -e -p iBoot.im4p.org iBoot.img4.org

• img4tool -e -m iBoot.im4m.org iBoot.img4.org

• img4tool -e -o iBoot.bin.org iBoot.im4p.org

Here's how to copy the PAYP structure from iBoot.im4p.org to a seperate file, which you'll later append to iBoot.im4p.

• openssl asn1parse -inform der -in iBoot.im4p.org -i -dump

This outputs the entire iBoot.im4p.org file in human-readable ASN1 format. The PAYP structure appears at the end, and looks something like this. Use the structure's offset (259719 in this case) in the next command.

259719:d=1  hl=2 l=  28 cons:  cont [ 0 ]        
259721:d=2  hl=2 l=  26 cons:   SEQUENCE          
259723:d=3  hl=2 l=   4 prim:    IA5STRING         :PAYP
259729:d=3  hl=2 l=  18 cons:    SET               
259731:d=4  hl=7 l=  11 cons:     priv [ 1768907638 ] 
259738:d=5  hl=2 l=   9 cons:      SEQUENCE          
259740:d=6  hl=2 l=   4 prim:       IA5STRING         :iocv
259746:d=6  hl=2 l=   1 prim:       INTEGER           :03

• xxd -p -s 259719 iBoot.im4p.org iBoot.payp.hex

iBoot.payp.hex is a hex dump. The following command will convert it to a binary (in DER format):

• xxd -p -r iBoot.payp.hex iBoot.payp.bin

Check the contents of iBoot.payp.bin by running the following command:

openssl asn1parse -inform der -in iBoot.payp.bin -i

Now copy iBoot.bin.org to iBoot.bin. Then patch out the call to validate_boot_object(). As mentioned above, you can do this instead of patching image4_validate_property_callback().

e5 03 04 aa 
04 00 80 52 

• In the Ghidra CodeBrowser, choose Search : For Instruction Patterns, then Edit Bytes and Input Mode Hex. Copy the above two lines of hexadecimal code and paste it into the Edit Bytes box, then choose Apply. Choose Search All and you should find one hit, with code that looks like the following:

                     FUN_700ac1fc                                    XREF[1]:     FUN_70063a78:70063fbc(c)  
700ac1fc e5 03 04 aa     mov        x5,x4
700ac200 04 00 80 52     mov        w4,#0x0
700ac204 01 00 00 14     b          LAB_700ac208

                     LAB_700ac208                                    XREF[1]:     700ac204(j)  
700ac208 7f 23 03 d5     pacibsp
700ac20c ff 03 03 d1     sub        sp,sp,#0xc0

• Double-click on the cross reference (FUN_70063a78:70063fbc(c) in this case). That should take you to code that looks like this:

70063fb0 e1 e3 0c 91     add        param_2,sp,#0x338
70063fb4 e4 43 03 91     add        param_5,sp,#0xd0
70063fb8 e3 03 16 aa     mov        param_4,x22
70063fbc 90 20 01 94     bl         FUN_700ac1fc                                     undefined FUN_700ac1fc()
70063fc0 a0 03 00 34     cbz        param_1,LAB_70064034
70063fc4 14 7b 00 51     sub        w20,w24,#0x1e
70063fc8 07 00 00 14     b          LAB_70063fe4

• In this example, right-click on the bl FUN_700ac1fc instruction and change it to mov x0,#0x0.

• On recent versions of macOS, there may be more than one cross reference. In this case you should double-click on each one, in turn, and change its target to mov x0,#0x0.

• Choose File : Export Program. Then choose Format : Raw Bytes, and overwrite iBoot.bin.

• Run img4tool iBoot.im4p.org, which will produce output something like this. Use the information from it to run the next command. The value of "desc" differs from one version of macOS to another.

img4tool version: 0.199-ed194718f9d6a035a432f2fdfe9fc639b72cba6c-RELEASE
Compiled with plist: YES
IM4P: ---------
type: ibot
desc: iBoot-8422.141.2
size: 0x0003f655

Compression: bvx2
Uncompressed size: 0x0007dde8
PAYP:
iocv: iocv: 3

IM4P does not contain KBAG values

• img4tool -c iBoot.im4p -t ibot -d "iBoot-8422.141.2" iBoot.bin

• dd if=iBoot.payp.bin >> iBoot.im4p

• Use Hex Fiend to open iBoot.im4p and correct its length value. Make sure File : Mode is set to Override.

• Observe iBoot.im4p's five-byte header and length value -- for example 308307DE0B. Convert the length (0x7DE0B) to decimal (515595) and add 30 (for the length of iBoot.payp.bin in this case). So the new length in this case is 515625 (== 0x7DE29).

• Correct the length value. The header in this case will now be 308307DE29. Save iBoot.im4p.

• img4tool -c iBoot.img4 -p iBoot.im4p -m iBoot.im4m.org

The patched Stage 2 module should now be in iBoot.img4.

The Kernel Cache (kernelcache)

As with the iBoot Stage 2 module, you'll need to unwrap the kernel cache, patch it, and rewrap it in img4 format.

• img4tool -e -p kernelcache.im4p.org kernelcache.org

• img4tool -e -m kernelcache.im4m.org kernelcache.org

• img4tool -e -o kernelcache.bin.org kernelcache.im4p.org

kernelcache.im4p.org, like iBoot.im4p.org, has a PAYP structure at its end. Copy the PAYP structure to a separate file, which you'll later append to kernelcache.im4p.

• openssl asn1parse -inform der -in kernelcache.im4p.org -i -dump

This outputs the entire kernelcache.im4p.org file in human-readable ASN1 format. The PAYP structure appears at the end, and looks something like this. Use the structure's offset (18030138 in this case) in the next command.

18030138:d=1  hl=3 l= 186 cons:  cont [ 0 ]        
18030141:d=2  hl=3 l= 183 cons:   SEQUENCE          
18030144:d=3  hl=2 l=   4 prim:    IA5STRING         :PAYP
18030150:d=3  hl=3 l= 174 cons:    SET               
18030153:d=4  hl=7 l=  19 cons:     priv [ 1801676144 ] 
18030160:d=5  hl=2 l=  17 cons:      SEQUENCE          
18030162:d=6  hl=2 l=   4 prim:       IA5STRING         :kcep
18030168:d=6  hl=2 l=   9 prim:       INTEGER           :FFFFFE0007B7D488
18030179:d=4  hl=7 l=  14 cons:     priv [ 1801677926 ] 
18030186:d=5  hl=2 l=  12 cons:      SEQUENCE          
18030188:d=6  hl=2 l=   4 prim:       IA5STRING         :kclf
18030194:d=6  hl=2 l=   4 prim:       INTEGER           :030E4000
18030200:d=4  hl=7 l=  19 cons:     priv [ 1801677935 ] 
18030207:d=5  hl=2 l=  17 cons:      SEQUENCE          
18030209:d=6  hl=2 l=   4 prim:       IA5STRING         :kclo
18030215:d=6  hl=2 l=   9 prim:       INTEGER           :FFFFFE0007004000
18030226:d=4  hl=7 l=  14 cons:     priv [ 1801677946 ] 
18030233:d=5  hl=2 l=  12 cons:      SEQUENCE          
18030235:d=6  hl=2 l=   4 prim:       IA5STRING         :kclz
18030241:d=6  hl=2 l=   4 prim:       INTEGER           :AF0000
18030247:d=4  hl=7 l=  11 cons:     priv [ 1801679462 ] 
18030254:d=5  hl=2 l=   9 cons:      SEQUENCE          
18030256:d=6  hl=2 l=   4 prim:       IA5STRING         :kcrf
18030262:d=6  hl=2 l=   1 prim:       INTEGER           :00
18030265:d=4  hl=7 l=  14 cons:     priv [ 1801679482 ] 
18030272:d=5  hl=2 l=  12 cons:      SEQUENCE          
18030274:d=6  hl=2 l=   4 prim:       IA5STRING         :kcrz
18030280:d=6  hl=2 l=   4 prim:       INTEGER           :02C64000
18030286:d=4  hl=7 l=  14 cons:     priv [ 1801680742 ] 
18030293:d=5  hl=2 l=  12 cons:      SEQUENCE          
18030295:d=6  hl=2 l=   4 prim:       IA5STRING         :kcwf
18030301:d=6  hl=2 l=   4 prim:       INTEGER           :02C64000
18030307:d=4  hl=7 l=  13 cons:     priv [ 1801680762 ] 
18030314:d=5  hl=2 l=  11 cons:      SEQUENCE          
18030316:d=6  hl=2 l=   4 prim:       IA5STRING         :kcwz
18030322:d=6  hl=2 l=   3 prim:       INTEGER           :480000

• xxd -p -s 18030138 kernelcache.im4p.org kernelcache.payp.hex

kernelcache.payp.hex is a hex dump. The following command will convert it to a binary (in DER format):

• xxd -p -r kernelcache.payp.hex kernelcache.payp.bin

Check the contents of kernelcache.payp.bin by running the following command:

openssl asn1parse -inform der -in kernelcache.payp.bin -i

Now copy kernelcache.bin.org to kernelcache.bin. Ghidra doesn't work well with kernel cache files, so I use Hopper Disassembler to patch kernelcache.bin.

• In Hopper choose Read Executable to Disassemble (CMD-SHIFT-O) and select kernelcache.bin. Then click on Loader, scroll down to com.apple.security.AppleVPBootPolicy and click OK.

• Select Labels, type "_validate_acm_context" and click on it in the list below. This should take you to the function, whose top few lines should look like this:

                     __validate_acm_context:
fffffe0008be57ec         pacibsp
fffffe0008be57f0         sub        sp, sp, #0x40
fffffe0008be57f4         stp        x20, x19, [sp, #0x20]
fffffe0008be57f8         stp        fp, lr, [sp, #0x30]
fffffe0008be57fc         add        fp, sp, #0x30
fffffe0008be5800         sturb      wzr, [fp, var_11]
fffffe0008be5804         cbz        w0, loc_fffffe0008be5868

• You want to find the calls to _validate_acm_context() from _command_create_linked_manifest() and _command_update_local_policy_for_kcos(), and patch each one out by replacing it with a nop instruction.

• Click on _validate_acm_context()'s first line (pacibsp), then press the "x" key. This should open a dialog listing all the calls to _validate_acm_context() from elsewhere in AppleVPBootPolicy. Find _command_create_linked_manifest() and _command_update_local_policy_for_kcos() in this list, and click on each of them in turn. The calls to _validate_acm_context() should look something like this:

fffffe0008bd4e74         add        x22, x21, #0x66
fffffe0008bd4e78         mov        w0, #0x1
fffffe0008bd4e7c         mov        x1, x22
fffffe0008bd4e80         bl         __validate_acm_context
fffffe0008bd4e84         tbnz       w0, 0x0, loc_fffffe0008bd4ea8

• Select the call to _validate_acm_context() (bl __validate_acm_context) and change to Hexadecimal mode (from ASM mode). For each of the following four hex values, double-click on it and replace it with values from the following list (binary code for a nop instruction). (Hit Enter to update each value after you've typed in its replacement.)

1f 20 03 d5

• Repeat the previous two steps until you've patched the calls to _validate_acm_context() from both _command_create_linked_manifest() and _command_update_local_policy_for_kcos().

• Choose File : New Executable and overwrite kernelcache.bin

• Use vbindiff kernelcache.bin.org kernelcache.bin to check your results. There should be just two changes, each to a four-byte value -- the length of a single AARCH64 instruction.

• Run img4tool kernelcache.im4p.org, which will produce output something like this. Use the information from it to run the next command. The value of "desc" differs from one version of macOS to another. Note that the current version of img4tool can't yet deal with kernelcache.im4p.org's PAYP structure.

img4tool version: 0.199-ed194718f9d6a035a432f2fdfe9fc639b72cba6c-RELEASE
Compiled with plist: YES
IM4P: ---------
type: krnl
desc: KernelManagement_host-354.140.3
size: 0x01131df6

Compression: bvx2
Uncompressed size: 0x03bd4000
PAYP:
kcep: kcep: [Error] img4tool: failed with exception:
[exception]:
what=assure failed
code=15597586
line=238
file=ASN1DERElement.cpp
commit count=199
commit sha  =ed194718f9d6a035a432f2fdfe9fc639b72cba6c

• img4tool -c kernelcache.im4p -t krnl -d "KernelManagement_host-354.140.3" kernelcache.bin

• dd if=kernelcache.payp.bin >> kernelcache.im4p

• Use Hex Fiend to open kernelcache.im4p and correct its length value. Make sure File : Mode is set to Override.

• Observe kernelcache.im4p's six-byte header and length value -- for example 308403BD4033. Convert the length (0x03BD4033) to decimal (62734387) and add 189 (for the length of kernelcache.payp.bin in this case). So the new length in this case is 62734576 (== 0x3BD40F0).

• Correct the length value. The header in this case will now be 308403BD40F0. Save kernelcache.im4p.

• img4tool -c kernelcache -p kernelcache.im4p -m kernelcache.im4m.org

The patched kernel cache should now be in kernelcache.

Copying the Patched Modules to their Final Destinations

iBoot Stage 0 (AVPBooter.vmapple2.bin)

AVPBooter.vmapple2.bin is a system file. By default it's protected by the macOS file system, and can't be changed. To get around this I borrow from How to Defang macOS System Protections. Before you follow these steps, you must make the settings changes I described above under Settings Changes.

• mkdir /tmp/mount

• Run mount at a Terminal prompt and observe its results, for example as follows. Use the contents of the first line in the next command.

/dev/disk5s1s1 on / (apfs, sealed, local, read-only, journaled)
devfs on /dev (devfs, local, nobrowse)
/dev/disk5s6 on /System/Volumes/VM (apfs, local, noexec, journaled, noatime, nobrowse)
/dev/disk5s2 on /System/Volumes/Preboot (apfs, local, journaled, nobrowse)
/dev/disk5s4 on /System/Volumes/Update (apfs, local, journaled, nobrowse)
/dev/disk1s2 on /System/Volumes/xarts (apfs, local, noexec, journaled, noatime, nobrowse)
/dev/disk1s1 on /System/Volumes/iSCPreboot (apfs, local, journaled, nobrowse)
/dev/disk1s3 on /System/Volumes/Hardware (apfs, local, journaled, nobrowse)
/dev/disk5s5 on /System/Volumes/Data (apfs, local, journaled, nobrowse, protect)
/dev/disk2s3 on /Volumes/Boot3 (apfs, sealed, local, read-only, journaled)
/dev/disk7s3 on /Volumes/Boot2 (apfs, sealed, local, read-only, journaled)
/dev/disk4s1 on /Volumes/Boot4 - Data (apfs, local, journaled, nobrowse, protect)
/dev/disk2s1 on /Volumes/Boot3 - Data (apfs, local, journaled, nobrowse, protect)
/dev/disk4s3 on /Volumes/Boot4 (apfs, sealed, local, read-only, journaled)
/dev/disk6s1 on /Volumes/Boot1 - Data (apfs, local, journaled, nobrowse, protect)
/dev/disk6s3 on /Volumes/Boot1 (apfs, sealed, local, read-only, journaled)
/dev/disk7s1 on /Volumes/Boot2 - Data (apfs, local, journaled, nobrowse, protect)
map auto_home on /System/Volumes/Data/home (autofs, automounted, nobrowse)

• sudo mount -o nobrowse -t apfs /dev/disk5s1 /tmp/mount

• cd /tmp/mount/System/Library/Frameworks/Virtualization.framework/Resources

• sudo cp /path/to/patched/AVPBooter.vmapple2.bin .

• sudo bless --mount /tmp/mount --bootefi --create-snapshot

• Reboot your host computer.

On reboot, check the contents of /System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin to make sure it's what you expect.

You need to keep your host computer in this state as long as you're using third party kernel extensions in your macOS guest VM. (Without the patched AVPBooter.vmapple2.bin it will simply refuse to start.)

If need be, use the following command to revert your snapshot (and AVPBooter.vmapple2.bin), then reboot your host computer:

sudo bless --mount / --last-sealed-snapshot

iBoot Stage 1 (LLB.img4)

You'll need to copy both LLB.img4 and logo.img4.org over their original contents in AuxiliaryStorage. This is because LLB.img4 might be a different length than LLB.img4.org, and Apple's virtualization infrastructure expects the logo image to immediately follow the LLB image. You can skip this section if you discovered above that your existing LLB image already contains a patch (applied to an earlier version of macOS in your guest VM).

• Use Hex Fiend to open LLB.img4, logo.img4.org and AuxiliaryStorage. Make sure Edit : Mode for AuxilaryStorage is Overwrite.

• Jump to offset 0x224000 in AuxiliaryStorage. If an LLB image exists at this location, its first two bytes should be 3083. If not, jump to offset 0x24000.

• Copy the contents of LLB.img4 (CMD-A, CMD-C) and paste them into AuxilaryStorage.

• Copy the contents of logo.img4.org and paste them into AuxiliaryStorage, then save the file.

iBoot Stage 2 (iBoot.img4) and the Kernel Cache (kernelcache)

Copy these files to your macOS guest VM and do the following there:

• Run kmutil inspect, and observe where the boot kernel cache exists in the VM's file system. The following is an example, which will get used in the following steps. The path's exact contents will differ from case to case.

/System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache

• Note the "Next Stage Image4 Hash" just before /System/Library/Caches -- in this case FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83.

• sudo cp kernelcache /System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache

• Run cd / and sudo find . -name iBoot.img4 -exec ls -al \{\} \;. Observe the hit whose path contains the Next Stage Image4 Hash. Use the results in the following command. For example:

• sudo cp iBoot.img4 /System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4

Shut down your your macOS guest VM and reboot it into Recovery Mode. Then perform the following steps. You'll be looking for the same Next Stage Image4 Hash as in the previous steps.

Run Terminal in Recovery Mode, then do the following:

• Run cd / and find . -name iBoot.img4 -exec ls -al \{\} \;. Expect two hits this time, which should look like the following:

/System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4
/System/Volumes/Data/private/tmp/Recovery/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4

• Copy iBoot.img4 over both of them.

You may also see another hit that looks like the following. Ignore it. You can tell by its date and file size that you already copied the patched iBoot.img4 over it above.

/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4

• Run cd / and find . -name kernelcache -exec ls -al \{\} \;. Once again expect two hits, which should look like the following (maybe plus one superfluous hit, as above).

/System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache
/System/Volumes/Data/private/tmp/Recovery/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache

• Copy kernelcache over both of them.

• Reboot your macOS guest VM and start playing with third-party kernel extensions on it!

can't say how impressive is this. thank you.

You're most welcome!

This is absolutely amazing work, thank you so much!

This is absolutely amazing work, thank you so much!

You're very welcome!

there is a unsigned apple extension for big sur from a DTK image for Intel TGL graphics - AppleIntelTGLGraphicsFramebuffer.kext - (https://www.insanelymac.com/forum/topic/358305-80-solved-iris-xe-igpu-on-tiger-lake-successfully-loaded-icllp-frambuffer-and-vram-also-recognizes-1536mb-however-some-issues/page/13/). Do you think this can be loaded somehow on real hardware ?

there is a unsigned apple extension for big sur from a DTK image for Intel TGL graphics - AppleIntelTGLGraphicsFramebuffer.kext - (https://www.insanelymac.com/forum/topic/358305-80-solved-iris-xe-igpu-on-tiger-lake-successfully-loaded-icllp-frambuffer-and-vram-also-recognizes-1536mb-however-some-issues/page/13/). Do you think this can be loaded somehow on real hardware ?

I've no idea. You can try. Your question is off topic. My document concerns loading third party kexts in Virtualization Framework guest VMs on Apple Silicon hardware.

Something changed in macOS 15 GM. My colleague confirms he can use the App Store. Unclear what else they may have fixed. https://developer.apple.com/documentation/virtualization/using_icloud_with_macos_virtual_machines

Something changed in macOS 15 GM. My colleague confirms he can use the App Store. Unclear what else they may have fixed. https://developer.apple.com/documentation/virtualization/using_icloud_with_macos_virtual_machines

I can use iCloud but not the App Store with macOS 15 (RC) on the host and in a utmapp VM. The iCloud settings panel has a note indicating that iCloud within a VM cannot be used to access Apple Services.

Something changed in macOS 15 GM. My colleague confirms he can use the App Store. Unclear what else they may have fixed. https://developer.apple.com/documentation/virtualization/using_icloud_with_macos_virtual_machines

Nope, same old nonsense. Out of the box, third party kexts still don't load in an Apple Virtualization framework VM. I tested in a macOS 15.0 RC guest on a macOS 15.0 RC host (build 24A335), using UTM 4.5.4 (the current version).

I expect this document's instructions will still work, without changes. I'll test that over the next day or two.

I expect this document's instructions will still work, without changes.

They do. I'll be waiting for the release on September 16th, but I don't expect things to change in this respect.

I hope the iCloud limitations will disappear in guest VMs running the release version of macOS 15.0. As I remember it, they weren't present in previous releases. We shall see.

Has anyone tested this on macOS 15.1 B3? I thought the instructions are pretty straightforward and easy todo but I end up with nothing changed. I am using VirtualBuddy instead of UTM but that should not make a difference. So after I apply all patches everything boots as normal but it will still complain on every boot that the Aux cache needs to be redone.

I just tried my instructions on macOS 15.1 Beta3 (build 24B5035e), and failed very early -- I couldn't get the "defanging" to work. I'll look into this, but it'll probably be a while before I figure it out.

In light of this, @pandora-SG, I have no idea what you were doing, or how you (apparently) got so far.

Oops, nevermind. I forgot to do csrutil disable and csrutil authenticated-root disable on the host. I'll keep going and report back in an hour or so.

I am not sure what you mean by "defanging". I simply patched all the files that you mentioned. With the only issue being that the replacements inside the VM seemed to have one less hit than you seemed to have. And after I put everything in place I got the same GUI errors you normally get as if no patch was there. I am currently putting all of this into a script that I can easily do all of this automatic for a new version. Over the weekend I will also try my luck to add the GDB stub to virtualbuddy.

OK, I just followed my own instructions to set up a macOS 15.1 Beta 3 guest, running on a macOS 15.1 Beta 3 host, to be able to load third party kernel extensions. I had no problems at all. My instructions work as is.

So. The first thing to try is to use UTM instead of VirtualBuddy, just to see what happens. There's also conceivably a problem with your kext. Have you tried my instructions on some other version of macOS (host and guest)? The kext I've been testing with is very simple -- basically just a hello world kext. I tried attaching its source as a 'tgz' archive, but github refused to allow it.

Edit: My KernelLogging kext is a close equivalent of the one I've been testing with. You could try testing with it. To get it to load, you'll need to comment out the line #define FAIL_START 1 in KernelLogging.c.

Just a heads up. I automated most of this and then tried it again with a macOS 15.1 Beta 3 (guest+host) and VirtualBuddy. It works now. I use the same KEXT that I used before. I am puzzled what went wrong in the other VM.

Glad to hear it. Just now I tried with VirtualBuddy and had no problems.

You probably just made a mistake. The instructions are long and complex. I considered automating (scripting) some of them. But there's no way you can script them all. And I decided it'd be better to keep it all by hand. That way it's easier to interleave detailed explanatory comments.

Over the weekend I will also try my luck to add the GDB stub to virtualbuddy.

Out of curiosity, do you mean the one in the Virtualization framework? The one that uses @interface _VZGDBDebugStubConfiguration : NSObject <NSCopying>?

Yeah I mean the private stuff in the Virtualization framework. I saw it in action a while ago but no code available. I am completely new to the virtualization framework so I thought it is a good task to add support for it in VirtualBuddy which I use most.

Aside from that I am also curious about the macOS 15 feature "research guests" - like there is csrutil allow-research-guests enable and there is some AVPBooter.research1 file in the framework.

Thanks for the info. As I mentioned above, I've written a patch for UTM that includes support for the Virtualization framework GDB stub. I still haven't published it, more out of laziness than anything else. It works reasonably well, and I relied on it for part of this document. But using it requires setting the amfi_get_out_of_my_way=1 boot arg in the host, which makes it inconvenient to use. UTM still works fine, but no browser will function.

I hadn't noticed the "research guests" feature. I'll look into it later.

Edit: I might have found out what "research guests" are: "Security Research Devices". Though I suppose they aren't iPhones. It might be possible for Apple to hack a Mac to be a research device in the same way that they hack an iPhone.

Thanks for this guide, it was super helpful. I was able to follow all the steps, and get to the end with a booting VM on MacOS 15.1 host and guest. I'm still running into issues loading kexts though. When I attempt a kextload the VM says that I need to approve the developer and restart. Each time I do same dialogue...

I'm wondering if I need to rebuild the kc and modify it after loading each 3rd party kext? Or should I only need to modify once and then load kexts? Maybe I'm not loading them properly?

EDIT: I'm also noticing a couple of things after the final step of copying the kernelcache and iBoot.img4 files over the originals in recovery.

After doing that and rebooting, I get prompted to verify the startup disk. I enter the password, and it boots back with SIP re-enabled for me automagically. I also see the file size of both kernelcache and iBoot.img4 are much smaller (closer to the originals VS the ones I copied over). Its like it is putting back the originals, and forcing SIP back on for me after modifying those files.

I haven't yet tested this document on the macOS 15.1 release. But it seems unlikely Apple would have made changes in a point release that broke this document. So it's most likely you're making some kind of mistake. I have no idea what kind.

In the next day or two I'll try things myself on the macOS 15.1 release, and report back with my results.

I just tested this document with the macOS 15.1 release as both host and guest. I had no problems. I tested with UTM 4.5.4 (the current version).

I suggest completely restoring your original settings and starting again from scratch. I also suggest that you copy iBoot.img4 and kernelcache to their destinations "separately". In other words: 1) Boot your guest "normally"; 2) Copy iBoot.img4 to its destination as per my instructions; 3) Reboot your guest into Recovery mode; 4) Copy iBoot.img4 to its destinations as per my instructions; 5) Reboot your guest into its "normal" mode. If step 5 works (if you don't get prompted to verify the startup disk), you know everything's been done correctly to this point. Now try steps 1 through 5 over again with kernelcache.

If step 5 in the iBoot.img4 series doesn't work, you've probably made a mistake with the LLB.img4 image (either creating it or copying it to its final destination in the AuxiliaryStorage file).