Running Third Party Kernel Extensions on Virtualization Framework macOS Guest VMs

As of macOS 12 (Monterey), Apple's Virtualization framework has nice support for macOS guest virtual machines, but with severe limitations: For example you can't install a macOS guest on Intel Macs, install guests with newer versions of macOS than the host, copy and paste between the host and the guest, or install third party kernel extensions in the guest. As usual for Apple, the functionality they do support is nicely implemented, but they've left out so much that the result is only marginally useful -- at least compared to third-party implementations available on Intel Macs, like VMware Fusion and Parallels.

I've been working on the last of these limitations, and have found a workaround. It's rather complex, but should be very useful for those developing kernel extensions for macOS on Apple Silicon, or (more likely) porting existing Intel kexts to Apple Silicon. Developing kernel extensions on bare metal is a pain -- not least because you might end up damaging your nice new Apple Silicon Mac. But doing it in a virtual machine isolates the potential damage to the VM itself -- at worst you'll only need to trash it and create a replacement.

Parts of my workaround are interesting in themselves, even if you don't use them to run third party kernel extensions on macOS guest VMs. For example I've found a way to patch the VM's kernel cache (containing the kernel and built-in kernel extensions), which should be very useful in reverse engineering macOS.

Most of the discoveries here are ones I've made on my own, by trial and error. But I had a very helpful starting point -- NyanSatan's Virtual-iBoot-Fun project.

Prerequisites

An Apple Silicon Mac running macOS 12 or higher. Mine is a 2020 Mac Mini (Macmini9,1).
A decent virtual machine host that uses Apple's Virtualization framework. I use UTM.
One or more decent disassemblers. I use Ghidra and Hopper Disassembler. I also installed Nick Botticelli's ghidra-iboot plugin
Tihmstar's img4tool.
A decent hex editor. I use Hex Fiend. I change its default edit mode to "Overwrite" and its line number format to "Hexadecimal".
A binary diff tool. I use VBinDiff.
A calculator with support for radix modes. Apple's Calculator in Programmer mode will do.

Background

My workaround patches three iBoot modules and the VM's kernel cache.

The iBoot modules load early in the macOS (and IOS) boot sequence, before the kernel. There are at least three variants of iBoot, used in three diffent "stages" -- Stage 0 (as I call it), Stage 1 and Stage 2. On bare metal, Stage 0 is implemented in hardware (in Apple's "Secure Enclave"). Stage 1 (aka LLB) and Stage 2 (iBoot properly so called) are implemented in software, though only the Stage 2 iBoot exists (inside iBoot.img4) in the macOS file system.

By design, the Secure Enclave Processor (SEP) is inaccessible to ordinary mortals. And on bare metal the LLB and iBoot modules are encrypted (using a key held in the SEP). But for a VM, the modules for all three stages are unencrypted. Stage 0 is implemented (on the host) in /System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin. Stage 1 is embedded (in img4 format) in an AuxiliaryStorage file (on the host) associated with the DMG file (with an img extension) that stores the VM's image. Stage 2 (inside iBoot.img4) exists in the VM's file system.

None of the iBoot variants call any external functions, and none have any symbols. So they're difficult to decipher. But the fact that the Virtualization framework uses them without encryption is a golden opportunity to reverse engineer them, and understand them better.

It's apparently common knowledge that the iBoot modules all contain a function to check the "digests" (DGST) of the many img4 images that govern the macOS boot process (including LLB, iBoot.img4 and the kernel cache itself). Each "digest" is a hash, used to ensure that none of these images has changed since they were "signed" by Apple. The source code for a similar function (image4_validate_property_callback()) is available here.

The images in question are those to be loaded at the next stage. So (among other things) Stage 0 iBoot checks the digest of the Stage 1 iBoot img4 image, Stage 1 iBoot checks the digest of the Stage 2 iBoot img4 file, and Stage 2 iBoot checks the digest of the kernelcache file. Aside from the file system itself, nothing checks the integrity of the stage 0 iBoot file (AVPBooter.vmapple2.bin). So all you need to stop these integrity checks is to defeat the file system's protection and patch this function (call it image4_validate_property_callback()) at every "stage".

As NyanSatan mentions, the standard approach is to patch out image4_validate_property_callback() completely -- to replace it with code that "returns 0". I find this causes problems, so my patch is more surgical -- I allow image4_validate_property_callback() to perform all its normal checks, and only patch its return value (to '0') just before it returns.

Some time ago I discovered that failed calls to _validate_acm_context() in the VM's AppleVPBootPolicy.kext stop the VM from creating an auxiliary kext cache. Since third-party kexts are stored here, this prevents them from loading in a Virtualization framework VM. The failed calls originate from two other functions in AppleVPBootPolicy.kext -- _command_create_linked_manifest() and _command_update_local_policy_for_kcos(). My workaround is to patch out the calls to _validate_acm_context() from both of these functions. This is a more "surgical" approach than patching out _validate_acm_context() itself, which is called from many additional functions in AppleVPBootPolicy.kext.

But this isn't enough by itself. An auxiliary kext cache does get created (you can see it using kmutil inspect). But macOS still complains that your third-party kext needs to be rebuilt. To fix this you need to go back to the Stage 2 iBoot module (inside iBoot.img4). One of its purposes is to fill the "device tree" with all the appropriate devices. But it (or the variant available in a Virtualization framework VM) refuses to allow any "AuxKC" entries to be added to chosen/memory-map in a VM, even when the auxiliary kext cache is present. Fixing this requires finding the function that checks whether a given boot object (originally in img4 format) should be processed, and patching out the call to it. Its first parameter is a four-character code -- for example illb, ibot, krnl or auxk. validate_boot_object() (my name for this function) indirectly calls image4_validate_property_callback(). So if you patch out the former you don't need to patch the latter. At some point I'll describe how I found validate_boot_object().

To display the contents of chosen/memory-map, run the following command in a Terminal prompt:

ioreg -p IODeviceTree -n "memory-map" -w 0 -r -t

Settings Changes (Host and Guest)

Some settings changes are required on the host by How to Defang macOS System Protections, which allows changes to AVPBooter.vmapple2.bin.

In System Settings, under Software Update, disable "Download new updates when available". Otherwise your host can become unbootable.
Boot into Recovery Mode, run Terminal and do the following:

csrutil disable
csrutil authenticated-root disable

You also, of course, need to create a macOS guest VM. Make sure it uses the Virtualization framework. In UTM this is accomplished by choosing "Virtualize" and then "macOS 12+".

Terminal will be used heavily below. So, to avoid lots of annoying prompts to give it permission to access files, it's best to give the Terminal app "full disk access" on the guest (in Privacy and Security in System Settings).

Then boot into Recovery Mode on the guest. On macOS 12 you'll need to use my hack to do this.

Run the Startup Security Utility. Choose Reduced Security and "Allow user management of kernel extensions from identified developers".
Run Terminal and do the following:
csrutil disable, then y to "Allow booting unsigned operating systems and any kernel extensions".

Special Considerations Once You've Made the Changes Described Here

Below you'll create (and boot from) an APFS snapshot on your host computer to accomodate changes to the Stage 0 iBoot module (AVMBooter.vmapple2.bin). You need to keep your host computer in this state as long as you're using third party kernel extensions in your macOS guest VM. (Without the patched AVPBooter.vmapple2.bin it will simply refuse to start.) You also need to keep csrutil's "authenticated-root" disabled, and keep "Download new updates when available" disabled under Software Update. But don't use Software Update to update macOS on your host while it's booted from this custom snapshot. I don't know that this will cause trouble, but I expect it might.

Use the following command to revert your snapshot (and AVPBooter.vmapple2.bin), then reboot your computer:

sudo bless --mount / --last-sealed-snapshot

Release version macOS 13 and 14 guest VMs can be upgraded to newer versions of macOS. But before you do so you should revert this document's changes to the Stage 2 iBoot module (iBoot.img4) and the kernel cache (kernelcache): Boot into Recovery Mode, run the Startup Security Utility, and choose Full Security. Afterwards you'll need to go through all the following steps again, from scratch. Among other things you'll need to create new iBoot.img4.org and kernelcache.org files, and possibly also a new LLB.img4.org file.

macOS 12 guests can't be upgraded. The Virtualization process (com.apple.Virtualization.VirtualMachine) crashes and your VM becomes unbootable. This is an Apple bug. I've seen it myself, and have seen it documented here. The same is true for beta-version macOS 14 guests, though the consequences are less severe.

In this case your only option is to create a new guest VM directly from an IPSW file, then work through the following steps.

Finding the Modules to be Patched

iBoot Stage 0 (`AVPBooter.vmapple2.bin`)

This one's on the host, and is easy to find. Copy it and rename the copy to AVPBooter.vmapple2.bin.org.

/System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin

iBoot Stage 1 (LLB)

This one's also on the host, but is harder to find. By default, VMs created by UTM are stored in the following directory:

~/Library/Containers/com.utmapp.UTM/Data/Documents

Each VM is stored in a "package" with the extension .utm. Inside the package, in its Data subdirectory, is a file named AuxiliaryStorage. LLB is embedded here, in img4 format. It's followed immediately by an img4 format image of the "logo". You'll need to get copies of both.

If you've upgraded your guest VM to a newer version of macOS, there will be two copies of the LLB image in the AuxiliaryStorage file, each followed by a "logo" image. Only the second set will be active. So that's the one you'll need to work on. And if you've already patched a previous version of macOS in your guest VM (following the instructions in this document), the second LLB image may still contain your earlier patch. If so, you won't need to repatch it.

The first LLB image is always at offset 0x24000. The second, if it exists, should be at offset 0x224000. To be sure you've found all the LLB images, and their correct locations, search (in your hex editor) on "illb" (the four character code for the LLB image).

Open AuxiliaryStorage in a hex editor. I use Hex Fiend, and will tailor my steps to it specifically.
Jump to offset 0x224000. If an LLB image exists at this location, its first two bytes should be 3083. If not, jump to offset 0x24000.
The img4 image is in DER format (used by ASN1), so the first five bytes are 3083 followed by a three digit hexadecimal number in big endian format (for example 037FD1). This number is the length of the image, exclusive of the length of its header. So the total length is 0x37FD6.
Select the first five bytes of the image, then choose "Extend Selection". In this case, you'd extend it by 0x37FD1 bytes. Scroll down to the end of the selection (without disturbing it) and check that it's in the correct location (just before the "logo" image). If it is, CMD-C to copy the image, CMD-N to open a new window, CMD-V to paste in its contents, and save the file as LLB.img4.org.
The header for the "logo" image should be 3082 followed by a two digit hexdecimal number (for example 36DA). Select the first four bytes, extend the selection by 0x36DA bytes and scroll down to its end. The following bytes should be a bunch of NULLs (00). CMD-C to copy the image, CMD-N to open another new window, CMD-V to paste in its contents, and save this second new file as logo.img4.org.

iBoot Stage 2 (`iBoot.img4`) and the Kernel Cache (`kernelcache`)

These modules are stored in files on the VM. So run the VM and do the following in it, at a Terminal prompt:

Run kmutil inspect, and observe where the boot kernel cache exists in the VM's file system. The following is an example, which will get used in the following steps. The path's exact contents will differ from case to case.

/System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache

Note the long hexadecimal number just before /System/Library/Caches -- in this case FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83. This is the "Next Stage Image4 Hash (nsih)", observable in the output of sudo bputil -d.
Copy this file and rename it to kernelcache.org.
cd / and sudo find . -name iBoot.img4 -exec ls -al \{\} \;. There will be at least two hits. Choose the one whose path contains the Next Stage Image4 Hash. Copy it and rename the copy to iBoot.img4.org. You might find the original iBoot.img4 here, for example:

/System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4

Patching These Modules

iBoot Stage 0 (`AVPBooter.vmapple2.bin.org`)

For this I use Ghidra with the ghidra-iboot plugin. Copy AVPBooter.vmapple2.bin.org to AVPBooter.vmapple2.bin. Then run Ghidra and create a new project. Then:

Run Ghidra's CodeBrowser and import AVPBooter.vmapple2.bin.
Choose Search : Program Text : All Fields. Type "0x4447" ('DG' of DGST) in "Search for", then choose "Search All". You should find two hits, both in the same function. Click on one to move the cursor to its location.
Once again choose Search : Program Text : All Fields, and type "retab" in "Search For". Then choose "Next". Now you should see the code that runs just before the function returns. It will look something like as follows. In this example, you want to change the instruction at address 0x00102a0c (mov x0,x20) to mov x0,#0x0. (AARCH64 machine code uses the X0 register to store a return value.)

001029f4 a8 83 5a f8     ldur       x8,[x29, #local_68]
001029f8 29 f9 37 d0     adrp       x9,0x70028000
001029fc 1f 20 03 d5     nop
00102a00 29 a1 40 f9     ldr        x9,[x9, #0x140]=>DAT_70028140
00102a04 3f 01 08 eb     cmp        x9,x8
00102a08 21 06 00 54     b.ne       LAB_00102acc
00102a0c e0 03 14 aa     mov        x0,x20
00102a10 fd 7b 4c a9     ldp        x29=>local_10,x30,[sp, #0xc0]
00102a14 f4 4f 4b a9     ldp        x20,x19,[sp, #local_20]
00102a18 f6 57 4a a9     ldp        x22,x21,[sp, #local_30]
00102a1c f8 5f 49 a9     ldp        x24,x23,[sp, #local_40]
00102a20 fa 67 48 a9     ldp        x26,x25,[sp, #local_50]
00102a24 fc 6f 47 a9     ldp        x28,x27,[sp, #local_60]
00102a28 ff 43 03 91     add        sp,sp,#0xd0
00102a2c ff 0f 5f d6     retab

Right click on the mov x0,0x20 instruction and choose "Patch Instruction". Then change x0,0x20 to x0,#0x0.
In Ghidra's CodeBrowser, choose File : Export Program. Then choose Format : Raw Bytes, and overwrite AVPBooter.vmapple2.bin.
Use vbindiff AVPBooter.vmapple2.bin.org AVPBooter.vmapple2.bin to check your results. There should be just one change, to a four-byte value -- the length of one AARCH64 instruction.

iBoot Stage 1 (`LLB.img4`)

All the other modules that need patching are "wrapped" in img4 format. So to get at their actual content you need to use img4tool to unpack them. Then you'll patch them and rewrap them in new img4 format files.

img4tool -e -p LLB.im4p.org LLB.img4.org
img4tool -e -m LLB.im4m.org LLB.img4.org
img4tool -e -o LLB.bin.org LLB.im4p.org

The iBoot Stage 1 module should now be in LLB.bin.org. Copy it to LLB.bin and patch it according to the instructions for the iBoot Stage 0 binary (AVPBooter.vmapple2.bin) above. If you've upgraded macOS in your guest VM from a version that you already patched, LLB.bin.org (and LLB.bin) may still contain your previous patch. In this case you don't need to repatch it, and may skip ahead to iBoot Stage 2.

Run img4tool LLB.im4p.org, which will produce output something like what follows. Use the information from it to run the next command. The value for "desc" differs from one version of macOS to another. Note that you do not want to use compression, even though img4tool supports it. img4tool uses Apple's libcompression.dylib to implement its compression (and decompression). But, even though it's the same type ("bvx2"), it's incompatible with the decompression used by the iBoot binaries, which don't have access to any external modules.

img4tool version: 0.199-ed194718f9d6a035a432f2fdfe9fc639b72cba6c-RELEASE
Compiled with plist: YES
IM4P: ---------
type: illb
desc: iBoot-8422.141.2
size: 0x000369df

Compression: bvx2
Uncompressed size: 0x0006dc90
IM4P does not contain KBAG values

img4tool -c LLB.im4p -t illb -d "iBoot-8422.141.2" LLB.bin
img4tool -c LLB.img4 -p LLB.im4p -m LLB.im4m.org

The patched iBoot Stage 1 module should now be in LLB.img4.

iBoot Stage 2 (`iBoot.img4`)

Once again you'll need to unwrap the iBoot Stage 2 binary, patch it, and then rewrap it in img4 format. But this time the process is more complicated: iBoot.im4p.org contains a PAYP structure, tacked on to its end, that you'll need to append to iBoot.im4p by hand.

img4tool -e -p iBoot.im4p.org iBoot.img4.org
img4tool -e -m iBoot.im4m.org iBoot.img4.org
img4tool -e -o iBoot.bin.org iBoot.im4p.org

Here's how to copy the PAYP structure from iBoot.im4p.org to a seperate file, which you'll later append to iBoot.im4p.

openssl asn1parse -inform der -in iBoot.im4p.org -i -dump

This outputs the entire iBoot.im4p.org file in human-readable ASN1 format. The PAYP structure appears at the end, and looks something like this. Use the structure's offset (259719 in this case) in the next command.

259719:d=1  hl=2 l=  28 cons:  cont [ 0 ]        
259721:d=2  hl=2 l=  26 cons:   SEQUENCE          
259723:d=3  hl=2 l=   4 prim:    IA5STRING         :PAYP
259729:d=3  hl=2 l=  18 cons:    SET               
259731:d=4  hl=7 l=  11 cons:     priv [ 1768907638 ] 
259738:d=5  hl=2 l=   9 cons:      SEQUENCE          
259740:d=6  hl=2 l=   4 prim:       IA5STRING         :iocv
259746:d=6  hl=2 l=   1 prim:       INTEGER           :03

xxd -p -s 259719 iBoot.im4p.org iBoot.payp.hex

iBoot.payp.hex is a hex dump. The following command will convert it to a binary (in DER format):

xxd -p -r iBoot.payp.hex iBoot.payp.bin

Check the contents of iBoot.payp.bin by running the following command:

openssl asn1parse -inform der -in iBoot.payp.bin -i

Now copy iBoot.bin.org to iBoot.bin. Then patch out the call to validate_boot_object(). As mentioned above, you can do this instead of patching image4_validate_property_callback().

e5 03 04 aa 
04 00 80 52

In the Ghidra CodeBrowser, choose Search : For Instruction Patterns, then Edit Bytes and Input Mode Hex. Copy the above two lines of hexadecimal code and paste it into the Edit Bytes box, then choose Apply. Choose Search All and you should find one hit, with code that looks like the following:

                     FUN_700ac1fc                                    XREF[1]:     FUN_70063a78:70063fbc(c)  
700ac1fc e5 03 04 aa     mov        x5,x4
700ac200 04 00 80 52     mov        w4,#0x0
700ac204 01 00 00 14     b          LAB_700ac208

                     LAB_700ac208                                    XREF[1]:     700ac204(j)  
700ac208 7f 23 03 d5     pacibsp
700ac20c ff 03 03 d1     sub        sp,sp,#0xc0

Double-click on the cross reference (FUN_70063a78:70063fbc(c) in this case). That should take you to code that looks like this:

70063fb0 e1 e3 0c 91     add        param_2,sp,#0x338
70063fb4 e4 43 03 91     add        param_5,sp,#0xd0
70063fb8 e3 03 16 aa     mov        param_4,x22
70063fbc 90 20 01 94     bl         FUN_700ac1fc                                     undefined FUN_700ac1fc()
70063fc0 a0 03 00 34     cbz        param_1,LAB_70064034
70063fc4 14 7b 00 51     sub        w20,w24,#0x1e
70063fc8 07 00 00 14     b          LAB_70063fe4

In this example, right-click on the bl FUN_700ac1fc instruction and change it to mov x0,#0x0.
On recent versions of macOS, there may be more than one cross reference. In this case you should double-click on each one, in turn, and change its target to mov x0,#0x0.
Choose File : Export Program. Then choose Format : Raw Bytes, and overwrite iBoot.bin.
Run img4tool iBoot.im4p.org, which will produce output something like this. Use the information from it to run the next command. The value of "desc" differs from one version of macOS to another.

img4tool version: 0.199-ed194718f9d6a035a432f2fdfe9fc639b72cba6c-RELEASE
Compiled with plist: YES
IM4P: ---------
type: ibot
desc: iBoot-8422.141.2
size: 0x0003f655

Compression: bvx2
Uncompressed size: 0x0007dde8
PAYP:
iocv: iocv: 3

IM4P does not contain KBAG values

img4tool -c iBoot.im4p -t ibot -d "iBoot-8422.141.2" iBoot.bin
dd if=iBoot.payp.bin >> iBoot.im4p
Use Hex Fiend to open iBoot.im4p and correct its length value. Make sure File : Mode is set to Override.
Observe iBoot.im4p's five-byte header and length value -- for example 308307DE0B. Convert the length (0x7DE0B) to decimal (515595) and add 30 (for the length of iBoot.payp.bin in this case). So the new length in this case is 515625 (== 0x7DE29).
Correct the length value. The header in this case will now be 308307DE29. Save iBoot.im4p.
img4tool -c iBoot.img4 -p iBoot.im4p -m iBoot.im4m.org

The patched Stage 2 module should now be in iBoot.img4.

The Kernel Cache (`kernelcache`)

As with the iBoot Stage 2 module, you'll need to unwrap the kernel cache, patch it, and rewrap it in img4 format.

img4tool -e -p kernelcache.im4p.org kernelcache.org
img4tool -e -m kernelcache.im4m.org kernelcache.org
img4tool -e -o kernelcache.bin.org kernelcache.im4p.org

kernelcache.im4p.org, like iBoot.im4p.org, has a PAYP structure at its end. Copy the PAYP structure to a separate file, which you'll later append to kernelcache.im4p.

openssl asn1parse -inform der -in kernelcache.im4p.org -i -dump

This outputs the entire kernelcache.im4p.org file in human-readable ASN1 format. The PAYP structure appears at the end, and looks something like this. Use the structure's offset (18030138 in this case) in the next command.

18030138:d=1  hl=3 l= 186 cons:  cont [ 0 ]        
18030141:d=2  hl=3 l= 183 cons:   SEQUENCE          
18030144:d=3  hl=2 l=   4 prim:    IA5STRING         :PAYP
18030150:d=3  hl=3 l= 174 cons:    SET               
18030153:d=4  hl=7 l=  19 cons:     priv [ 1801676144 ] 
18030160:d=5  hl=2 l=  17 cons:      SEQUENCE          
18030162:d=6  hl=2 l=   4 prim:       IA5STRING         :kcep
18030168:d=6  hl=2 l=   9 prim:       INTEGER           :FFFFFE0007B7D488
18030179:d=4  hl=7 l=  14 cons:     priv [ 1801677926 ] 
18030186:d=5  hl=2 l=  12 cons:      SEQUENCE          
18030188:d=6  hl=2 l=   4 prim:       IA5STRING         :kclf
18030194:d=6  hl=2 l=   4 prim:       INTEGER           :030E4000
18030200:d=4  hl=7 l=  19 cons:     priv [ 1801677935 ] 
18030207:d=5  hl=2 l=  17 cons:      SEQUENCE          
18030209:d=6  hl=2 l=   4 prim:       IA5STRING         :kclo
18030215:d=6  hl=2 l=   9 prim:       INTEGER           :FFFFFE0007004000
18030226:d=4  hl=7 l=  14 cons:     priv [ 1801677946 ] 
18030233:d=5  hl=2 l=  12 cons:      SEQUENCE          
18030235:d=6  hl=2 l=   4 prim:       IA5STRING         :kclz
18030241:d=6  hl=2 l=   4 prim:       INTEGER           :AF0000
18030247:d=4  hl=7 l=  11 cons:     priv [ 1801679462 ] 
18030254:d=5  hl=2 l=   9 cons:      SEQUENCE          
18030256:d=6  hl=2 l=   4 prim:       IA5STRING         :kcrf
18030262:d=6  hl=2 l=   1 prim:       INTEGER           :00
18030265:d=4  hl=7 l=  14 cons:     priv [ 1801679482 ] 
18030272:d=5  hl=2 l=  12 cons:      SEQUENCE          
18030274:d=6  hl=2 l=   4 prim:       IA5STRING         :kcrz
18030280:d=6  hl=2 l=   4 prim:       INTEGER           :02C64000
18030286:d=4  hl=7 l=  14 cons:     priv [ 1801680742 ] 
18030293:d=5  hl=2 l=  12 cons:      SEQUENCE          
18030295:d=6  hl=2 l=   4 prim:       IA5STRING         :kcwf
18030301:d=6  hl=2 l=   4 prim:       INTEGER           :02C64000
18030307:d=4  hl=7 l=  13 cons:     priv [ 1801680762 ] 
18030314:d=5  hl=2 l=  11 cons:      SEQUENCE          
18030316:d=6  hl=2 l=   4 prim:       IA5STRING         :kcwz
18030322:d=6  hl=2 l=   3 prim:       INTEGER           :480000

xxd -p -s 18030138 kernelcache.im4p.org kernelcache.payp.hex

kernelcache.payp.hex is a hex dump. The following command will convert it to a binary (in DER format):

xxd -p -r kernelcache.payp.hex kernelcache.payp.bin

Check the contents of kernelcache.payp.bin by running the following command:

openssl asn1parse -inform der -in kernelcache.payp.bin -i

Now copy kernelcache.bin.org to kernelcache.bin. Ghidra doesn't work well with kernel cache files, so I use Hopper Disassembler to patch kernelcache.bin.

In Hopper choose Read Executable to Disassemble (CMD-SHIFT-O) and select kernelcache.bin. Then click on Loader, scroll down to com.apple.security.AppleVPBootPolicy and click OK.
Select Labels, type "_validate_acm_context" and click on it in the list below. This should take you to the function, whose top few lines should look like this:

                     __validate_acm_context:
fffffe0008be57ec         pacibsp
fffffe0008be57f0         sub        sp, sp, #0x40
fffffe0008be57f4         stp        x20, x19, [sp, #0x20]
fffffe0008be57f8         stp        fp, lr, [sp, #0x30]
fffffe0008be57fc         add        fp, sp, #0x30
fffffe0008be5800         sturb      wzr, [fp, var_11]
fffffe0008be5804         cbz        w0, loc_fffffe0008be5868

You want to find the calls to _validate_acm_context() from _command_create_linked_manifest() and _command_update_local_policy_for_kcos(), and patch each one out by replacing it with a nop instruction.
Click on _validate_acm_context()'s first line (pacibsp), then press the "x" key. This should open a dialog listing all the calls to _validate_acm_context() from elsewhere in AppleVPBootPolicy. Find _command_create_linked_manifest() and _command_update_local_policy_for_kcos() in this list, and click on each of them in turn. The calls to _validate_acm_context() should look something like this:

fffffe0008bd4e74         add        x22, x21, #0x66
fffffe0008bd4e78         mov        w0, #0x1
fffffe0008bd4e7c         mov        x1, x22
fffffe0008bd4e80         bl         __validate_acm_context
fffffe0008bd4e84         tbnz       w0, 0x0, loc_fffffe0008bd4ea8

Select the call to _validate_acm_context() (bl __validate_acm_context) and change to Hexadecimal mode (from ASM mode). For each of the following four hex values, double-click on it and replace it with values from the following list (binary code for a nop instruction). (Hit Enter to update each value after you've typed in its replacement.)

1f 20 03 d5

Repeat the previous two steps until you've patched the calls to _validate_acm_context() from both _command_create_linked_manifest() and _command_update_local_policy_for_kcos().
Choose File : New Executable and overwrite kernelcache.bin
Use vbindiff kernelcache.bin.org kernelcache.bin to check your results. There should be just two changes, each to a four-byte value -- the length of a single AARCH64 instruction.
Run img4tool kernelcache.im4p.org, which will produce output something like this. Use the information from it to run the next command. The value of "desc" differs from one version of macOS to another. Note that the current version of img4tool can't yet deal with kernelcache.im4p.org's PAYP structure.

img4tool version: 0.199-ed194718f9d6a035a432f2fdfe9fc639b72cba6c-RELEASE
Compiled with plist: YES
IM4P: ---------
type: krnl
desc: KernelManagement_host-354.140.3
size: 0x01131df6

Compression: bvx2
Uncompressed size: 0x03bd4000
PAYP:
kcep: kcep: [Error] img4tool: failed with exception:
[exception]:
what=assure failed
code=15597586
line=238
file=ASN1DERElement.cpp
commit count=199
commit sha  =ed194718f9d6a035a432f2fdfe9fc639b72cba6c

img4tool -c kernelcache.im4p -t krnl -d "KernelManagement_host-354.140.3" kernelcache.bin
dd if=kernelcache.payp.bin >> kernelcache.im4p
Use Hex Fiend to open kernelcache.im4p and correct its length value. Make sure File : Mode is set to Override.
Observe kernelcache.im4p's six-byte header and length value -- for example 308403BD4033. Convert the length (0x03BD4033) to decimal (62734387) and add 189 (for the length of kernelcache.payp.bin in this case). So the new length in this case is 62734576 (== 0x3BD40F0).
Correct the length value. The header in this case will now be 308403BD40F0. Save kernelcache.im4p.
img4tool -c kernelcache -p kernelcache.im4p -m kernelcache.im4m.org

The patched kernel cache should now be in kernelcache.

Copying the Patched Modules to their Final Destinations

iBoot Stage 0 (`AVPBooter.vmapple2.bin`)

AVPBooter.vmapple2.bin is a system file. By default it's protected by the macOS file system, and can't be changed. To get around this I borrow from How to Defang macOS System Protections. Before you follow these steps, you must make the settings changes I described above under Settings Changes.

mkdir /tmp/mount
Run mount at a Terminal prompt and observe its results, for example as follows. Use the contents of the first line in the next command.

/dev/disk5s1s1 on / (apfs, sealed, local, read-only, journaled)
devfs on /dev (devfs, local, nobrowse)
/dev/disk5s6 on /System/Volumes/VM (apfs, local, noexec, journaled, noatime, nobrowse)
/dev/disk5s2 on /System/Volumes/Preboot (apfs, local, journaled, nobrowse)
/dev/disk5s4 on /System/Volumes/Update (apfs, local, journaled, nobrowse)
/dev/disk1s2 on /System/Volumes/xarts (apfs, local, noexec, journaled, noatime, nobrowse)
/dev/disk1s1 on /System/Volumes/iSCPreboot (apfs, local, journaled, nobrowse)
/dev/disk1s3 on /System/Volumes/Hardware (apfs, local, journaled, nobrowse)
/dev/disk5s5 on /System/Volumes/Data (apfs, local, journaled, nobrowse, protect)
/dev/disk2s3 on /Volumes/Boot3 (apfs, sealed, local, read-only, journaled)
/dev/disk7s3 on /Volumes/Boot2 (apfs, sealed, local, read-only, journaled)
/dev/disk4s1 on /Volumes/Boot4 - Data (apfs, local, journaled, nobrowse, protect)
/dev/disk2s1 on /Volumes/Boot3 - Data (apfs, local, journaled, nobrowse, protect)
/dev/disk4s3 on /Volumes/Boot4 (apfs, sealed, local, read-only, journaled)
/dev/disk6s1 on /Volumes/Boot1 - Data (apfs, local, journaled, nobrowse, protect)
/dev/disk6s3 on /Volumes/Boot1 (apfs, sealed, local, read-only, journaled)
/dev/disk7s1 on /Volumes/Boot2 - Data (apfs, local, journaled, nobrowse, protect)
map auto_home on /System/Volumes/Data/home (autofs, automounted, nobrowse)

sudo mount -o nobrowse -t apfs /dev/disk5s1 /tmp/mount
cd /tmp/mount/System/Library/Frameworks/Virtualization.framework/Resources
sudo cp /path/to/patched/AVPBooter.vmapple2.bin .
sudo bless --mount /tmp/mount --bootefi --create-snapshot
Reboot your host computer.

On reboot, check the contents of /System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin to make sure it's what you expect.

You need to keep your host computer in this state as long as you're using third party kernel extensions in your macOS guest VM. (Without the patched AVPBooter.vmapple2.bin it will simply refuse to start.)

If need be, use the following command to revert your snapshot (and AVPBooter.vmapple2.bin), then reboot your host computer:

sudo bless --mount / --last-sealed-snapshot

iBoot Stage 1 (`LLB.img4`)

You'll need to copy both LLB.img4 and logo.img4.org over their original contents in AuxiliaryStorage. This is because LLB.img4 might be a different length than LLB.img4.org, and Apple's virtualization infrastructure expects the logo image to immediately follow the LLB image. You can skip this section if you discovered above that your existing LLB image already contains a patch (applied to an earlier version of macOS in your guest VM).

Use Hex Fiend to open LLB.img4, logo.img4.org and AuxiliaryStorage. Make sure Edit : Mode for AuxilaryStorage is Overwrite.
Jump to offset 0x224000 in AuxiliaryStorage. If an LLB image exists at this location, its first two bytes should be 3083. If not, jump to offset 0x24000.
Copy the contents of LLB.img4 (CMD-A, CMD-C) and paste them into AuxilaryStorage.
Copy the contents of logo.img4.org and paste them into AuxiliaryStorage, then save the file.

iBoot Stage 2 (`iBoot.img4`) and the Kernel Cache (`kernelcache`)

Copy these files to your macOS guest VM and do the following there:

Run kmutil inspect, and observe where the boot kernel cache exists in the VM's file system. The following is an example, which will get used in the following steps. The path's exact contents will differ from case to case.

/System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache

Note the "Next Stage Image4 Hash" just before /System/Library/Caches -- in this case FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83.
sudo cp kernelcache /System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache
Run cd / and sudo find . -name iBoot.img4 -exec ls -al \{\} \;. Observe the hit whose path contains the Next Stage Image4 Hash. Use the results in the following command. For example:
sudo cp iBoot.img4 /System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4

Shut down your your macOS guest VM and reboot it into Recovery Mode. Then perform the following steps. You'll be looking for the same Next Stage Image4 Hash as in the previous steps.

Run Terminal in Recovery Mode, then do the following:

Run cd / and find . -name iBoot.img4 -exec ls -al \{\} \;. Expect two hits this time, which should look like the following:

/System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4
/System/Volumes/Data/private/tmp/Recovery/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4

Copy iBoot.img4 over both of them.

You may also see another hit that looks like the following. Ignore it. You can tell by its date and file size that you already copied the patched iBoot.img4 over it above.

/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/usr/standalone/firmware/iBoot.img4

Run cd / and find . -name kernelcache -exec ls -al \{\} \;. Once again expect two hits, which should look like the following (maybe plus one superfluous hit, as above).

/System/Volumes/Preboot/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache
/System/Volumes/Data/private/tmp/Recovery/8467D650-E8D9-4F4C-9403-F51E736C25B0/boot/FB72884642D3490C1D6A0C25D6901AD49BFF7A168B2A851361DD40B393D5FE8E730EF7417B83303610CA04EF15C5CD83/System/Library/Caches/com.apple.kernelcaches/kernelcache

Copy kernelcache over both of them.
Reboot your macOS guest VM and start playing with third-party kernel extensions on it!

steven-michaud/ThirdPartyKexts.md

Running Third Party Kernel Extensions on Virtualization Framework macOS Guest VMs

Prerequisites

Background

Settings Changes (Host and Guest)

Special Considerations Once You've Made the Changes Described Here

Finding the Modules to be Patched

iBoot Stage 0 (`AVPBooter.vmapple2.bin`)

iBoot Stage 1 (LLB)

iBoot Stage 2 (`iBoot.img4`) and the Kernel Cache (`kernelcache`)

Patching These Modules

iBoot Stage 0 (`AVPBooter.vmapple2.bin.org`)

iBoot Stage 1 (`LLB.img4`)

iBoot Stage 2 (`iBoot.img4`)

The Kernel Cache (`kernelcache`)

Copying the Patched Modules to their Final Destinations

iBoot Stage 0 (`AVPBooter.vmapple2.bin`)

iBoot Stage 1 (`LLB.img4`)

iBoot Stage 2 (`iBoot.img4`) and the Kernel Cache (`kernelcache`)

steven-michaud commented Sep 12, 2024

steven-michaud commented Sep 13, 2024

pandora-SG commented Sep 13, 2024

steven-michaud commented Sep 13, 2024 •

edited

Loading

htonl commented Nov 11, 2024 •

edited

Loading

steven-michaud commented Nov 11, 2024

steven-michaud commented Nov 11, 2024

steven-michaud/ThirdPartyKexts.md

Running Third Party Kernel Extensions on Virtualization Framework macOS Guest VMs

Prerequisites

Background

Settings Changes (Host and Guest)

Special Considerations Once You've Made the Changes Described Here

Finding the Modules to be Patched

iBoot Stage 0 (AVPBooter.vmapple2.bin)

iBoot Stage 1 (LLB)

iBoot Stage 2 (iBoot.img4) and the Kernel Cache (kernelcache)

Patching These Modules

iBoot Stage 0 (AVPBooter.vmapple2.bin.org)

iBoot Stage 1 (LLB.img4)

iBoot Stage 2 (iBoot.img4)

The Kernel Cache (kernelcache)

Copying the Patched Modules to their Final Destinations

iBoot Stage 0 (AVPBooter.vmapple2.bin)

iBoot Stage 1 (LLB.img4)

iBoot Stage 2 (iBoot.img4) and the Kernel Cache (kernelcache)

steven-michaud commented Sep 12, 2024

steven-michaud commented Sep 13, 2024

pandora-SG commented Sep 13, 2024

steven-michaud commented Sep 13, 2024 • edited Loading

htonl commented Nov 11, 2024 • edited Loading

steven-michaud commented Nov 11, 2024

steven-michaud commented Nov 11, 2024

iBoot Stage 0 (`AVPBooter.vmapple2.bin`)

iBoot Stage 2 (`iBoot.img4`) and the Kernel Cache (`kernelcache`)

iBoot Stage 0 (`AVPBooter.vmapple2.bin.org`)

iBoot Stage 1 (`LLB.img4`)

iBoot Stage 2 (`iBoot.img4`)

The Kernel Cache (`kernelcache`)

iBoot Stage 0 (`AVPBooter.vmapple2.bin`)

iBoot Stage 1 (`LLB.img4`)

iBoot Stage 2 (`iBoot.img4`) and the Kernel Cache (`kernelcache`)

steven-michaud commented Sep 13, 2024 •

edited

Loading

htonl commented Nov 11, 2024 •

edited

Loading