Control Flow Guard (CFG/CFGuard) for mingw-w64

cfguard-for-mingw-w64.md

Control Flow Guard (CFG/CFGuard) for mingw-w64

Control Flow Guard is a security mitigation that verifies the target address of indirect calls. It works by having the compiler insert a check at indirect call sites to verify the validity of the call target, and also the linker write the necessary data and flags into the PE/COFF image to enable the feature on Windows' end.

This feature was first introduced on Windows 8.1 and came with VS 2015. Existing mingw-w64 toolchains do not support it. As far as I can tell, GCC does not know about this, neither does ld.bfd. LLVM does have it implemented for the MSVC target and it can be used with mingw-w64 through some hacks.

Technical details

These are needed to use control flow guard:

1. Adding checks to indirect calls

The checks are basically a transformation of this:

leaq   target_func(%rip), %rax
callq  *%rax

...into this:

movl	$target_func, %ecx
calll	*___guard_check_icall_fptr
calll	*%ecx

...or in the case of x86_64:

leaq   target_func(%rip), %rax
callq  *__guard_dispatch_icall_fptr(%rip)

__guard_check_icall_fptr and __guard_dispatch_icall_fptr are the symbol names used by MSVC and LLVM to hold the pointer to the CFGuard check functions.

There needs to be a way for users to disable CFGuard checks for a specific function. MSVC and Clang supports __declspec(guard(nocf)) for this.

Since https://reviews.llvm.org/D132661 has landed, this attribute also works when compiling for MinGW target.

2. Collecting all valid call targets into object file

CFGuard has to know what addresses are valid call targets, which means the compiler needs to keep track of them and write them into the object file. Any functions that has its address taken shall be considered, because chances are they will be used in an indirect call somewhere else.

MSVC (technically I'm just describing what LLVM/Clang does for the MSVC target) uses these sections:

• .gfids$y - "Guard Function ID": These are symbols within this object file which shall be considered valid call targets.
• .giats$y - "Guard Address-Taken IAT Entry": These are functions from the IAT which has had its address taken, which shall be considered valid call targets.
• .gljmp$y - "Guard Long Jump Target": These are locations which shall be considered valid long jump targets.
• .gehcont$y - "Guard EH Continuation": These are locations which shall be considered valid exception handling continuation targets. Note: Guard EH Continuation is a later addition in VS 2019 16.7 and only supported on 64-bit. It is only enabled when /guard:ehcont is passed to the compiler. Technically this is not part of CFGuard, but a separate feature called Control-flow Enforcement Technology (CET). Let's ignore this for now.

Clang uses the assembly directive .symidx to build these sections. Each entry is a 4-byte index of the symbol in the symbols table.

3. Set @feat.00 symbol

The 0x800 bit shall be set in this value to indicate that this object has been compiled with CFGuard.

4. Supply placeholder CFGuard check/dispatch functions and pointers

VC runtime supplies void *__guard_check_icall_fptr, and also void *__guard_dispatch_icall_fptr for x86_64. They are initialized with the addresses of default placeholder CFGuard check/dispatch functions which does no checking. This makes the executables backward-compatible with older Windows that does not support CFGuard.

5. Link the guard tables

The linker shall create the tables (GuardCFFunctionTable, GuardAddressTakenIatEntryTable, GuardLongJumpTargetTable and GuardEHContinuationTable) by collecting all symbols referenced from the guard sections (.gfids$y, .giats$y, .gljmp$y and .gehcont$y respectively) from all object files.

The format of the tables are described in MS docs.

6. Include the Load Config directory

The load config directory contains fields essential to the operation of CFG. LINK.exe takes the _load_config_used symbol and uses it as the load config directory. LLD does the same thing, though it considers this to be optional. Ld.bfd likely does not handle this at the moment.

VC runtime appears to supply a default _load_config_used symbol. It apparently can be overridden from user code, though I don't see this in official docs. LINK.exe does have checks in place to attempt to verify the validity of the structure of this symbol (it needs to contain certain values). Mingw-w64 does not currently supply this symbol, but user code can supply it and LLD will take that.

Specific data for the _load_config_used structure is provided by linker- supplied symbols as absolute VA (at least for LLD). The definition of _load_config_used shall include them.

• __guard_fids_table
• __guard_fids_count
• __guard_flags - CFGuard flags for _load_config_used.GuardFlags, should be IMAGE_GUARD_CF_INSTRUMENTED | IMAGE_GUARD_CF_FUNCTION_TABLE_PRESENT (0x500) for CFGuard-enabled images. In addition, if the image includes a "Guard Long Jump Target" table, the IMAGE_GUARD_CF_LONGJUMP_TABLE_PRESENT bit should be set. This value is 32-bit.
• __guard_iat_table
• __guard_iat_count
• __guard_longjmp_table
• __guard_longjmp_count
• __guard_eh_cont_table: TODO
• __guard_eh_cont_count: TODO

In addition, these symbols are also to be referenced in _load_config_used:

• __guard_check_icall_fptr - location that stores a pointer to the CFGuard check function. _load_config_used.GuardCFCheckFunction shall point to this location.
• __guard_dispatch_icall_fptr - location that stores a pointer to the CFGuard dispatch function (only for x86_64). _load_config_used.GuardCFCheckDispatch shall point to this location for x86_64 images.

There are also these symbols that are not applicable for mingw-w64:

• __security_cookie: used by MSVC for /GS buffer security checks (it may also be reused for other hardening features); GCC and Clang supports -fstack-protector, but __stack_chk_guard is used instead (Clang uses __security_cookie only with the MSVC target).
• __safe_se_handler_table, __safe_se_handler_count: this table is only for i686, and mingw-w64 does not support SafeSEH on i686. (TODO: needs verification)
• __enclave_config: ???

7. Set the DLLCharacteristics field in the image optional header

The IMAGE_DLL_CHARACTERISTICS_GUARD_CF (0x4000) bit shall be set.

Enabling with Clang

See also mstorsjo/llvm-mingw#301.

Additions needed in mingw-w64

To support the use case of enabling CFGuard with Clang, mingw-w64 should:

• Supply __guard_check_icall_fptr and its placeholder implementation.
• Supply __guard_dispatch_icall_fptr and its placeholder implementation (only for x86_64).
• Supply a default _load_config_used structure.

All these can be included in libmingwex.a within mingw-w64-crt.

Alternatively, the user can supply these symbols themselves as a stopgap measure.

Toolchain distribution changes

When compiling the toolchain, everything needs to be compiled and linked with the CFGuard-enabling flags, with the exception of sanitizer runtimes (the API hooking mechanism can trip off the CFGuard checks).

Compiler and linker flags

https://reviews.llvm.org/D132810 adds -mguard=cf and -mguard=cf-nochecks to Clang's GNU driver.

https://reviews.llvm.org/D132808 adds --guard-cf and --guard-longjmp to LLD's MinGW driver. These flags are set automatically if linking with the clang or clang++ driver with the proper -mguard=xxx flags.

These flags are available since LLVM 16.

Legacy info for LLVM 15 or earlier

Before these, Clang does not map any of the CFGuard flags from the GCC-style driver frontend, therefore we need to use the cc1 flags by prepending them with -Xclang.

clang-cl option	cc1 option
/guard:cf	-cfguard
/guard:cf,nochecks	-cfguard-no-checks
/guard:ehcont	-ehcontguard

Similarly, the MinGW driver of LLD currently does not map the CFGuard flags to its COFF linker, therefore we need to use the link.exe-style flags by prepending them with -Xlink.

• -guard:cf
• -guard:longjmp: needed to link the "Guard Long Jump Target" table into the final image.
• -guard:cf,longjmp: this doesn't work via -Wl because it splits arguments by comma. Use -Xlinker -Xlink -Xlinker -guard:cf,longjmp instead. Also, -guard:longjmp already implies -guard:cf so it is not really necessary to specify both.

Note: For the MSVC linker /guard:cf actually implies /guard:cf,longjmp. It seems that LLD not generating the long jump table for /guard:cf would be a bug. Addressed in https://reviews.llvm.org/D132901.

When compiling and linking in a single step, the command line may look like this:

clang++ main.cpp -o main.exe -Xclang -cfguard -Wl,-Xlink,-guard:cf

Additional attributes

Clang supports the function attribute __declspec(guard(nocf)) to disable CFGuard checks inside a specific function. This will be usable in MinGW mode without needing hacks after D132302 has landed.

For GCC and ld.bfd

Here is what I think should be done:

1. Ld.bfd needs to be taught how to link the guard tables and the load config directory. An option should be added to enable this.
2. As a starting point, GCC should be taught how to generate the guard table sections, equivalent to the -cfguard-no-checks option of Clang.
3. After gaining support for the above, GCC should be taught to implement CFGuard checks by transforming indirect calls as required, with the support of a new function attribute guard(nocf) to disable these checks for the specific function.

Excellent writeup! A couple question remain for me though:

• When the OS runs the code with cfguard activated, what happens? I presume the loader rewrites the addresses in __guard_check_icall_fptr and __guard_dispatch_icall_fptr to point somewhere else, to OS provided routines for checking things? (Curious thought - the function that ends up used instead of the dummy we default initialize it to - which DLL provides it?)

• If the _load_config_used object file would end up pulled in, when linking with a linker that isn't CFG aware, it would result in a failed link due to the linker not providing the synthesized tables, which are referenced in that structure, right?

When the OS runs the code with cfguard activated, what happens?

The check function raises a fail fast exception (I'm guessing __fastfail) with an exception code 0xc0000409 which is STATUS_STACK_BUFFER_OVERRUN (but as Raymond Chen wrote in a post all fail fast exceptions use this code so it doesn't mean much).

I presume the loader rewrites the addresses in __guard_check_icall_fptr and __guard_dispatch_icall_fptr to point somewhere else, to OS provided routines for checking things? (Curious thought - the function that ends up used instead of the dummy we default initialize it to - which DLL provides it?)

Yep (the loader gets the location of these addresses from the load config directory.) The check functions are in ntdll according to the debugger.

If the _load_config_used object file would end up pulled in, when linking with a linker that isn't CFG aware, it would result in a failed link due to the linker not providing the synthesized tables, which are referenced in that structure, right?

Yes, I suppose.

It seems clang itself also supports the -Xclang -cfguard parameter on Linux and macOS. But is the -Wl,-Xlink,-guard:cf also needed on those platforms?

It seems clang itself also supports the -Xclang -cfguard parameter on Linux and macOS. But is the -Wl,-Xlink,-guard:cf also needed on those platforms?

If you mean cross-compiling for the mingw-w64 target, keep in mind that Clang on all platforms are capable of cross-compiling and they behave mostly the same (except for some specific build options) so yes, you do need to specify the same options when linking.

Otherwise, please beware that Control Flow Guard relies on Windows OS feature so specifying that option for other targets will either do nothing or crash Clang. -Xclang passes options directly to cc1 which doesn't do much checking.

Indeed, enabling control flow guard on other platforms than Windows causes crashes.

I need a little help, have tried everything and every combination of flags but the output is always the same, all tried with latest msys2 mingw64 with clang 16.0.4:

ld.lld: warning: Control Flow Guard is enabled but '_load_config_used' is missing

eg.:

clang++ -c -g -std=c++2a -mguard=cf -o main.o main.cpp
clang++ -fuse-ld=lld -Wl,-subsystem,console -mguard=cf -o a.exe main.o

or as one command:

clang++ -g -std=c++2a -fuse-ld=lld -mguard=cf main.cpp

or as you have described above:

clang++ -g -std=c++2a -fuse-ld=lld -Xclang -cfguard -Wl,-Xlink,-guard:cf main.cpp

Also tried to link against the libmingwex.a:

clang++ -c -g -std=c++2a -mguard=cf -o main.o main.cpp
clang++ -fuse-ld=lld -Wl,-subsystem,console -mguard=cf -o a.exe main.o C:/msys64/mingw64/lib/libmingwex.a

Every time the same result:

ld.lld: warning: Control Flow Guard is enabled but '_load_config_used' is missing

Where is the problem, is actually possible to compile with Control Flow Guard on msys2?

@silverqx use the clang64 environment of MSYS2. Support is tracked on msys2/MINGW-packages#16833.

This requires the mingw-w64 CRT itself to be built with CFG, which is only supported when built with Clang. (This was added in mingw-w64/mingw-w64@7d5c59b.) Because of this, the mingw64 and ucrt64 environments of MSYS2, which builds the CRT with GCC, will not support CFG as long as it is not supported by GCC, even if you build your application with Clang.

@alvinhochun Great, thx for replying, so that was the reason why.

Hey, any progress with ehcont mingw-w64?

I have not looked into ehcont and don't plan to do so in the near future. If you would like to look into it, feel free to do so.

The /guard:ehcont looks pretty complex to implement, isn't an easy thing.

Since Clang should already support ehcont for MSVC target, it may not be too difficult to hook it up for mingw target. Of course this still requires investigating what exactly it does and how it is implemented. You also need some test cases to verify that it does work.