Conversion of Wine builtins to PE format #wine #mingw #llvm #gcc #pe #elf #commandlinefu

conversion_of_wine_builtins_to_pe_format.md

Conversion of Wine builtins to PE format

Status of Conversion

List of Wine builtin fake executables in Wine installation as of Wine 7.16:

mainline-install-7.16-x86_64/lib/wine/x86_64-windows/openal32.dll
mainline-install-7.16-x86_64/lib/wine/x86_64-windows/opengl32.dll
mainline-install-7.16-x86_64/lib/wine/i386-windows/openal32.dll
mainline-install-7.16-x86_64/lib/wine/i386-windows/opengl32.dll

List of modules excluded from statistics (considered legacy or OS/platform specific):

• capi2032 (unixlib only)
• wineandroid.drv (converted as of Wine 7.11)
• winecoreaudio.drv (converted as of Wine 6.23)
• winemac.drv (converted as of Wine 7.10)
• wineoss.drv (converted as of Wine 7.8)

List of Wine builtin fake executables installed in shared WoW64 WINEPREFIX as of Wine 7.16:

$ grep -ralZP "Wine .* DLL" .wine/drive_c | xargs -r0i bash -c \
    "if ! grep -obUa \"Wine builtin DLL\" \"{}\" 2>/dev/null | grep -q \"64:Wine builtin DLL\" ; \
    then echo \"{}\" ; fi"  | sort

.wine/drive_c/windows/rundll.exe
.wine/drive_c/windows/system32/openal32.dll
.wine/drive_c/windows/system32/opengl32.dll
.wine/drive_c/windows/system32/spool/drivers/win40/0/wineps16.drv
.wine/drive_c/windows/system/ddeml.dll
.wine/drive_c/windows/system/mmsystem.dll
.wine/drive_c/windows/syswow64/avifile.dll
.wine/drive_c/windows/syswow64/commdlg.dll
.wine/drive_c/windows/syswow64/comm.drv
.wine/drive_c/windows/syswow64/compobj.dll
.wine/drive_c/windows/syswow64/ctl3d.dll
.wine/drive_c/windows/syswow64/ctl3dv2.dll
.wine/drive_c/windows/syswow64/ddhelp.exe
.wine/drive_c/windows/syswow64/dispdib.dll
.wine/drive_c/windows/syswow64/display.drv
.wine/drive_c/windows/syswow64/dosx.exe
.wine/drive_c/windows/syswow64/dsound.vxd
.wine/drive_c/windows/syswow64/gdi.exe
.wine/drive_c/windows/syswow64/imm.dll
.wine/drive_c/windows/syswow64/keyboard.drv
.wine/drive_c/windows/syswow64/krnl386.exe
.wine/drive_c/windows/syswow64/lzexpand.dll
.wine/drive_c/windows/syswow64/mouse.drv
.wine/drive_c/windows/syswow64/msacm.dll
.wine/drive_c/windows/syswow64/msvideo.dll
.wine/drive_c/windows/syswow64/ole2conv.dll
.wine/drive_c/windows/syswow64/ole2disp.dll
.wine/drive_c/windows/syswow64/ole2.dll
.wine/drive_c/windows/syswow64/ole2nls.dll
.wine/drive_c/windows/syswow64/ole2prox.dll
.wine/drive_c/windows/syswow64/ole2thk.dll
.wine/drive_c/windows/syswow64/olecli.dll
.wine/drive_c/windows/syswow64/olesvr.dll
.wine/drive_c/windows/syswow64/openal32.dll
.wine/drive_c/windows/syswow64/opengl32.dll
.wine/drive_c/windows/syswow64/rasapi16.dll
.wine/drive_c/windows/syswow64/setupx.dll
.wine/drive_c/windows/syswow64/shell.dll
.wine/drive_c/windows/syswow64/sound.drv
.wine/drive_c/windows/syswow64/storage.dll
.wine/drive_c/windows/syswow64/stress.dll
.wine/drive_c/windows/syswow64/system.drv
.wine/drive_c/windows/syswow64/toolhelp.dll
.wine/drive_c/windows/syswow64/typelib.dll
.wine/drive_c/windows/syswow64/user.exe
.wine/drive_c/windows/syswow64/ver.dll
.wine/drive_c/windows/syswow64/w32sys.dll
.wine/drive_c/windows/syswow64/win32s16.dll
.wine/drive_c/windows/syswow64/win87em.dll
.wine/drive_c/windows/syswow64/winaspi.dll
.wine/drive_c/windows/syswow64/windebug.dll
.wine/drive_c/windows/syswow64/wing.dll
.wine/drive_c/windows/syswow64/winnls.dll
.wine/drive_c/windows/syswow64/winoldap.mod
.wine/drive_c/windows/syswow64/winsock.dll
.wine/drive_c/windows/syswow64/wintab.dll
.wine/drive_c/windows/twain.dll
.wine/drive_c/windows/winhelp.exe

NOTE: The discrepancy between the number of fake builtin Wine executables in the Wine installation and the WINEPREFIX is due to placeholders for Win16 binaries in NE format (.dll16, .drv16) in WINEPREFIX.

The following script determines the number of all Wine builtin fake executables and builtin executables in PE format for a range of Wine releases. It outputs the statistics in markdown table format for embedding into this gist.

#!/bin/bash

DEFAULT_WINE_VERSIONS="7.{16..0} 6.{23..0} 5.{22..0}"
# override by env: WINE_VERSIONS="7.{4..5}" <script>
WINE_VERSIONS=${WINE_VERSIONS:-$DEFAULT_WINE_VERSIONS}

echo "| Wine release | builtin fake executables | builtin PE executables |"
echo "|:----|:----:|----:|"
for wine_ver in $(eval echo $WINE_VERSIONS)  ; do
    builtin_fake=()
    builtin_pe=()
    files=( $(grep -ralP "Wine .* DLL" ~/projects/wine/mainline-install-$wine_ver-x86_64/{lib,lib64} 2>/dev/null) )
    for file in ${files[@]}; do
        [[ $file =~ ".so" ]] && continue
        if grep -obUa "Wine builtin DLL" $file 2>/dev/null | grep -q "64:Wine builtin DLL" ; then
            builtin_pe+=( $file )
        else
            builtin_fake+=( $file )
        fi
    done
    echo "| $wine_ver | ${#builtin_fake[@]} | ${#builtin_pe[@]} |"
done

Wine release	builtin fake executables	builtin PE executables
7.16	4	1426
7.15	4	1424
7.14	4	1424
7.13	4	1424
7.12	6	1420
7.11	6	1420
7.10	6	1418
7.9	6	1418
7.8	6	1420
7.7	8	1416
7.6	8	1414
7.5	8	1414
7.4	10	1412
7.3	16	1404
7.2	16	1984
7.1	16	1980
7.0	16	1976
6.23	18	1974
6.22	22	1968
6.21	22	1964
6.20	32	1952
6.19	114	1864
6.18	125	1851
6.17	129	1847
6.16	133	1839
6.15	133	1835
6.14	135	1831
6.13	137	1829
6.12	137	1821
6.11	135	1815
6.10	133	1813
6.9	135	1809
6.8	137	1807
6.7	139	1797
6.6	139	1787
6.5	143	1777
6.4	145	1769
6.3	145	1766
6.2	147	1764
6.1	147	1762
6.0	147	1762
5.22	151	1756
5.21	179	1728
5.20	181	1718
5.19	183	1712
5.18	189	1704
5.17	191	1702
5.16	193	1700
5.15	193	1700
5.14	177	1694
5.13	177	1692
5.12	177	1690
5.11	179	1684
5.10	179	1680
5.9	179	1670
5.8	179	1670
5.7	179	1666
5.6	177	1664
5.5	184	1655
5.4	184	1653
5.3	186	1649
5.2	186	1645
5.1	192	1641
5.0	192	1639

NOTE: Since this is a bi-arch Wine build, the statistics include 32-bit and 64-bit binaries.

History

Initial announcement in What's new in Wine 5.0

What's new in Wine 5.0
======================


*** PE modules

- Most modules are built in PE format (Portable Executable, the
  Windows binary format) instead of ELF when the MinGW compiler is
  available. This helps various copy protection schemes that check
  that the on-disk and in-memory contents of system modules are
  identical.

- The actual PE binaries are copied into the Wine prefix instead of
  the fake DLL files. This makes the prefix look more like a real
  Windows installation, at the cost of some extra disk space.

- Modules that have been converted to PE can use standard wide-char C
  functions, as well as wide-char character constants like L"abc".
  This makes the code easier to read.

- Not all modules have been converted to PE yet; this is an ongoing
  process that will continue during the Wine 5.x development series.

- The Wine C runtime is updated to support linking to MinGW-compiled
  binaries; it is used by default instead of the MinGW runtime when
  building DLLs.

What's new in Wine 5.1

- Support for using LLVM-MinGW as PE cross-compiler.

What's new in Wine 5.5

- Better support for debug information in PE files.

What's new in Wine 5.6

- A few more modules converted to PE.

What's new in Wine 5.9

- Initial support for splitting dlls into PE and Unix parts.
- Support for generating PDB files when building PE dlls.

What's new in Wine 5.12

- NTDLL converted to PE format.

What's new in Wine 5.14

- Beginnings of PE conversion of the MSVCRT libraries.

What's new in Wine 5.17

- ADVAPI32 library converted to PE.

What's new in Wine 5.18

- USER32 library converted to PE.

What's new in Wine 5.19

- KERNEL32 library converted to PE.

What's new in Wine 5.21

- GDI32 library converted to PE.

What's new in Wine 5.22

- C runtime libraries converted to PE.

What's new in Wine 6.0

- Core modules in PE format.

What's new in Wine 6.3

- WineGStreamer library converted to PE.

What's new in Wine 6.6

- DWrite and DnsApi libraries converted to PE.

What's new in Wine 6.7

- NetApi32, WLDAP32, and Kerberos libraries converted to PE.

What's new in Wine 6.8

- Secur32 library converted to PE.

What's new in Wine 6.9

- WPCAP library converted to PE.

What's new in Wine 6.10

- WinePulse library converted to PE.

What's new in Wine 6.12

- More work towards WinSock PE conversion.

What's new in Wine 6.13

- More work towards WinSock PE conversion.
- Some progress on the IPHLPAPI PE conversion.

What's new in Wine 6.15

- WinSock (WS2_32) library converted to PE.

What's new in Wine 6.17

- WineCfg program converted to PE.

What's new in Wine 6.18

- Shell32 and WineBus libraries converted to PE.

What's new in Wine 6.19

- IPHlpApi, NsiProxy, WineDbg and a few other modules converted to PE.

What's new in Wine 6.20

- MSXml, XAudio, DInput and a few other modules converted to PE.
- A few system libraries are bundled with the source to support PE builds.

What's new in Wine 6.21

- WinSpool, GPhoto, and a few other modules converted to PE.

What's new in Wine 6.23

- Mount manager and CoreAudio driver converted to PE.
- Optional support for using the distribution's PE libraries.

What's new in Wine 7.0

- Most modules converted to PE format.

What's new in Wine 7.3

- Progress on the PE conversion of USER32 and WineALSA.

What's new in Wine 7.4

- WineD3D, D3D12 and DXGI modules converted to PE.

What's new in Wine 7.5

- ALSA driver converted to PE.

What's new in Wine 7.6

- More progress on the PE conversion of graphics drivers.

What's new in Wine 7.7

- More progress on the PE conversion of the X11 and OSS drivers.

What's new in Wine 7.8

- X11 and OSS drivers converted to PE.

What's new in Wine 7.9

- Preliminary work towards PE conversion of macOS driver.

What's new in Wine 7.10

- macOS driver converted to PE.

What's new in Wine 7.11

- Android driver converted to PE.

What's new in Wine 7.13

- USB driver converted to PE.

---

Links

• What's new in Wine 5.0
• What's new in Wine 5.1
• What's new in Wine 5.5
• What's new in Wine 5.6
• What's new in Wine 5.9
• What's new in Wine 5.12
• What's new in Wine 5.14
• What's new in Wine 5.17
• What's new in Wine 5.18
• What's new in Wine 5.19
• What's new in Wine 5.21
• What's new in Wine 5.22
• What's new in Wine 6.0
• What's new in Wine 6.3
• What's new in Wine 6.6
• What's new in Wine 6.7
• What's new in Wine 6.8
• What's new in Wine 6.9
• What's new in Wine 6.10
• What's new in Wine 6.12
• What's new in Wine 6.13
• What's new in Wine 6.15
• What's new in Wine 6.17
• What's new in Wine 6.18
• What's new in Wine 6.19
• What's new in Wine 6.20
• What's new in Wine 6.21
• What's new in Wine 6.23
• What's new in Wine 7.0
• What's new in Wine 7.3
• What's new in Wine 7.4
• What's new in Wine 7.5
• What's new in Wine 7.6
• What's new in Wine 7.7
• What's new in Wine 7.8
• What's new in Wine 7.9
• What's new in Wine 7.10
• What's new in Wine 7.11
• What's new in Wine 7.13

A quick search on their gitlab it looks like it's still that 5 things remaining from @n3rdopolis's comment, but they're moving forward with winemac.drv.

grep -R mcygwin | grep EXTRADLLFLAGS
dlls/wineusb.sys/Makefile.in:EXTRADLLFLAGS = -Wl,--subsystem,native -mcygwin
dlls/wineandroid.drv/Makefile.in:EXTRADLLFLAGS = -mcygwin
dlls/opengl32/Makefile.in:EXTRADLLFLAGS = -Wl,--image-base,0x7a800000 -mcygwin
dlls/openal32/Makefile.in:EXTRADLLFLAGS = -Wb,--prefer-native -mcygwin

7.10 out, 4 left.

3 to go!

grep -R mcygwin | grep  EXTRADLLFLAGS
dlls/wineusb.sys/Makefile.in:EXTRADLLFLAGS = -Wl,--subsystem,native -mcygwin
dlls/opengl32/Makefile.in:EXTRADLLFLAGS = -Wl,--image-base,0x7a800000 -mcygwin
dlls/openal32/Makefile.in:EXTRADLLFLAGS = -Wb,--prefer-native -mcygwin

7.12 out,
looks like something for wineusb.sys https://gitlab.winehq.org/wine/wine/-/merge_requests/356 was made last minute
but it's still having the -mcygwin flag for now, so still at 3

The way they give bit by bit every release makes me feel like they had the whole thing ready a long time ago.
In the merge request the commits were made a month ago.

I think stuff like this take longer to merge than to write. This patch for proton only got in a couple days ago after two months of wait ValveSoftware/wine#142

7.13 out

dlls/opengl32/Makefile.in:EXTRADLLFLAGS = -Wl,--image-base,0x7a800000 -mcygwin
dlls/openal32/Makefile.in:EXTRADLLFLAGS = -Wb,--prefer-native -mcygwin

Two to go!

7.14 out but nothing happening to remaining 2

https://gitlab.winehq.org/wine/wine/-/commits/master/dlls/openal32
https://gitlab.winehq.org/wine/wine/-/commits/master/dlls/opengl32

They just completed opengl32.dll today!
And it seems because openal32.dll is not actually part of Windows, they dropped it. Meaning there are 0 left

Only instance of mcygwin now is in the build system that would handle that flag

$ grep -R mcygwin
tools/winegcc/winegcc.c:                    if (strcmp("-mcygwin", opts.args.str[i]) == 0)
tools/makedep.c:                       !strarray_exists( &make->extradllflags, "-mcygwin" );
$

Oh nice!
There seem to be a bit of a regression hope they fix it.

Why was this merged, knowing that it caused a significant performance drop across a wide range of games? Why was it necessary to not even wait for merge request 1324 to be finalized?