Fix IDA 7.5/7.6/7.7SP1 crashing on idapython3.dll in Wine

idafix.md

Description

For some reason IDA executes FreeLibrary() to the plugin immediately after getting its PLUGIN structure's address, so later invocations of the plugin lead to calls to nowhere (that was supposed to be python3.dll). Simply patching the location of the FreeLibrary() call fixes the issue. The location is easy to find: go by cross-references to a place where the call to FreeLibrary is followed by a reference to the string "%s: incompatible plugin version..." and NOP it away.

7.5

ida.dll

+001c1d20  15 9b e6 e3 ff 48 8b 4d  88 48 85 c9 74 0e 90 90  |.....H.M.H..t...|
+001c1d30  90 90 90 90 48 c7 45 88  00 00 00 00 48 8b 4c 24  |....H.E.....H.L$|

ida64.dll

+001cb050  15 83 53 e3 ff 48 8b 4d  88 48 85 c9 74 0e 90 90  |..S..H.M.H..t...|
+001cb060  90 90 90 90 48 c7 45 88  00 00 00 00 48 8b 4c 24  |....H.E.....H.L$|

7.6

ida.dll

+001cb6f0  15 03 4d e3 ff 48 8b 4d  88 48 85 c9 74 0e 90 90  |..M..H.M.H..t...|
+001cb700  90 90 90 90 48 c7 45 88  00 00 00 00 48 8b 4c 24  |....H.E.....H.L$|

ida64.dll

+001d53f0  15 0b b0 e2 ff 48 8b 4d  88 48 85 c9 74 0e 90 90  |.....H.M.H..t...|
+001d5400  90 90 90 90 48 c7 45 88  00 00 00 00 48 8b 4c 24  |....H.E.....H.L$|

7.7SP1

ida.dll Patching not required.

-EAA2A     00 48 85 c9 74 0a ff 15  ea 0d
+EAA2A     00 48 85 c9 eb 0a ff 15  ea 0d

ida64.dll

-EEC80     ff 15 9a 5b 26 00 48 89  7d 00
+EEC80     90 90 90 90 90 90 90 90  90 90

Hey, would really appreciate it if you could provide a more detailed walkthrough on how to patch and set up wine and python3 to work correctly, primarily for 7.7 SP1

First - you should disable idapython - rename ~/${IDA_WINE_PREFIX}/drive_c/Program Files/IDA 7.7/plugins/idapython3_64.dll to ~/${IDA_WINE_PREFIX}/drive_c/Program Files/IDA 7.7/plugins/idapython3_64.dll.disabled.

Open IDA 64bit, load DLL (ida.dll or ida64.dll) with default settings, reject debug info download.
Then search for a sequence of bytes "incompatible plugin version" (with quotes).
Double-click the only search result and press 'X' to go to xrefs to this string.
Select the only xref and press OK.
Press F5. Dismiss all dialogue windows with 'Ok' button.
You will see something like this:

    if ( hLibModule )
    {
      FreeLibrary(hLibModule);
      hLibModule = 0i64;
    }
    qfree((void *)lpPathName);
    if ( *v47 != 700 )
      sub_100002D40("%s: incompatible plugin version, skipped\n", v8);

The FreeLibrary call and hLibModule assignment are bogus, and should be patched out.

So, click on FreeLibrary(hLibModule); line, then click on IDA View A tab and right-click anywhere in disassembly. Select Synchronize with -> Pseudocode A.
You should see something like this:

.text:00000001000EB627 loc_1000EB627:                          ; CODE XREF: sub_1000EAF20+6F0↑j
.text:00000001000EB627                 mov     rcx, [rbp+140h+hLibModule] ; hLibModule
.text:00000001000EB62B                 test    rcx, rcx
.text:00000001000EB62E                 jz      short loc_1000EB63A
.text:00000001000EB630                 call    cs:FreeLibrary
.text:00000001000EB636                 mov     [rbp+140h+hLibModule], rdi

The "jz short loc_1000EB63A" should be patched to "jmp short loc_1000EB63A".
So, click on "jz short loc_1000EB63A" line and select "Edit->Patch program->Assemble..." from window menu.
You should see a dialogue window with an Instruction field, and "jz short loc_1000EB63A" in this field. Change it to "jmp short loc_1000EB63A" and press Ok, then press Cancel.
You may now return to Pseudocode A tab and make sure that bogus code block is now gone.
When DLL was fixed - save it using "Edit->Patch program->Apply patches to input file..." from window menu. Select "Create backup" and press OK, then close IDA without saving the database.
Congratulations, you have patched your IDA to fix this bug!
Now you should enable idapython - rename ~/${IDA_WINE_PREFIX}/drive_c/Program Files/IDA 7.7/plugins/idapython3_64.dll.disabled back to ~/$(IDA_WINE_PREFIX/drive_c/Program Files/IDA 7.7/plugins/idapython3_64.dll.

For Python - just install Python for Windows to the same Wine prefix.
You may also need to run ~/${IDA_WINE_PREFIX}/drive_c/Program Files/IDA 7.7/idapyswitch.exe and select your Python.

Also, for Wine - go to Options->Colors, select dark color theme and click OK. This will fix black interface parts (which became an issue with 7.7SP1 on Wine).

Okay, let me give it a try right now.

Thank you, it worked 😅