This is reproduced on Visual Studio 2017 15.7.1.
- Download Angelscript from here: http://www.angelcode.com/angelscript/sdk/files/angelscript_2.32.0.zip
- Extract it to some folder, for me this is
D:\Dev\msvc_bug_test
- Open
sdk/angelscript/projects/msvc2015/angelscript.sln
in Visual Studio 2017 - It will ask to upgrade the project to SDK 10.0.17134.0 and toolset v141, so we do that
- We switch the build to Release and build the project
- Close the Visual Studio solution, and make a new solution/project, I'll call it
bugtest
- Switch to Release configuration
- Switch the runtime library to
/MT
in order to match Angelscript - Add include/library directories to the project:
d:\Dev\msvc_bug_test\sdk\angelscript\include\
d:\Dev\msvc_bug_test\sdk\angelscript\lib\
- Create a new file main.cpp and add this code:
#include <cstdio>
#include <cassert>
#include <angelscript.h>
#pragma comment(lib, "angelscript.lib")
static void MessageCallback(const asSMessageInfo* msg, void* param)
{
printf("%s\n", msg->message);
}
int main()
{
int r = 0;
asIScriptEngine* engine = asCreateScriptEngine();
r = engine->SetMessageCallback(asFUNCTION(MessageCallback), nullptr, asCALL_CDECL); assert(r >= 0);
asIScriptModule* mod = engine->GetModule("Scripts", asGM_CREATE_IF_NOT_EXISTS);
r = mod->AddScriptSection("Test", "class Foo { void Bar() { } } void Main() { Foo f; f.DoesNotExist(); }"); assert(r >= 0);
r = mod->Build();
return 0;
}
- Put a breakpoint on the
printf
call inMessageCallback
- Run the application, the breakpoint should get hit
- Switch to disassembly view
- Step out of the callback function
- Observe return assembly: (this is where the optimization bug occurs)
00340EB8 FF 75 0C push dword ptr [param2]
00340EBB FF 75 08 push dword ptr [param1]
00340EBE FF D6 call esi
}
}
00340EC0 8B 4D F4 mov ecx,dword ptr [ebp-0Ch]
00340EC3 64 89 0D 00 00 00 00 mov dword ptr fs:[0],ecx
00340ECA 5E pop esi
00340ECB 8B E5 mov esp,ebp
00340ECD 5D pop ebp
00340ECE C2 10 00 ret 10h
- Note that
pop esi
is supposed to pop the thisptr back. Instead, it's the value ofparam1
- Step out of the function
- Observe the return assembly:
preMessage.isSet = false;
00348A86 8B 45 08 mov eax,dword ptr [section]
00348A89 C6 86 BC 0B 00 00 00 mov byte ptr [esi+0BBCh],0
- Note that
preMessage
is a field in the class, andesi
here is used as the thisptr to set it to false, but esi is not the actual thisptr, it's some value on the stack