luajit-crash-on-first-instruction-of-mcode.txt

mail message modified:

Xiaochen Wang <[email protected]>
11:07 AM (2 hours ago)
to web-07

hi mike,

We still have an issue, always crashed on first instruction of tracing generated machine code.

(gdb) bt
#0  0x000000001f6b6b23 in ?? ()
#1  0x000000000070b660 in ?? () at lib_string.c:285
#2  0x00000000006d912b in lj_BC_FUNCC () at buildvm_x86.dasc:809
#3  0x00000000005759dc in ngx_http_lua_run_thread (L=L@entry=0x41755378, r=r@entry=0x7f4dbf60a400, ctx=ctx@entry=0x7f4ff5d82b40, nrets=nrets@entry=0) at mod_lua/src/ngx_http_lua_util.c:1015
#4  0x000000000058fdf2 in ngx_http_lua_ssl_cert_by_chunk (L=0x41755378, r=0x7f4dbf60a400) at mod_lua/src/ngx_http_lua_ssl_certby.c:531
#5  0x00000000005906a6 in ngx_http_lua_ssl_cert_handler (ssl_conn=ssl_conn@entry=0x7f4dcc810000, data=<optimized out>) at mod_lua/src/ngx_http_lua_ssl_certby.c:319
...

(gdb) ltracebymcode 0x000000001f6b6b23
(GCtrace*)0x42c83538 (trace #432)
machine code start addr: 0x1f6b6b23
machine code end addr: 0x1f6b6d8a
@/home/admin/tengine/conf/dycert/dycert/cert.lua:1037
^-------------------------------------------- frame 0 address is machine code start address. 

(gdb) x/100i $0
The history is empty.
(gdb) x/100i 0x000000001f6b6b23
=> 0x1f6b6b23:  mov    0x417554b0,%eax 
                           ^----------------- crashed on first instruction of tracing generated machine code (I found that this mcode is generated by  first SLOAD IR of traced lua code)

   0x1f6b6b2a:  mov    0x20(%rax),%eax
   0x1f6b6b2d:  sub    %edx,%eax
   0x1f6b6b2f:  cmp    $0x78,%eax
...

According to kernel dmesg error line, we found that error is 15, meaning memory address 0x417554b0 is protected fault.
 *   bit 0 ==    1: protection fault
 *   bit 1 ==    1: write access
 *   bit 2 ==    1: user-mode access
 *   bit 3 ==    1: use of reserved bit detected

dmesg: t-coresystem-te[111557]: segfault at 1f6b6b23 ip 000000001f6b6b23 sp 00007fff8c50e270 error 15

BTW, we found many coredumps are similar to this, they all crashed on first instruction of tracing-generated machine code ( for different traced lua code).

segfault is "at 1f6b6b23" ,  not at 0x417554b0 (&g.vmstate), this is mcode first instruction.

(gdb) maintenance info sections
...
    0x1f670000->0x1f6f0000 at 0x000ba000: load5 ALLOC LOAD READONLY CODE HAS_CONTENTS  
    ^------------------------- for 0x1f6b6b23 (mcode first instruction)
...
    0x41755000->0x41775000 at 0x014b0000: load87 ALLOC LOAD HAS_CONTENTS 
    ^------------------------- for 0x417554b0 (&g.vmstate)
...



CPU info:
===========
#lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                48
On-line CPU(s) list:   0-47
Thread(s) per core:    2
Core(s) per socket:    24
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz
Stepping:              4
CPU MHz:               2501.000
BogoMIPS:              4998.90
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              1024K
L3 cache:              33792K
NUMA node0 CPU(s):     0-47

OS info
=======
$ uname -a
3.10.0-1548558.327.alicdn.alios7.x86_64 #1 SMP Thu Jan 25 10:26:17 CST 2018 x86_64 x86_64 x86_64 GNU/Linu

same issue posted on luajit mailing list: https://www.freelists.org/post/luajit/SIGSEGV-in-LuaJIT-21-VM,11
( same issue posted on stackoverflow: https://stackoverflow.com/questions/40609455/why-is-this-mov-causing-a-sigsegv%C2%A0 )

log: ____9._1284