infinite-loop-trick.log

$ cat test.js
function foo () { while (true) { } }
function bar () { return foo(); }
bar();
$ node test.js &
$ gdb attach $(pidof node)
0x00000bf778c63d5f in ?? ()
(gdb) b v8::internal::Runtime_StackGuard
Breakpoint 1 at 0x84a1f0
(gdb) print 'v8::V8::TerminateExecution'(0)
$2 = 0
(gdb) c
Continuing.

Breakpoint 1, 0x000000000084a1f0 in v8::internal::Runtime_StackGuard(v8::internal::Arguments, v8::internal::Isolate*) ()
(gdb) print V8_Fatal("a", 11, "c")


#
# Fatal error in a, line 11
# c
#


==== Stack trace ============================================

Security context: 0x11330e857229 <JS Object>#0#
    1: foo(aka foo) [/home/mraleph/test.js:~1] (this=0)
    2: bar [/home/mraleph/test.js:2] (this=0x11330e857349 <JS Global Object>#1#)
    3: /* anonymous */ [/home/mraleph/test.js:3] (this=0xb6d64f79f11 <an Object>#2#,exports=0xb6d64f79f11 <an Object>#2#,require=0xb6d64f7c651 <JS Function require>#3#,module=0xb6d64f79e19 <a Module>#4#,__filename=0xb6d64f76a69 <String[38]: /home/mraleph/test.js>,__dirname=0xb6d64f7cfb1 <String[30]: /home/mraleph>)

/// and so forth

infinite-loop-trick.md

This is a step by step hack to get JavaScript stack trace from a node process that is stuck in the infinite loop.

Imagine you have a JavaScript code with an infinite loop

$ cat test.js
function foo () { while (true) { } }
function bar () { return foo(); }
bar();

You node process will get stuck if you try to run it

$ node test.js &

How can you get the stacktrace to troubleshoot the issue? First attach GDB debugger to the node process:

$ gdb attach $(pidof node)

It will break execution in some random place of your infinite loop:

0x00000bf778c63d5f in ?? ()

Unfortunately GDB does not know anything about native code that V8 generated from JavaScript source so it can't print stack trace itself:

(gdb) bt
#0  0x00000bf778c63d5f in ?? ()
#1  0x00000000beeddead in ?? ()
#2  0x00000000beeddead in ?? ()
#3  0x000011330e857349 in ?? ()
#4  0x00000b6d64f7d231 in ?? ()
#5  0x00000b6d64f7d1f9 in ?? ()
#6  0x00007fffffffde30 in ?? ()
#7  0x00000bf778c63a12 in ?? ()
#8  0x0000000000000000 in ?? ()

Things we can do with the paused node process are additionally limited by absence of many symbols in the debug build.

V8 has built-in facilities that can traverse the stack and print the trace, however they can't be used if execution is paused in an arbitrary place, it has to reach a safepoint where the stack is in consistent state. Each loop compiled by V8 in fact has such a safepoint at the back edge of the loop called a stack guard. Stack guards are used to implement internal interrupt system used by V8.

Lets put a break point into a function that handles stack guard interrupts

(gdb) b v8::internal::Runtime_StackGuard
Breakpoint 1 at 0x84a1f0

schedule an interrupt

(gdb) print 'v8::V8::TerminateExecution'(0)
$2 = 0

and continue execution.

(gdb) c
Continuing.

Once execution is resumed it is bound to hit stack guard, discover a pending interrupt and enter Runtime_StackGuard to handle it.

Breakpoint 1, 0x000000000084a1f0 in v8::internal::Runtime_StackGuard(v8::internal::Arguments, v8::internal::Isolate*) ()

Now we are in a place where V8 itself can traverse stack and print stack trace for us. Unfortunately functions that do that are not visible to the debugger in the release build of V8. However there is a function V8_Fatal which is used to report fatal errors in V8 and terminate V8 process. It also prints full stack trace before terminating the process to assist debugging.

(gdb) print V8_Fatal("a", 11, "c")


#
# Fatal error in a, line 11
# c
#


==== Stack trace ============================================

Security context: 0x11330e857229 <JS Object>#0#
  1: foo(aka foo) [/home/mraleph/test.js:~1] (this=0)
  2: bar [/home/mraleph/test.js:2] (this=0x11330e857349 <JS Global Object>#1#)
  3: /* anonymous */ [/home/mraleph/test.js:3] (this=0xb6d64f79f11 <an Object>#2#,exports=0xb6d64f79f11 <an Object>#2#,require=0xb6d64f7c651 <JS Function require>#3#,module=0xb6d64f79e19 <a Module>#4#,__filename=0xb6d64f76a69 <String[38]: /home/mraleph/test.js>,__dirname=0xb6d64f7cfb1 <String[30]: /home/mraleph>)

Here is our stack trace. Notice that it is the process itself that prints the trace, not GDB so it will go whereever your stderr is redirected (e.g. forever will redirect it to a log file).

The above code didn't work for me in Node.js 4.7.3 / V8 4.5.103.43. This command does the job:

gdb -batch \
    -ex "b v8::internal::Runtime_StackGuard" \
    -ex "p 'v8::Isolate::GetCurrent'()" \
    -ex "p 'v8::Isolate::TerminateExecution'(\$1)" \
    -ex "c" \
    -ex "p printf(\"FATAL ERROR: Application is not responding and will be killed\")" \
    -ex "p 'v8::internal::Runtime_DebugTrace'(0, 0, (void *)(\$1))" \
    attach ${PID}

Note that ${PID} is a variable holding the PID of your process.

I ran into a really hard to diagnose infinite loop due to a timing issue. It reproduces only on really fast machines due to timing and if i try to open dev tools and run the app, the hang doesn't happen as the timing changes. This is the renderer process in an electron application. There are no logs, I'm not able to attach dev tools once the app goes into an infinite loop. I tried attaching to visual studio when the issue occurs but i only get callstacks with v8 and blink, and of course none of the javascript callstack is available. Does the methology described here work on windows with visual studio or windbg? Does anyone know how i can get the js callstack on windbg or vs2019?