Skip to content

Instantly share code, notes, and snippets.

View mranney's full-sized avatar

Matt Ranney mranney

View GitHub Profile
@mranney
mranney / gist:7834019
Created December 6, 2013 23:38
Using DTrace to catch a really tricky bug in production
dtrace -n 'syscall::writev:entry /execname == "node" && arg2 > 3/ { printf("iovcnt: %d", arg2); }' -n 'syscall::writev:return /execname == "node" && arg1 == -1 && errno == 22/ { printf("%s %s %d\\n", curpsinfo->pr_psargs, probefunc, errno); jstack(100, 8000); }'
dtrace: description 'syscall::writev:entry ' matched 1 probe
dtrace: description 'syscall::writev:return ' matched 1 probe
1 10260 writev:entry iovcnt: 57
1 10261 writev:return node --nouse-idle-notification /voxer/server/node_router/node_router.js /voxer/ writev 22\n
libc.so.1`__writev+0x15
node`uv__write.part.5+0x18f
node`uv_write2+0x1b9
node`uv_write+0x35
node`_ZN4node10StreamWrap6WritevERKN2v89ArgumentsE+0x678
background(117,177,220);
stroke(31,134,46);
fill(31,134,46);
rect(450, 5, 10, 100);
rect(50, 200, 10, 100);
fill(56,42,237);
> var b = []; for (var i = 0; i < 1024 * 1024 ; i++) { b.push(new Buffer(1024)); }
1048576
> process.memoryUsage()
{ rss: 141910016,
heapTotal: 104296088,
heapUsed: 101654176 }
> var b = []; for (var i = 0; i < 1024 * 1024 ; i++) { b.push(new Buffer(2048)); }
1048576
> process.memoryUsage()
{ rss: 778919936,
mjr@prod-2119:~ [zookeeper]$ gdb /opt/local/bin/node ./core.82428
GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-sun-solaris2".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
> ::fpregs
_fp_hw 0x03 (80387 chip with SSE)
cw 0x0c7f (IM|DM|ZM|OM|UM|PM|RC|RTZ|P)
sw 0x0c77 (TOP=0t1) (IE|DE|ZE|UE|PE|SF|C2)
xcp sw 0x0000 (0)
ipoff port_poll+0x49
cssel 0x0
dtoff 0x8047bd8
dtsel 0x0
> *0x87ebddc
libm.so.2`floor: subl $0x8,%esp
libm.so.2`floor+3: fwait
libm.so.2`floor+4: fnstcw (%esp)
libm.so.2`floor+7: fldl 0xc(%esp)
libm.so.2`floor+0xb: movw (%esp),%cx
libm.so.2`floor+0xf: orw $0xc00,%cx
libm.so.2`floor+0x14: xorw $0x800,%cx
libm.so.2`floor+0x19: movw %cx,0x4(%esp)
libm.so.2`floor+0x1e: fldcw 0x4(%esp)
> libm.so.2`floor::dis
libm.so.2`floor: subl $0x8,%esp
libm.so.2`floor+3: fwait
libm.so.2`floor+4: fnstcw (%esp)
libm.so.2`floor+7: fldl 0xc(%esp)
libm.so.2`floor+0xb: movw (%esp),%cx
libm.so.2`floor+0xf: orw $0xc00,%cx
libm.so.2`floor+0x14: xorw $0x800,%cx
libm.so.2`floor+0x19: movw %cx,0x4(%esp)
libm.so.2`floor+0x1e: fldcw 0x4(%esp)
> _ZN2v88internal23Runtime_DateCurrentTimeENS0_9ArgumentsEPNS0_7IsolateE-0x3219db::dis
PLT:sin: *ERROR--unknown op code**
PLT:pow: jmp *0x87ebdd8
PLT:pow: pushl $0x7d0
PLT:pow: jmp -0xfc0 <0x81de188>
PLT=libm.so.2`floor: jmp *0x87ebddc
PLT=libm.so.2`floor: pushl $0x7d8
PLT=libm.so.2`floor: jmp -0xfd0 <0x81de188>
PLT:fmod: jmp *0x87ebde0
PLT:fmod: pushl $0x7e0
@mranney
mranney / gist:5549123
Created May 9, 2013 17:38
Date.now() gets stuck after a while and then only updates slowly.
djg@prod-2122:~$ sudo rc /tmp/repl/prod-metricsserver4.sock
prod-metricsserver4> Date.now()
1368115118080
prod-metricsserver4> Date.now()
1368115118080
prod-metricsserver4> Date.now()
1368115118080
.... much time passes ...
make[1]: Entering directory `/voxer/deploy/node/out'
cc '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' '-DL_ENDIAN' '-DOPENSSL_THREADS' '-DPURIFY' '-D_REENTRANT' '-DOPENSSL_NO_DTLS1' '-DOPENSSL_NO_SOCK' '-DOPENSSL_NO_DGRAM' '-DOPENSSL_NO_GOST' '-DOPENSSL_NO_HW_PADLOCK' '-DOPENSSL_NO_TTY' '-DAES_ASM' '-DBF_ASM' '-DBNCO_ASM' '-DBN_ASM' '-DCPUID_ASM' '-DDES_ASM' '-DLIB_BN_ASM' '-DOPENSSL_BN_ASM' '-DOPENSSL_CPUID_OBJ' '-DRIP_ASM' '-DWHIRLPOOL_ASM' '-DWP_ASM' '-DENGINESDIR="/dev/null"' '-DOPENSSLDIR="/etc/ssl"' '-DTERMIOS' '-D__EXTENSIONS__' -I../deps/openssl -I../deps/openssl/openssl -I../deps/openssl/openssl/crypto -I../deps/openssl/openssl/crypto/asn1 -I../deps/openssl/openssl/crypto/evp -I../deps/openssl/openssl/crypto/store -I../deps/openssl/openssl/include -I../deps/openssl/config/k8 -Wall -m64 -pthreads -O2 -fno-strict-aliasing -ffunction-sections -fdata-sections -fno-tree-vrp -fno-omit-frame-pointer -MMD -MF /voxer/deploy/node/out/Release/.deps//voxer/deploy/node/out/Release/obj.target/openssl/deps/ope