Debug segfaults that only happen outside of gdb

In some rare cases, you may encounter bugs that happen only in normal execution, but not with GDB. It's really annoying; if you try to investigate it, it just disappears. Such bugs are called Haisenbug, and you can debug it with GDB by using a trick described here.

The idea is using signal handlers in programs. When SIGSEGV is caught, a signal handler is called. You can sleep for a long time in the signal handler, and then attach the program by using gdb attach.

You can define signal handlers as following:

void segv_handler(int sig) {
  sleep(100000000);
}

then register the signal handler segv_handler by using sigaction:

struct sigaction sa;
sa.sa_flags   = 0;
sa.sa_handler = segv_handler;
sigemptyset(&sa.sa_mask);
sigaction(SIGSEGV, &sa, NULL);

When the program starts to sleep, you can attach the program by

$ gdb attach <pid>

sudo might be required.

Sample code:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <signal.h>

void segv_handler(int sig) {
  /* printf() cannot be used because it's not async-signal-safe */
  write(2, "segfault\n", 9);
  sleep(100000000);
}

int main() {
  /* register a signal handler for SIGSEGV */
  struct sigaction sa;
  sa.sa_flags   = 0;
  sa.sa_handler = segv_handler;
  sigemptyset(&sa.sa_mask);
  if (sigaction(SIGSEGV, &sa, NULL) == -1) {
    perror("sigaction");
    exit(1);
  }

  /* raise SIGSEGV */
  int* a = 0;
  *a = 1;

  return 0;
}

s417-lama/debug_segfaults_that_only_happen_outside_of_gdb.md

Debug segfaults that only happen outside of gdb