gdb [1] is great, but it's way too easy to move past a point of interest (e.g. "darn I used next
when I
meant step
"), forget to observe something, or just not remember all of the details that you might have
been observing.
rr
lets you "go backwards" or, put differently, grants the power of "do over".
In more detail:
-
Missed a return value while observing execution? Use
reverse-step
orreverse-next
to "go back" far enough and usefinish
to see the missed return value. -
Choosing an initial breakpoint for investigation isn't as tough because one can
reverse-continue
to a subsequently set breakpoint that is "behind" where one is currently. -
It's much easier to get a better idea concerning the qusetion "how did we end up here?".
-
Recorded content may be shareable with others -- see
rr pack
.
$ which rr
/usr/bin/rr
$ sudo dmesg | grep PMU
[ 0.690026] Performance Events: PEBS fmt3+, Skylake events, 32-deep LBR, full-width counters, Intel PMU driver.
[ 0.690026] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 5.590973] RAPL PMU: API unit is 2^-32 Joules, 5 fixed counters, 655360 ms ovfl timer
[ 5.590976] RAPL PMU: hw unit of domain pp0-core 2^-14 Joules
[ 5.590977] RAPL PMU: hw unit of domain package 2^-14 Joules
[ 5.590978] RAPL PMU: hw unit of domain dram 2^-14 Joules
[ 5.590979] RAPL PMU: hw unit of domain pp1-gpu 2^-14 Joules
[ 5.590980] RAPL PMU: hw unit of domain psys 2^-14 Joules
$ sysctl kernel.perf_event_paranoid
kernel.perf_event_paranoid = 4
$ sudo sysctl kernel.perf_event_paranoid=1
$ cd ~/src/janet.debug
$ make
$ echo "3 8" > test-parse.janet
$ cat test-parse.janet
3 8
$ rr record ./build/janet test-parse.janet
rr: Saving execution to trace directory `/home/user/.local/share/rr/janet-0'.
$ rr replay
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/user/.local/share/rr/janet-0/mmap_hardlink_4_janet...
Really redefine built-in command "restart"? (y or n) [answered Y; input not from terminal]
Remote debugging using 127.0.0.1:24436
Reading symbols from /lib64/ld-linux-x86-64.so.2...
Reading symbols from /usr/lib/debug/.build-id/61/ef896a699bb1c2e4e231642b2e1688b2f1a61e.debug...
BFD: warning: system-supplied DSO at 0x6fffd000 has a section extending past end of file
0x00007f68924162b0 in _start () from /lib64/ld-linux-x86-64.so.2
(rr) b src/core/parse.c:janet_parser_consume
Breakpoint 1 at 0x5645f031cd69: file src/core/parse.c, line 701.
(rr) c
Continuing.
Breakpoint 1, janet_parser_consume (parser=0x5645f1bd5050, c=51 '3') at src/core/parse.c:701
701 int consumed = 0;
(rr) l
696 }
697
698 /* Public API */
699
700 void janet_parser_consume(JanetParser *parser, uint8_t c) {
701 int consumed = 0;
702 janet_parser_checkdead(parser);
703 if (c == '\r') {
704 parser->line++;
705 parser->column = 0;
(rr) step
702 janet_parser_checkdead(parser);
(rr) reverse-step
Breakpoint 1, janet_parser_consume (parser=0x55a77e648050, c=51 '3') at src/core/parse.c:701
701 int consumed = 0;
(rr) reverse-step
cfun_parse_consume (argc=3, argv=0x55a77e638700) at src/core/parse.c:931
931 janet_parser_consume(p, view.bytes[i]);
(rr) l
926 view.len -= offset;
927 view.bytes += offset;
928 }
929 int32_t i;
930 for (i = 0; i < view.len; i++) {
931 janet_parser_consume(p, view.bytes[i]);
932 switch (janet_parser_status(p)) {
933 case JANET_PARSE_ROOT:
934 case JANET_PARSE_PENDING:
935 break;
(rr) quit
A debugging session is active.
Inferior 1 [process 25112] will be detached.
Quit anyway? (y or n) y
Detaching from program: /home/user/.local/share/rr/janet-1/mmap_hardlink_4_janet, process 25112
[Inferior 1 (process 25112) detached]
$ sudo sysctl kernel.perf_event_paranoid=4
Debugging a program in reverse is the most natural way to debug software. The cause of the bug is somewhere in the past, and the developer starts off from the bug manifestation. What the developer is actually doing is trying to get to the source of the bug by going bit by bit to the origin of the bug.
via: https://johnnysswlab.com/rr-the-magic-of-recording-and-replay-debugging/
And rr has quickly become the number one tool I reach for when debugging complicated C++ code. rr only runs on Linux and I don't even use Linux as my day-to-day operating system! But rr provides such a great debugging experience, and gives me such a huge productivity boost, that I will reboot into Fedora just to use rr for all but the most trivial bugs.
via: https://fitzgeraldnick.com/2015/11/02/back-to-the-futurre.html
- https://rr-project.org/
- https://github.com/rr-debugger/rr/wiki/Usage
- https://github.com/rr-debugger/rr/wiki/Related-work
- https://github.com/rr-debugger/rr/wiki/Testimonials
- http://notes.secretsauce.net/notes/2017/02/24_interfacing-rr-to-gdb-in-gnu-emacs.html ->
M-x gdb
,rr replay -i=mi
- https://johnnysswlab.com/rr-the-magic-of-recording-and-replay-debugging/
- https://fitzgeraldnick.com/2015/11/02/back-to-the-futurre.html
- https://www.moviequotedb.com/movies/real-genius/quote_19420.html
[1] lldb is nice too! Haven't tried, but apaprently someone has rr and lldb working together. [2] https://github.com/rr-debugger/rr/wiki/Will-rr-work-on-my-system