Skip to content

Instantly share code, notes, and snippets.

@flavorjones
Last active June 19, 2021 16:14
Show Gist options
  • Save flavorjones/ad958336c7fd09c92cee3a9bd911d628 to your computer and use it in GitHub Desktop.
Save flavorjones/ad958336c7fd09c92cee3a9bd911d628 to your computer and use it in GitHub Desktop.
reproduce Ruby memory usage error in `iseq_peephole_optimize`

This gist reproduces a memory usage error in Ruby's iseq_peephole_optimize.

  1. Compile Ruby with ASan (using CFLAGS="-fsanitize=address -ggdb")
  2. Turn off memory leak detection (which is not relevant)export ASAN_OPTIONS=detect_leaks=0
  3. Run ruby ./repro.rb

You should see something like:

before require
=================================================================
==862472==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6160005b2590 at pc 0x558b3ec6bd8d bp 0x7fffb9eb3280 sp 0x7fffb9eb3270
READ of size 4 at 0x6160005b2590 thread T0
    #0 0x558b3ec6bd8c in iseq_peephole_optimize /home/flavorjones/code/oss/ruby/compile.c:3091
    #1 0x558b3ec6f6ed in iseq_optimize /home/flavorjones/code/oss/ruby/compile.c:3576
    #2 0x558b3ec61c76 in iseq_setup_insn /home/flavorjones/code/oss/ruby/compile.c:1452
    #3 0x558b3ec5ef9e in rb_iseq_compile_node /home/flavorjones/code/oss/ruby/compile.c:855
    #4 0x558b3e852e2e in rb_iseq_new_with_opt /home/flavorjones/code/oss/ruby/iseq.c:891
    #5 0x558b3ec612e2 in new_child_iseq /home/flavorjones/code/oss/ruby/compile.c:1335
    #6 0x558b3ec867c2 in compile_iter /home/flavorjones/code/oss/ruby/compile.c:6877
    #7 0x558b3ec8e6b8 in iseq_compile_each0 /home/flavorjones/code/oss/ruby/compile.c:7936
    #8 0x558b3ec8de78 in iseq_compile_each /home/flavorjones/code/oss/ruby/compile.c:7863
    #9 0x558b3ec5e857 in rb_iseq_compile_node /home/flavorjones/code/oss/ruby/compile.c:795
    #10 0x558b3e852e2e in rb_iseq_new_with_opt /home/flavorjones/code/oss/ruby/iseq.c:891
    #11 0x558b3ec612e2 in new_child_iseq /home/flavorjones/code/oss/ruby/compile.c:1335
    #12 0x558b3ec95f42 in iseq_compile_each0 /home/flavorjones/code/oss/ruby/compile.c:8884
    #13 0x558b3ec8de78 in iseq_compile_each /home/flavorjones/code/oss/ruby/compile.c:7863
    #14 0x558b3ec8e474 in iseq_compile_each0 /home/flavorjones/code/oss/ruby/compile.c:7908
    #15 0x558b3ec8de78 in iseq_compile_each /home/flavorjones/code/oss/ruby/compile.c:7863
    #16 0x558b3ec5e71d in rb_iseq_compile_node /home/flavorjones/code/oss/ruby/compile.c:787
    #17 0x558b3e852e2e in rb_iseq_new_with_opt /home/flavorjones/code/oss/ruby/iseq.c:891
    #18 0x558b3ec612e2 in new_child_iseq /home/flavorjones/code/oss/ruby/compile.c:1335
    #19 0x558b3ec96815 in iseq_compile_each0 /home/flavorjones/code/oss/ruby/compile.c:8949
    #20 0x558b3ec8de78 in iseq_compile_each /home/flavorjones/code/oss/ruby/compile.c:7863
    #21 0x558b3ec8e474 in iseq_compile_each0 /home/flavorjones/code/oss/ruby/compile.c:7908
    #22 0x558b3ec8de78 in iseq_compile_each /home/flavorjones/code/oss/ruby/compile.c:7863
    #23 0x558b3ec5e963 in rb_iseq_compile_node /home/flavorjones/code/oss/ruby/compile.c:801
    #24 0x558b3e852e2e in rb_iseq_new_with_opt /home/flavorjones/code/oss/ruby/iseq.c:891
    #25 0x558b3e85298b in rb_iseq_new_top /home/flavorjones/code/oss/ruby/iseq.c:838
    #26 0x558b3e866037 in load_iseq_eval /home/flavorjones/code/oss/ruby/load.c:589
    #27 0x558b3e868c8e in require_internal /home/flavorjones/code/oss/ruby/load.c:1070
    #28 0x558b3e869461 in rb_require_string /home/flavorjones/code/oss/ruby/load.c:1148
    #29 0x558b3e8675db in rb_f_require_relative /home/flavorjones/code/oss/ruby/load.c:857
    #30 0x558b3eb819ed in ractor_safe_call_cfunc_1 /home/flavorjones/code/oss/ruby/vm_insnhelper.c:2847
    #31 0x558b3eb838e5 in vm_call_cfunc_with_frame /home/flavorjones/code/oss/ruby/vm_insnhelper.c:3023
    #32 0x558b3eb8cc36 in vm_sendish /home/flavorjones/code/oss/ruby/vm_insnhelper.c:4596
    #33 0x558b3eb9ac52 in vm_exec_core /home/flavorjones/code/oss/ruby/insns.def:775
    #34 0x558b3ebc9289 in rb_vm_exec /home/flavorjones/code/oss/ruby/vm.c:2173
    #35 0x558b3ebcb906 in rb_iseq_eval_main /home/flavorjones/code/oss/ruby/vm.c:2421
    #36 0x558b3e79b431 in rb_ec_exec_node /home/flavorjones/code/oss/ruby/eval.c:317
    #37 0x558b3e79b755 in ruby_run_node /home/flavorjones/code/oss/ruby/eval.c:375
    #38 0x558b3e783758 in main main.c:47
    #39 0x7f2be47930b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
    #40 0x558b3e78356d in _start (/home/flavorjones/.rbenv/versions/3.1.0.dev-with-asan/bin/ruby+0xf656d)

0x6160005b2590 is located 0 bytes to the right of 528-byte region [0x6160005b2380,0x6160005b2590)
allocated by thread T0 here:
    #0 0x7f2be4ccbbc8 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
    #1 0x558b3e7ec8e0 in objspace_xmalloc0 /home/flavorjones/code/oss/ruby/gc.c:11379
    #2 0x558b3e7ecc9f in ruby_xmalloc2_body /home/flavorjones/code/oss/ruby/gc.c:11623
    #3 0x558b3e7f466f in ruby_xmalloc2 /home/flavorjones/code/oss/ruby/gc.c:13571
    #4 0x558b3e8508ea in new_arena /home/flavorjones/code/oss/ruby/iseq.c:581
    #5 0x558b3e850c93 in prepare_iseq_build /home/flavorjones/code/oss/ruby/iseq.c:619
    #6 0x558b3e852e11 in rb_iseq_new_with_opt /home/flavorjones/code/oss/ruby/iseq.c:889
    #7 0x558b3ec612e2 in new_child_iseq /home/flavorjones/code/oss/ruby/compile.c:1335
    #8 0x558b3ec867c2 in compile_iter /home/flavorjones/code/oss/ruby/compile.c:6877
    #9 0x558b3ec8e6b8 in iseq_compile_each0 /home/flavorjones/code/oss/ruby/compile.c:7936
    #10 0x558b3ec8de78 in iseq_compile_each /home/flavorjones/code/oss/ruby/compile.c:7863
    #11 0x558b3ec5e857 in rb_iseq_compile_node /home/flavorjones/code/oss/ruby/compile.c:795
    #12 0x558b3e852e2e in rb_iseq_new_with_opt /home/flavorjones/code/oss/ruby/iseq.c:891
    #13 0x558b3ec612e2 in new_child_iseq /home/flavorjones/code/oss/ruby/compile.c:1335
    #14 0x558b3ec95f42 in iseq_compile_each0 /home/flavorjones/code/oss/ruby/compile.c:8884
    #15 0x558b3ec8de78 in iseq_compile_each /home/flavorjones/code/oss/ruby/compile.c:7863
    #16 0x558b3ec8e474 in iseq_compile_each0 /home/flavorjones/code/oss/ruby/compile.c:7908
    #17 0x558b3ec8de78 in iseq_compile_each /home/flavorjones/code/oss/ruby/compile.c:7863
    #18 0x558b3ec5e71d in rb_iseq_compile_node /home/flavorjones/code/oss/ruby/compile.c:787
    #19 0x558b3e852e2e in rb_iseq_new_with_opt /home/flavorjones/code/oss/ruby/iseq.c:891
    #20 0x558b3ec612e2 in new_child_iseq /home/flavorjones/code/oss/ruby/compile.c:1335
    #21 0x558b3ec96815 in iseq_compile_each0 /home/flavorjones/code/oss/ruby/compile.c:8949
    #22 0x558b3ec8de78 in iseq_compile_each /home/flavorjones/code/oss/ruby/compile.c:7863
    #23 0x558b3ec8e474 in iseq_compile_each0 /home/flavorjones/code/oss/ruby/compile.c:7908
    #24 0x558b3ec8de78 in iseq_compile_each /home/flavorjones/code/oss/ruby/compile.c:7863
    #25 0x558b3ec5e963 in rb_iseq_compile_node /home/flavorjones/code/oss/ruby/compile.c:801
    #26 0x558b3e852e2e in rb_iseq_new_with_opt /home/flavorjones/code/oss/ruby/iseq.c:891
    #27 0x558b3e85298b in rb_iseq_new_top /home/flavorjones/code/oss/ruby/iseq.c:838
    #28 0x558b3e866037 in load_iseq_eval /home/flavorjones/code/oss/ruby/load.c:589
    #29 0x558b3e868c8e in require_internal /home/flavorjones/code/oss/ruby/load.c:1070

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/flavorjones/code/oss/ruby/compile.c:3091 in iseq_peephole_optimize
Shadow bytes around the buggy address:
  0x0c2c800ae460: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2c800ae470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c2c800ae480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c2c800ae490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c2c800ae4a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c2c800ae4b0: 00 00[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2c800ae4c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2c800ae4d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c2c800ae4e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c2c800ae4f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c2c800ae500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==862472==ABORTING
STDERR.puts "in file"
class Foo
def bar
baz do |x|
next unless Integer === x
end
end
end
#! /usr/bin/env ruby
require "coverage"
Coverage.start
STDERR.puts "before require"
require_relative "foo"
STDERR.puts "after require"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment