Skip to content

Instantly share code, notes, and snippets.

@jrevels
Last active September 28, 2018 18:56
Show Gist options
  • Save jrevels/cd5660e80dc957e92d0e9e7b6da6a434 to your computer and use it in GitHub Desktop.
Save jrevels/cd5660e80dc957e92d0e9e7b6da6a434 to your computer and use it in GitHub Desktop.
$ cat ../../wip.jl
using InteractiveUtils
# code_llvm(exp, Tuple{Float64})

function f(x)
  _1 = exp(x)
  _2 = exp(x)
  return _1 + _2
end

ccall(:jl_breakpoint, Nothing, (Any,), nothing)
code_llvm(f, Tuple{Float64}; dump_module=true)

jl_breakpoint is to have a good starting point for the debug session

$ rr record ./julia ../../wip.jl
rr: Saving execution to trace directory `/home/tbesard/.local/share/rr/julia-3'.
; ModuleID = 'f'
source_filename = "f"
target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128-ni:10:11:12:13"
target triple = "x86_64-unknown-linux-gnu"

%jl_value_t = type opaque

; Function f
; Location: /home/tbesard/Julia/julia-dev/wip.jl:5
; Function Attrs: sspstrong
define double @julia_f_-1577664882(double) #0 {
top:
  %1 = call double @julia_exp_-1577664881(double %0)
; Location: /home/tbesard/Julia/julia-dev/wip.jl:6
  %2 = call double @julia_exp_-1577664881(double %0)
; Location: /home/tbesard/Julia/julia-dev/wip.jl:7
; Function +; {
; Location: float.jl:395
  %3 = fadd double %1, %2
;}
  ret double %3
}

define noalias nonnull %jl_value_t addrspace(10)* @jfptr_f_-1577664881(%jl_value_t addrspace(10)* nocapture readnone, %jl_value_t addrspace(10)** nocapture readonly, i32) #1 {
top:
  %thread_ptr = call i8* asm "movq %fs:0, $0", "=r"()
  %ptls_i8 = getelementptr i8, i8* %thread_ptr, i64 -10920
  %ptls = bitcast i8* %ptls_i8 to %jl_value_t***
  %3 = getelementptr inbounds %jl_value_t addrspace(10)*, %jl_value_t addrspace(10)** %1, i64 1
  %4 = bitcast %jl_value_t addrspace(10)** %3 to double addrspace(10)**
  %5 = load double addrspace(10)*, double addrspace(10)** %4, align 8
  %6 = addrspacecast double addrspace(10)* %5 to double addrspace(11)*
  %7 = load double, double addrspace(11)* %6, align 8
  %8 = call double @julia_f_-1577664882(double %7)
  %9 = bitcast %jl_value_t*** %ptls to i8*
  %10 = call noalias nonnull %jl_value_t addrspace(10)* @jl_gc_pool_alloc(i8* %9, i32 1424, i32 16) #2
  %11 = bitcast %jl_value_t addrspace(10)* %10 to %jl_value_t addrspace(10)* addrspace(10)*
  %12 = getelementptr %jl_value_t addrspace(10)*, %jl_value_t addrspace(10)* addrspace(10)* %11, i64 -1
  store %jl_value_t addrspace(10)* addrspacecast (%jl_value_t* inttoptr (i64 140151795587744 to %jl_value_t*) to %jl_value_t addrspace(10)*), %jl_value_t addrspace(10)* addrspace(10)* %12
  %13 = bitcast %jl_value_t addrspace(10)* %10 to double addrspace(10)*
  store double %8, double addrspace(10)* %13, align 8
  ret %jl_value_t addrspace(10)* %10
}

; Function Attrs: allocsize(1)
declare noalias nonnull %jl_value_t addrspace(10)* @julia.gc_alloc_obj(i8*, i64, %jl_value_t addrspace(10)*) #2

; Function Attrs: nounwind readnone speculatable
declare void @llvm.dbg.declare(metadata, metadata, metadata) #3

declare double @julia_exp_-1577664881(double)

; Function Attrs: argmemonly nounwind
declare void @llvm.lifetime.start.p0i8(i64, i8* nocapture) #4

; Function Attrs: argmemonly nounwind
declare void @llvm.lifetime.end.p0i8(i64, i8* nocapture) #4

; Function Attrs: allocsize(1)
declare noalias nonnull %jl_value_t addrspace(10)* @jl_gc_pool_alloc(i8*, i32, i32) #2

; Function Attrs: allocsize(1)
declare noalias nonnull %jl_value_t addrspace(10)* @jl_gc_big_alloc(i8*, i64) #2

; Function Attrs: nounwind readnone speculatable
declare void @llvm.dbg.value(metadata, metadata, metadata) #3

attributes #0 = { sspstrong }
attributes #1 = { "thunk" }
attributes #2 = { allocsize(1) }
attributes #3 = { nounwind readnone speculatable }
attributes #4 = { argmemonly nounwind }

!llvm.module.flags = !{!0, !1}
!llvm.dbg.cu = !{!2}

!0 = !{i32 2, !"Dwarf Version", i32 4}
!1 = !{i32 1, !"Debug Info Version", i32 3}
!2 = distinct !DICompileUnit(language: DW_LANG_C89, file: !3, producer: "julia", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, enums: !4)
!3 = !DIFile(filename: "/home/tbesard/Julia/julia-dev/wip.jl", directory: ".")
!4 = !{}

everything recorded. notice how the exp function doesn't have noreturn

$ rr replay
GNU gdb (GDB) 8.2
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/tbesard/Julia/julia-dev/build/debug/usr/bin/julia-debug...done.
Really redefine built-in command "restart"? (y or n) [answered Y; input not from terminal]
Remote debugging using 127.0.0.1:15821
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
0x00007f77aee7f000 in _start () from /lib64/ld-linux-x86-64.so.2

start with going to the breakpoint

(rr) break jl_breakpoint
Function "jl_breakpoint" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (jl_breakpoint) pending.
(rr) c
Continuing.
[New Thread 8780.8781]

Thread 1 hit Breakpoint 1, jl_breakpoint (v=0x7f779cec8008) at /home/tbesard/Julia/julia-dev/src/rtutils.c:1052
1052    }

now go to the function constructor

(rr) break llvm::Function::Function
Breakpoint 2 at 0x7f77a9525610 (3 locations)
(rr) c
Continuing.

Thread 1 hit Breakpoint 2, 0x00007f77ae6a3e50 in llvm::Function::Function(llvm::FunctionType*, llvm::GlobalValue::LinkageTypes, llvm::Twine const&, llvm::Module*)@plt ()
   from /home/tbesard/Julia/julia-dev/build/debug/usr/bin/../lib/libjulia-debug.so.1

right, breaking in the plt is not useful. lets disable those breakpoints

(rr) info br
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00007f77ae75fe5f in jl_breakpoint at /home/tbesard/Julia/julia-dev/src/rtutils.c:1052
        breakpoint already hit 1 time
2       breakpoint     keep y   <MULTIPLE>         
        breakpoint already hit 1 time
2.1                         y     0x00007f77a9525610 <llvm::Function::Function(llvm::FunctionType*, llvm::GlobalValue::LinkageTypes, llvm::Twine const&, llvm::Module*)@plt>
2.2                         y     0x00007f77a99902cb in llvm::Function::Function(llvm::FunctionType*, llvm::GlobalValue::LinkageTypes, llvm::Twine const&, llvm::Module*)
                                                   at /home/tbesard/Julia/julia-dev/deps/srccache/llvm-6.0.1/lib/IR/Function.cpp:213
2.3                         y     0x00007f77ae6a3e50 <llvm::Function::Function(llvm::FunctionType*, llvm::GlobalValue::LinkageTypes, llvm::Twine const&, llvm::Module*)@plt>
(rr) dis 2.1
(rr) dis 2.3
(rr) info br
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00007f77ae75fe5f in jl_breakpoint at /home/tbesard/Julia/julia-dev/src/rtutils.c:1052
        breakpoint already hit 1 time
2       breakpoint     keep y   <MULTIPLE>         
        breakpoint already hit 1 time
2.1                         n     0x00007f77a9525610 <llvm::Function::Function(llvm::FunctionType*, llvm::GlobalValue::LinkageTypes, llvm::Twine const&, llvm::Module*)@plt>
2.2                         y     0x00007f77a99902cb in llvm::Function::Function(llvm::FunctionType*, llvm::GlobalValue::LinkageTypes, llvm::Twine const&, llvm::Module*)
                                                   at /home/tbesard/Julia/julia-dev/deps/srccache/llvm-6.0.1/lib/IR/Function.cpp:213
2.3                         n     0x00007f77ae6a3e50 <llvm::Function::Function(llvm::FunctionType*, llvm::GlobalValue::LinkageTypes, llvm::Twine const&, llvm::Module*)@plt>
(rr) c
Continuing.

Thread 1 hit Breakpoint 2, llvm::Function::Function (this=0x55a94c7fc7e8, Ty=0x55a94c7ae020, Linkage=llvm::GlobalValue::ExternalLinkage, name=..., ParentModule=0x55a94c324100)
    at /home/tbesard/Julia/julia-dev/deps/srccache/llvm-6.0.1/lib/IR/Function.cpp:213
213           NumArgs(Ty->getNumParams()) {

at a useful point now. let's go up to julia

(rr) up
#1  0x00007f77ae7df9b9 in llvm::Function::Create (Ty=0x55a94c7ae020, Linkage=llvm::GlobalValue::ExternalLinkage, N=..., M=0x55a94c324100) at /home/tbesard/Julia/julia-dev/build/debug/usr/include/llvm/IR/Function.h:138
138         return new Function(Ty, Linkage, N, M);
(rr) up
#2  0x00007f77ae784fb9 in get_specsig_function (M=0x55a94c324100, name="julia_Type_-1577664898", sig=0x7f779d637ce0, jlrettype=0x7f779d637f10) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5165
5165            f = Function::Create(ftype, GlobalVariable::ExternalLinkage, name, M);
(rr) n
211         : GlobalObject(Ty, Value::FunctionVal,
(rr) up
#1  0x00007f77ae7df9b9 in llvm::Function::Create (Ty=0x55a94c7ae020, Linkage=llvm::GlobalValue::ExternalLinkage, N=..., M=0x55a94c324100) at /home/tbesard/Julia/julia-dev/build/debug/usr/include/llvm/IR/Function.h:138
138         return new Function(Ty, Linkage, N, M);
(rr) up
#2  0x00007f77ae784fb9 in get_specsig_function (M=0x55a94c324100, name="julia_Type_-1577664898", sig=0x7f779d637ce0, jlrettype=0x7f779d637f10) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5165
5165            f = Function::Create(ftype, GlobalVariable::ExternalLinkage, name, M);

we want to inspect f, so go past the current line in the current frame using a temporary breakpoint

(rr) tbreak +1
Temporary breakpoint 3 at 0x7f77ae784fc0: file /home/tbesard/Julia/julia-dev/src/codegen.cpp, line 5166.
(rr) c
Continuing.

Thread 1 hit Temporary breakpoint 3, get_specsig_function (M=0x55a94c324100, name="julia_Type_-1577664898", sig=0x7f779d637ce0, jlrettype=0x7f779d637f10) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
5166            f->setAttributes(attributes);

we can inspect the constructed f here

(rr) call f->dump()

declare { i8 } @julia_Type_-1577664898(%jl_value_t addrspace(10)*, [1 x i8] addrspace(11)*)

looks good, let's put a final breakpoint here

(rr) break
Breakpoint 4 at 0x7f77ae784fc0: file /home/tbesard/Julia/julia-dev/src/codegen.cpp, line 5166.
(rr) info br
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00007f77ae75fe5f in jl_breakpoint at /home/tbesard/Julia/julia-dev/src/rtutils.c:1052
        breakpoint already hit 1 time
2       breakpoint     keep y   <MULTIPLE>         
        breakpoint already hit 2 times
2.1                         n     0x00007f77a9525610 <llvm::Function::Function(llvm::FunctionType*, llvm::GlobalValue::LinkageTypes, llvm::Twine const&, llvm::Module*)@plt>
2.2                         y     0x00007f77a99902cb in llvm::Function::Function(llvm::FunctionType*, llvm::GlobalValue::LinkageTypes, llvm::Twine const&, llvm::Module*)
                                                   at /home/tbesard/Julia/julia-dev/deps/srccache/llvm-6.0.1/lib/IR/Function.cpp:213
2.3                         n     0x00007f77ae6a3e50 <llvm::Function::Function(llvm::FunctionType*, llvm::GlobalValue::LinkageTypes, llvm::Twine const&, llvm::Module*)@plt>
4       breakpoint     keep y   0x00007f77ae784fc0 /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
(rr) del 2

now just continue and inspect f until we're at the correct function

(rr) c
Continuing.

Thread 1 hit Breakpoint 4, get_specsig_function (M=0x55a94c324100, name="julia_throw_inexacterror_-1577673445", sig=0x7f77a479c500 <jl_system_image_data+41707584>, jlrettype=0x7f779d20c018)
    at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
5166            f->setAttributes(attributes);
(rr) call f->dump()

declare void @julia_throw_inexacterror_-1577673445(%jl_value_t addrspace(10)*, %jl_value_t addrspace(10)*, i64)

(rr) c
Continuing.

Thread 1 hit Breakpoint 4, get_specsig_function (M=0x55a94c324100, name="julia_#code_llvm_-1577664895", sig=0x7f779d6f2cb0, jlrettype=0x7f779d1fc320) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
5166            f->setAttributes(attributes);
(rr) call f->dump()

declare void @"julia_#code_llvm_-1577664895"({ i8 } addrspace(11)*, %jl_value_t addrspace(10)*, %jl_value_t addrspace(10)*)

(rr) c
Continuing.

Thread 1 hit Breakpoint 4, get_specsig_function (M=0x55a94c324100, name="julia_getindex_-1577673124", sig=0x7f77a3e5ea60 <jl_system_image_data+32017824>, jlrettype=0x7f779d20c018) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
5166            f->setAttributes(attributes);
(rr) call f->dump()

declare void @julia_getindex_-1577673124(i64)

(rr) c
Continuing.

Thread 1 hit Breakpoint 4, get_specsig_function (M=0x55a94c87bbf0, name="julia_throw_inexacterror_-1577673445", sig=0x7f77a479c500 <jl_system_image_data+41707584>, jlrettype=0x7f779d20c018)
    at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
5166            f->setAttributes(attributes);
(rr) call f->dump()

declare void @julia_throw_inexacterror_-1577673445(%jl_value_t addrspace(10)*, %jl_value_t addrspace(10)*, i64)

ugh, plenty of functions being created, maybe we should have warmed up the cache first. let's add a conditional breakpoint instead

(rr) p name
$1 = "julia_throw_inexacterror_-1577673445"

(rr) call name.find("inexact", 0)
$6 = 12

(rr) call name.find("exp", 0)
$7 = 18446744073709551615

(rr) call name.find("exp", 0)
$8 = 18446744073709551615

(rr) p std::string::npos
$9 = 18446744073709551615

(rr) break if name.find("exp", 0) != std::string::npos
Note: breakpoint 4 also set at pc 0x7f77ae784fc0.
Breakpoint 6 at 0x7f77ae784fc0: file /home/tbesard/Julia/julia-dev/src/codegen.cpp, line 5166.
(rr) info br
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00007f77ae75fe5f in jl_breakpoint at /home/tbesard/Julia/julia-dev/src/rtutils.c:1052
        breakpoint already hit 1 time
4       breakpoint     keep y   0x00007f77ae784fc0 in get_specsig_function(llvm::Module*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, _jl_value_t*, _jl_value_t*)
                                                   at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
        breakpoint already hit 22 times
6       breakpoint     keep y   0x00007f77ae784fc0 /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
        stop only if name.find("exp", 0) != std::string::npos (host evals)
(rr) del 4
(rr) c
Continuing.

Thread 1 hit Breakpoint 6, get_specsig_function (M=0x55a94c87bbf0, name="julia_exp_-1577664881", sig=0x7f779e73ee00, jlrettype=0x7f77a1fd9aa0 <jl_system_image_data+15840>) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
5166            f->setAttributes(attributes);
(rr) call f->dump()

declare double @julia_exp_-1577664881(double)

there we go dunno. but the rr usage is not important here i didn't actually reverse debug this is all forwards it's just convenient in case you miss something, to be able to go back or to be able to watch raw pointers

(rr) bt
#0  get_specsig_function (M=0x55a94c87bbf0, name="julia_exp_-1577664881", sig=0x7f779e73ee00, jlrettype=0x7f77a1fd9aa0 <jl_system_image_data+15840>) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
#1  0x00007f77ae77a1e0 in emit_function (lam=0x7f779e6b7510, src=0x7f779ec5d8c0, world=18446744073709551615, declarations=0x7f779e6b7570, params=0x7fff09638640) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5372
#2  0x00007f77ae7788f5 in jl_compile_linfo (pli=0x7fff09634978, src=0x7f779ec5d8c0, world=18446744073709551615, params=0x7fff09638640) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:1141

looks like this hit is actually exp being emitted. turns out it isn't in the sysimg after all. this also means we don't expect the attrs to be here, since they are inferred by optimization

continue to the actual lookup of exp when emitting f

(rr) c
Continuing.

Thread 1 hit Breakpoint 6, get_specsig_function (M=0x55a94c324100, name="julia_exp_-1577664881", sig=0x7f779e73ee00, jlrettype=0x7f77a1fd9aa0 <jl_system_image_data+15840>) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
5166            f->setAttributes(attributes);
(rr) call f->dump()

declare double @julia_exp_-1577664881(double)

(rr) bt
#0  get_specsig_function (M=0x55a94c324100, name="julia_exp_-1577664881", sig=0x7f779e73ee00, jlrettype=0x7f77a1fd9aa0 <jl_system_image_data+15840>) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:5166
#1  0x00007f77ae7bf0d5 in emit_call_specfun_other (ctx=..., li=0x7f779e6b7510, specFunctionObject=..., argv=0x7fff09634740, nargs=2, inferred_retty=0x7f77a1fd9aa0 <jl_system_image_data+15840>)
    at /home/tbesard/Julia/julia-dev/src/codegen.cpp:2942
#2  0x00007f77ae7b823c in emit_invoke (ctx=..., ex=0x7f779ec52030, rt=0x7f77a1fd9aa0 <jl_system_image_data+15840>) at /home/tbesard/Julia/julia-dev/src/codegen.cpp:3079

ok, this is better

so I guess we need to improve emit_call_specfun_other. easier, since we have a full lambdainfo there. maybe we can put the llvm function attrs in it, make it possible to serialize that data into the sysimg too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment