startup-files-and-sequences.md

If you link a program with a compiler driver (clang/gcc) in a standard way (not -nostdlib), the following components are usually on the linker command line.

• crt1.o (glibc/musl): -no-pie/-pie/-static-pie
  • crt1.o: -no-pie
  • Scrt1.o: -pie, -shared
  • rcrt1.o: -static-pie
  • gcrt1.o:
• crti.o (glibc/musl)
• crtbegin.o
  • crtbegin.o: -no-pie
  • crtbeginS.o: -pie, -shared
  • crtbeginT.o: -static-pie
• user input
• -lstdc++
• Some combination of -lc -lgcc_s -lgcc -lgcc_eh
• crtn.o (glibc/musl)
• crtend.o
  • crtend.o: -no-pie
  • crtendS.o: -pie, -shared
  • crtendT.o: -static-pie

crt1.o

This file is only used by executables.

In glibc, the file is -r linked from csu/start.c csu/abi-note.c csu/init.c csu/static-reloc.c. It used to call __libc_start_main with arguments main, __libc_csu_init, __libc_csu_fini (defined by libc_nonshared.a(elf-init.oS)). From BZ #23323 onwards, on most architectures, start.S:_start calls __libc_main_start with two zero arguments instead, and __libc_csu_init and __libc_csu_fini are moved into csu/libc-start.c.

In musl, this file calls __libc_start_main with main, _init, and _fini.

crti.o/crtn.o

crti.o defines _init in the .init section and _fini in the .fini section. The defined _init is a fragment which is expected to be concatenated with other files and finally crtn.o to get the full definition.

In glibc x86-64,

# crti.o
Disassembly of section .init:

0000000000000000 <_init>:
       0: 48 83 ec 08                   subq    $8, %rsp
       4: 48 8b 05 00 00 00 00          movq    (%rip), %rax  # b <_init+0xb>
                0000000000000007:  R_X86_64_REX_GOTPCRELX       __gmon_start__-0x4
       b: 48 85 c0                      testq   %rax, %rax
       e: 74 02                         je      0x12 <_init+0x12>
      10: ff d0                         callq   *%rax

Disassembly of section .fini:

0000000000000000 <_fini>:
       0: 48 83 ec 08                   subq    $8, %rsp

# crtn.o
Disassembly of section .init:

0000000000000000 <.init>:
       0:       addq    $8, %rsp
       4:       retq

Disassembly of section .fini:

0000000000000000 <.fini>:
       0:       addq    $8, %rsp
       4:       retq

The linker defines DT_INIT if _init (default value for -init) is defined, and DT_FINI if _fini is defined.

The section fragment idea is fragile. On RISC-V, DT_INIT is not used.

glibc defines the files in sysdeps/aarch64/crt[in].S.

crti.o calls __gmon_start__ (gmon profiling system) if defined. This is used by gcc -pg.

crtbegin.o/crtend.o

libgcc/crtstuff.c

If __LIBGCC_INIT_ARRAY_SECTION_ASM_OP__ is not defined and __LIBGCC_INIT_SECTION_ASM_OP__ is defined,

• crtend.o defines a .init section which calls __do_global_ctors_aux. __do_global_ctors_aux calls the static constructors in the .ctors section.
• crtbegin.o defines a .fini section which calls __do_global_dtors_aux. __do_global_dtors_aux calls the static constructors in the .dtors section.
• crtbegin.o defines .ctors and .dtors with a single -1 value.
• crtend.o defines .ctors and .dtors with a single 0 value.

On modern distributions, __LIBGCC_INIT_ARRAY_SECTION_ASM_OP__ is 0 and crtend.o contains no .text/.ctors/.dtors.

glibc startup sequence

Below the control flows are flattened.

Dynamically linked executable

In rtld:

• sysdeps/x86_64/dl-machine.h:_user
• elf/rtld.c:_dl_start
• sysdeps/x86_64/dl-machine.h:_dl_start_user
• elf/dl-init.c:_dl_init
• Jump to the main executable e_entry

In the main executable:

• sysdeps/x86_64/start.S:_start
• csu/libc-start.c:__libc_start_main, the SHARED branch
• (if ELF_INITFINI is defined) Run DT_INIT
• Run DT_INITARRAY
• Run main
• Run exit

Statically linked executable

In the main executable:

• sysdeps/x86_64/start.S:_start
• csu/libc-start.c:__libc_start_main, the !SHARED branch
• _dl_relocate_static_pie
• ARCH_SETUP_IREL
• ARCH_SETUP_TLS
• csu/libc-start.c:call_init
  • Run [__preinit_array_start, __preinit_array_end)
  • (if ELF_INITFINI is defined) Run _init
  • Run [__init_array_start, __init_array_end)
• Run main
• Run exit

musl startup sequence

For a dynamically linked executable, the rtld process:

• arch/x86_64/crt_arch.h:_dlstart
• ldso/dlstart.c:_dlstart_c
• ldso/dynlink.c:__dls2 relocate rtld
• ldso/dynlink.c:__dls2b setup early thread pointer
• ldso/dynlink.c:__dls3
• Jump to the main executable e_entry

In the main executable:

• arch/x86_64/crt_arch.h:_start
• crt/crt1.c:_start_c
• src/env/__libc_start_main.c:__libc_start_main
• __init_libc initialize auxv/TLS/stack protector/etc
• libc_start_main_stage2
• __libc_start_init
• exit(main(argc, argv, envp));

__libc_start_init has different behaviors for dynamically and statically linked executables. For a dynamically linked executable: it runs DT_INIT (unless NO_LEGACY_INITFINI) then DT_INIT_ARRAY. Note: libc.so has a dummy _init.

helpful, i'm confused in musl and libgcc