ABIs for MMU-less systems

abis-for-mmu-less-systems

## Linux binfmt loaders

`fs/Kconfig.binfmt` defines a few loaders.

* `BINFMT_ELF` defaults to y and depends on `MMU`.
* `BINFMT_ELF_FDPIC` defaults to y when `BINFMT_ELF` is not selected. A few architecture support `BINFMT_ELF_FDPIC` for NOMMU. ARM supports FDPIC even with a MMU.
* `BINFMT_FLAT` is provided for a few architectures.

`BINFMT_AOUT`, removed in 2022, had been supported for alpha/arm/x86-32.

## BFLT

uClinux uses an object file format called Binary Flat format (BFLT).
<https://myembeddeddev.blogspot.com/2010/02/uclinux-flat-file-format.html> has an introduction.

The Linux kernel has supported the format before the git era (`fs/binfmt_flat.c`).
Both version 2 (`OLD_FLAT_VERSION` in the kernel) and version 4 are supported.
Shared library support was [removed](https://git.kernel.org/linus/70578ff3367dd4ad8f212a9b5c05cffadabf39a8) in April 2022.

BFLT is not for relocatable files. An image file is typically converted from ELF using [elf2flt](https://github.com/uclinux-dev/elf2flt).
`ld-elf2flt` is a ld wrapper that invokes `elf2flt` when the option `-elf2flt` is seen.

GCC's m68k port supports `-msep-data`, a special `-fPIC` mode, which assumes that text and data segments are placed in different areas of memory.
This option is used for XIP (eXecute In Place).

## `-mno-pic-data-is-text-relative`

kpatch (live kernel patching) uses this option [for s390x](https://github.com/dynup/kpatch/commit/10002f5aa671de2878252aaa48f585457d39638a).

## FDPIC

The Linux kernel has supported the format before the git era (`fs/binfmt_elf_fdpic.c`).
Only `ET_DYN` executables are supported. Each port that supports FDPIC defines an `EI_OSABI` value to be checked by the loader.

Several architectures define a FDPIC ABI.

* [_The FR-V FDPIC ABI_](https://www.fsfla.org/~lxoliva/writeups/FR-V/FDPIC-ABI.txt), initial version in 2004
* _Blackfin FDPIC ABI_
* [_The SH FDPIC ABI_](https://sourcery.sw.siemens.com/public/docs/sh-fdpic/sh-fdpic-abi.txt), initial version in 2008
* [_ARM FDPIC ABI_](https://github.com/mickael-guene/fdpic_doc), initial version in 2013.
* There is a 2020 draft for RISC-V: [[RFC] RISC-V ELF FDPIC psABI addendum](https://groups.google.com/a/groups.riscv.org/g/sw-dev/c/ZjYUJswknQ4/m/WYRRylTwAAAJ)

Here is a summary.

The read-only sections, which can be shared, are commonly referred to as the "text segment", whereas the writable sections are non-shared and commonly referred to as the "data segment".
The GOT must reside in the data segment. Special entries called "function descriptors" also reside in the GOT.

A call-clobbered register is reserved as the FDPIC register, used to access the data segment.
A function call needs to ensure that the FDPIC register value can be restored, if the caller may call more functions.
The FDPIC register can be spilled into a stack slot or a call-saved register.

When the address of a function is taken, the address of its descriptor is obtained, not that of the entry point.
The descriptor, resides in the GOT, contains pointers to both the function's entry point and its FDPIC register value.
The two GOT entries are relocated by a dynamic relocation of type `R_*_FUNCDESC_VALUE` (e.g. [`R_FRV_FUNCDESC_VALUE`](https://www.fsfla.org/~lxoliva/writeups/FR-V/FDPIC-ABI.txt#:~:text=The%20R_FRV_FUNCDESC_VALUE%20relocation%20is%20used%20to)).
Calling a function pointer, including calling a PLT entry, sets both the FDPIC register and PC.

To obtain the address of a symbol, GOT and FDPIC-register-relative addressing can be used.
If the symbol is non-preemptible, the symbol can be assumed to be located at a fixed offset within the text or data segments.
If we know it resides in the text or data segment, then we can use a PC-relative or FDPIC-register-relative code sequence, respectively.
For a non-preemptible function, we compute the address of the function descriptor by adding an offset to the FDPIC register.

If the symbol is preemptible, the code sequence loads a GOT entry, relocated by a dynamic relocation `R_*_FUNCDESC`, which contains the function descriptor address

Let's see address-taken operations from code.
```c
void fun() {}
__attribute__((visibility("hidden"))) void hidden_fun() {}
void *addr_fun() { return fun; }
void *addr_hidden_fun() { return hidden_fun; }
```
```asm
// arm-linux-gnueabihf-gcc -c -fpic -mfdpic -Wa,--fdpic
// non-preemptible
  ldr     r0, .L8            // r0 = &.got[n] - FDPIC
  add     r0, r0, r9         // r0 = &.got[n]
  ...
.L8:
// Linker resolves this to &.got[n] - FDPIC. .got[n], relocated by R_ARM_FUNCDESC_VALUE, is the function descriptor, 
  .word   hidden_fun(GOTOFFFUNCDESC) // R_ARM_GOTOFFFUNCDESC(hidden_fun)

// preemptible
  ldr     r3, .L4            // r3 =
  ldr     r0, [r9, r3]
  ...
.L4:
// Linker resolves this to &.got[n] - FDPIC. .got[n], relocated by R_ARM_FUNCDESC, contains the function descriptor address.
  .word   fun(GOTFUNCDESC)   // R_ARM_GOTFUNCDESC(fun)
```

Then, let's see a global variable initialized by the address of a function and a C++ virtual table.
```ccpp
struct A { virtual void foo(); };
void call(A *a) { a->foo(); }
auto *var_call = call;
```

```asm
// arm-linux-gnueabihf-g++ -c -fpic -mfdpic -Wa,--fdpic
  ldr     r3, [r0]           // load vtable
...
  ldr     r3, [r3]           // load vtable entry `.word _ZN1A3fooEv(FUNCDESC)`
  ldr     r9, [r3, #4]       // load FDPIC register value
  ldr     r3, [r3]           // load foo's entry point
  blx     r3

.section        .data.rel,"aw"
var_call:
// Function descriptor address, relocated by R_ARM_FUNCDESC dynamic relocation
  .word   _Z4callP1A(FUNCDESC) // R_ARM_FUNCDESC

.section        .data.rel.ro,"aw"
_ZTV1A:
  .word   0
  .word   _ZTI1A
// Function descriptor address, relocated by R_ARM_FUNCDESC dynamic relocation
  .word   _ZN1A3fooEv(FUNCDESC) // R_ARM_FUNCDESC
```

### Thread-local storage

_ARM FDPIC ABI_ defines static TLS relocations `R_ARM_TLS_GD32_FDPIC, R_ARM_TLS_LDM32_FDPIC, R_ARM_TLS_IE32_FDPIC` to be relative to GOT, as opposed to their non-FDPIC counterpart relative to PC.

### Toolchain notes

`-mfdpic`, only makes sense for `-fpie`/`-fpic`, enables FDPIC code generation.
Like `-mno-pic-data-is-text-relative`, external data accesses use a different base register, r9 for arm.
In addition, external function calls save and restore r9.

gas's arm port needs `--fdpic` to assemble FDPIC-related relocation types.