Skip to content

Instantly share code, notes, and snippets.

@cfriedt
Last active August 14, 2026 12:01
Show Gist options
  • Select an option

  • Save cfriedt/7c8d8d59d62345caee8127d71e9fe86b to your computer and use it in GitHub Desktop.

Select an option

Save cfriedt/7c8d8d59d62345caee8127d71e9fe86b to your computer and use it in GitHub Desktop.
zephyr-process-support.md
# POSIX_MULTI_PROCESS support for Zephyr (posix-next)
## Implementation status (branch `posix-multi-process`)
**Naming/layering rules (Chris, 2026-08-14, apply to all process work):**
- `k_`/`K_` prefixes: APIs and constants that do not allocate or free at runtime.
The kernel layer never refers to `sys_` or POSIX symbols.
- `sys_`/`SYS_` prefixes: APIs and constants involved in runtime allocation or
deallocation (lib/os). May use `k_`; never refers to POSIX.
- POSIX declarations derive their constants from the `K_`/`SYS_` counterparts and
wrap the `k_`/`sys_` functions. Existing precedent: `k_spawn()` takes a
caller-provided thread + stack (no allocation, correctly `k_`); the
pool-allocating spawn belongs at `sys_spawn()`. There is no `k_fork()`: POSIX
`fork()` wraps `k_clone()` with the COW flags directly (fork-as-a-name lives in
libc/POSIX, as on Linux).
**HISTORY LINEARIZED 2026-08-12** (force-pushed; pre-rewrite tip preserved as
`backup-prelinearize-29863b116`). The 16 milestone-ordered commits were rebuilt
into 11 layer-ordered commits on `520f32e1a`, per Chris's direction (native
impl → native tests/docs → POSIX impl → POSIX tests → POSIX docs; fixes to
main-owned patches first, folded into the patch that introduced the issue):
| Commit | Layer | What it is |
|--------|-------|------------|
| `e53c7adce` | fix-for-main | Both k-signal fixes **folded into `k-signal.patch` itself** (purge dying thread's pending signals + skip delivery under `k_is_in_user_syscall()`); standalone fix patches deleted; context-only regens of k-thread-cancel/k-timer-clock/k-timer-signal/k-msgq-notify/sys-aio. Fold candidate for main's `4978872c2` on next main rewrite. |
| `c86dad7be` | native kernel | `k-process-core.patch`: full k_process API surface (now introduces the **low-bit `K_PID_PGRP()` encoding** directly; waitpid-tag fix patch folded in). |
| `c0ff4f768` | native kernel | `k-process-exit-wait.patch`: lifecycle, k_spawn, exit/wait, init start, `tests/kernel/process`. |
| `32aca11de` | native kernel | `k-process-sigchld.patch`: SIGCHLD on child zombify. |
| `9d2bd0a79` | native kernel | `k-process-pgrp-session.patch`: k_setpgid/k_kill_pgrp/k_kill + process_sig_target. |
| `2bccdcbbc` | native libc | `libc-multi-process-headers.patch`: sys/wait.h shims, CLD_*. |
| `ba6bdb650` | native libc | `libc-common-exit.patch`: common exit()/atexit(). |
| `2b4d42fda` | native libc | `libc-common-under-exit.patch`: `_exit()` → `k_exit()`. |
| `d1566d030` | POSIX impl | All module impl (headers, one-file-per-function, final kill()/atfork shape). |
| `b3aa3c177` | POSIX tests | tests/posix/multi_process (+exit app), signals/threads_base updates. |
| `5927fea93` | POSIX docs | doc yaml data (sys/wait.h mapping, typo-alias removal). |
Verified: applied zephyr tree **byte-identical** to the pre-rewrite verified
tree; 119-patch series applies from pristine v4.4.1; patches.yml sha256s
consistent at every commit; smoke twister 32/32 (kernel process+signal, posix
multi_process+signals on qemu_cortex_m3 + qemu_riscv32). Follow-up flagged:
native layer still has no Zephyr-tree docs page (doc/kernel/services/) — add
during the fine-tuning pass. Series count is now 119 (three standalone fix
patches folded away).
Pre-rewrite milestone history (superseded, kept for context):
| Commit | Milestone | What landed |
|--------|-----------|-------------|
| `d99a4a165` | M0a | Kernel process API surface + `<sys/wait.h>`/`sys/types.h` headers (declarations, stubs). Now also carries the syscall-gating folded in during M1c. |
| `5faa513a8` | M0b | Defined single-process fallback behaviour for the whole option group; real `pthread_atfork` (elastipool registry); 23 ZTESTs; exit console app; `muti_process` typo fix. |
| `f32f5e43c` | (libc) | `exit()`/`atexit()` promoted to common libc (`COMMON_LIBC_EXIT`), user-mode-safe (atomics + `Z_LIBC_DATA`), bounded CAS registry. Fixes picolibc stub-atexit. |
| `6739f0706` | M1 (a+b) | `k_process` lifecycle: thread-group membership, static-process start, `k_exit`/`k_waitpid`, **`k_spawn`**; reserved swapper/init/kthreadd; `tests/kernel/process`. |
| `4ffcc97a2` | M1c | POSIX layer backed by real `k_process` (`POSIX_MULTI_PROCESS` selects `PROCESS`); identity/wait wrappers over `k_*`; process-aware `kill`/`sigqueue`; process syscalls gated `CONFIG_PROCESS`; `CLD_*` in libc shims. |
| `49beb4afc` | M1d+CI | k_pid_of/k_process_by_pid syscalls (userspace fix). |
| `c61f32e83` | M1e | `_exit()` → `k_exit()` (`COMMON_LIBC_UNDERSCORE_EXIT`); a process's libc `exit()` terminates that process alone. |
| `79f07c6fa` | M1f | init process (pid 1) started at boot from a SYS_INIT static process. |
| `39b4322cd` | **M2** | **Process groups + sessions**: new `k_setpgid`/`k_kill_pgrp` kernel syscalls (`k-process-pgrp-session.patch`); `setpgid()` wrapper; `kill(pid<=0)` group fan-out; kernel `test_setpgid`/`test_kill_pgrp` + POSIX `test_setpgid` + `kill(0,0)` signals check. |
**M2 COMPLETE** (2026-08-11). Process-group/session control done: `k_setpgid` moves a
process (self or child) into a group under the same-session / not-a-session-leader rules;
`k_kill_pgrp` fans a signal out to every member's group leader (CONFIG_SIGNAL-gated,
`-ENOSYS` without). Module `kill()` now handles `pid<=0` (pid 0 = caller's group, pid<-1 =
group `-pid`); **kill() is process-directed, pthread_kill() thread-directed** (user
guidance). Verified green on the full runci matrix *except* qemu_riscv32, which fails the
**pre-existing M1 spawn/waitpid breakage** (confirmed at the M1f baseline with M2 stashed —
not an M2 regression; riscv32 was never in M1's verified set). Series now **118 patches**.
**M2.1 (review refinements, 2026-08-11)** — folded into the M2 commits (now `cf83dcb53`
M2 + `57168beea` purge; series **120 patches**), per Chris's feedback that this belongs
below the line:
- **`k_kill(pid, signo)` kernel syscall** owns the full POSIX pid encoding; module
`kill()` is a thin wrapper. `kill()` is process-directed, `pthread_kill()` thread-directed.
- **Process-directed dispatch to the first *unblocked* thread** (`process_sig_target`),
used by `k_kill`/`k_kill_pgrp`; delivered to an eligible member, not always the leader.
- **`z_sig_purge_thread()`** frees a dying thread's queued-but-undelivered signals from the
halt path (new `k-signal-purge-thread-on-exit.patch`); `k_sig_lock` made a leaf (raise
after unlock) so the purge is safe under the scheduler lock.
Verified on the runci matrix (all green except pre-existing qemu_riscv32; the one
qemu_x86_64 kernel.process.init "fail" is an SMP harness flake — the suite passes 13/13).
**M2.2 (root-cause fix, 2026-08-11)** — commit `571e6620c`, series **121 patches**:
- **Nested-syscall signal-delivery fault ROOT-CAUSED AND FIXED**
(`k-signal-nested-syscall-delivery.patch`). The generated `z_<syscall>` wrapper runs
`z_syscall_sig_check()` on its kernel-direct branch; when `k_kill()` (a user syscall)
calls the `k_sig_queue()` wrapper in kernel context, that ran signal delivery
(`z_sig_trampoline`→`k_sig_return`→`arch_sig_post`) against the outer syscall frame with
no matching `arch_sig_pre()`, corrupting it. Fix: `z_syscall_sig_check()` returns early
under `k_is_in_user_syscall()` (the arch userspace-return path delivers instead).
- **`kill()` is now FULLY THIN** over `k_kill()` for all pid encodings — the M2 self-delivery
workaround is removed. `tests/posix/signals` passes on qemu_x86_64 (SMP userspace).
**Known follow-ups:**
- **SIGCHLD to an ignoring/never-consuming parent** leaks a slab slot (distinct from the
thread-exit purge); the guidance is "kernel-mode parents must explicitly unmask the signals
they care about" (test-side). See [[sigchld-ignoring-parent-leak]].
- **threads_base.userspace atfork fault** — RESOLVED (`b624d7ebf`): the M0b elastipool atfork
registry was kernel .bss touched from user mode; moved to the libc app-memory partition
(`static Z_LIBC_DATA` array + atomic, atexit-style).
- **qemu_riscv32 spawn/waitpid** — RESOLVED (`846cda184`): K_PID_PGRP selector used sign discrimination; process pointers at 0x80000000 are negative on 32-bit. Now low-bit tagged (`k-process-waitpid-pgrp-tag.patch`).
**Note**: branch hashes changed (CI fixes folded into M0a/M0b, syscall fix bundled on top). Full-matrix twister via CI surfaced three issues now fixed: `<sys/wait.h>` W* redefinition (#undef+redefine), exec/fork/setsid tests unsafe on host libc (skip under NATIVE_LIBC), and `k_pid_of`/`k_process_by_pid` hanging in user mode on riscv64 PMP (made syscalls).
**Verified**: `tests/kernel/process` 11/11 and `tests/posix/multi_process` 23/23 on
qemu_cortex_m3 (kernel-mode, userspace, minimal libc), qemu_x86, qemu_riscv64; signals
19/19, realtime_signals green; process syscalls absent when `CONFIG_PROCESS=n`; series
applies from pristine (**117 patches**).
**M1 COMPLETE** (M1a–M1f). Prior **Remaining** note (now done): M1e — SIGCHLD emission on child zombify (needs `k_sig_info`
`si_pid`/`si_status` extension via a new `k-process-signal` patch, plus halt-path-safe
deferral: set per-process `sigchld_pending` under the scheduler lock, submit a shared
scan-work whose handler re-validates `ZOMBIE` state under `process_lock` to avoid
slot-reuse races). **M1f** — init-process start (a `SYS_INIT`-launched static pid-1 process).
**Housekeeping debt**: M1c and M1d bundle their kernel-patch regenerations into the
feature commit rather than folding into the introducing commits (autosquash conflicted
on wholesale patch regeneration + entangled `patches.yml`); a pre-upstream cleanup
rebase should fold them. History was rewritten (M0b/M1), so the open PR needs a
force-push.
**Next milestones** (see roadmap): M2 `k_spawn`+`posix_spawn`, M3 exec via LLEXT,
M4 COW fork on MMU, M5 groups/sessions + SIGCHLD completeness.
## Context
The posix-next module implements POSIX Subprofiling Option Groups for Zephyr. The
`POSIX_MULTI_PROCESS` group requires: `_Exit, _exit, assert, atexit, clock, execl, execle,
execlp, execv, execve, execvp, exit, fork, getpgrp, getpgid, getpid, getppid, getsid,
setsid, sleep, times, wait, waitid, waitpid`. Today only `getpid()` (constant 42),
`times()` (no child times), and `sleep()` exist; Zephyr has **no process concept at all**.
The goal is to introduce a real process abstraction into the Zephyr kernel (as upstreamable
patches in the module's patch series), an exec loader, common-libc ISO C pieces, and the
POSIX module glue + tests. The user (Chris) will hand-implement much of the kernel/arch
work; this plan is the reviewable blueprint.
## Decisions (made by user)
- **Process model — tiered**: a first-class `k_process` kernel object (thread group +
memory domain + pid + parent/children + exit status) that works on MPU **and** MMU
targets. Full address-space semantics only where MMU per-domain page tables exist.
- **Static definition idiom (user requirement)**: processes are definable at build time
via a `K_PROCESS_DEFINE()` macro placing objects in an iterable section, like other
Zephyr kernel objects — the kernel manages statically allocated processes the same
way. The system reserves three pids, all kernel-defined with that same macro:
**pid 0 = swapper/idle process** (per-CPU idle threads; sleeps when nothing else is
runnable), **pid 1 = init process** (parent of all userspace threads + processes),
**pid 2 = kthreadd** (parent process of all kernel threads).
- **init process start**: the init process (pid 1) is a statically defined process
started at boot (`SYS_INIT`/`K_PROCESS_DEFINE`), independent of what the image's
`main()` does; the image `main()` keeps today's semantics.
- **fork() — COW on MMU arches only** (x86, arm64). On MMU-less targets (MPU/PMP),
`fork()` returns **ENOSYS** and **`posix_spawn()` is the process-creation story**.
- **exec() — LLEXT from filesystem** (`fs_loader`), PATH search for `execlp`/`execvp`.
- **Bring-up platforms**: qemu_x86 (MMU/COW), qemu_cortex_a53 (MMU/COW),
qemu_riscv64 (PMP → spawn tier), mps2/an385 (MPU → spawn tier).
## Key exploration findings (ground truth)
### Kernel (zephyr/ @ v4.4.1 + 116-patch series applied)
- `k_mem_domain` already models an address space with a thread membership list;
membership inherited at `k_thread_create()`. Per-domain page tables exist on x86
(`arch_mem_domain.ptables` + CR3 swap) and arm64 (per-domain TTBR0 + ASID).
- **Blocker #1**: kernel/Kconfig.vm invariant — all page tables share one VA→PA map
(permission views only); `virt_region_bitmap` in kernel/mmu.c is global; no page-frame
refcounting (needed for COW).
- `typedef k_tid_t k_pid_t` exists (thread.h:417); explicit `TODO(k_process)` markers at
kernel.h:1995 (k_timer sig_fn_domain/perms_src) and sys/timer.h:51,104.
- Signal subsystem (k-signal.patch) is process-shaped already: `k_sig_queue(k_pid_t,…)`,
`K_SIG_CHLD` (default Ign), user-mode delivery on arm/arm64/riscv/x86. Gaps: `k_sig_info`
lacks si_pid/si_status; signal action DB is global not per-process; STOP/CONT ignored.
- Thread lifecycle: `thread_state` currently uses all 8 bits — if a per-thread state
bit is ever needed, **widen `thread_state` to `uint16_t`** (user decision). The
implemented design sidesteps this: the zombie lives in `k_process` after the threads
are freed, so no new thread_state bit is required. `k_thread_exit/result/rejoin/
cancel/cleanup/detach` + `sys_thread_recycle()` reaper exist from the series.
- fdtable: single global static array (lib/os/zvfs/zvfs_fdtable.c) with per-slot
`K_OBJ_FILE` objects; FD_CLOEXEC stored (`fd_flags`) but never acted on; CWD global
(zvfs_cwd.c). k_object perms are per-thread bitmaps (`CONFIG_MAX_THREAD_BYTES`=2 ⇒ 16
userspace threads) → **becomes 16 userspace *processes*** once permissions move to
`k_process` (see Per-process fds/cwd and the kernel-risk note).
- `typedef k_tid_t k_pid_t` (thread.h) — **kept as-is (user decision)**: `k_sig_queue`
stays thread-directed, process APIs take `struct k_process *` explicitly, and
`kill`/`sigqueue` resolve a pid to its process and deliver to the group leader. This
is preferred over flipping `k_pid_t` to a process handle; it keeps the existing
signal path intact and is what is implemented through M1e.
- LLEXT: loads ET_REL/relocatable/shared (no ET_EXEC/PIE, no argv/envp/entry ABI); has
`llext_add_domain()`, fs/buf loaders, per-image mem partitions; working
load→domain→K_USER-thread recipe in tests/subsys/llext/src/test_llext.c:116-202.
### libc
- `_exit` is a per-libc spin-forever stub (minimal's is NON-weak); `_Exit` exists nowhere;
atexit registries are per-libc (minimal 8-slot; picolibc PICOEXIT; newlib reent).
- `abort()` is already a common-libc unit — `libc-isoc-signal-handling.patch` and
`libc-common-tmpfile.patch` are the established patterns for common-libc promotion.
- picolibc `clock()` lights up via `times()` + `CLOCK_PROVIDED`; newlib needs `_times`.
- After `main()` returns, `bg_thread_main` runs no atexit, propagates no status.
- newlib hook pattern: `#ifndef CONFIG_POSIX_MULTI_PROCESS` around `_getpid` etc.
### Module
- unistd.h already declares all 17 MULTI_PROCESS functions with Doxygen.
- `sys/wait.h` exists nowhere (toolchain copies declare only wait/waitpid; no waitid/
idtype_t/id_t/P_*). `id_t`/`idtype_t` absent from sys/types.h.
- `kill()` overloads pid>0 as a `k_thread *`; `POSIX_THIS_PID`=42 shared with signals.
- `environ` is one global `char **` (options/shared/env_common.c); `pthread_atfork()` is a
documented no-op that must become real.
- siginfo_t already has si_pid/si_status, SIGCHLD, CLD_* — plumbing gap is kernel-side.
- tests/posix/multi_process exists (assert/atexit/clock/getpid/sleep/times, six twister
variants; scenario-name typo "muti_process" to fix).
## Constraints
- New kernel work = **new patches** in zephyr/patches/zephyr/ registered in patches.yml
(plain unified diffs, upstreamable; never touch frozen patches).
- One ZTEST per unit (`test_<function>`), aspects as section helpers.
- Copyright: novel .c → FPES form; headers/tests/Kconfig → ZPC.
- runci.sh for full twister runs; no build-dir litter.
- User hand-implements the deep kernel/arch pieces; module/libc/test work can be delegated.
---
## Design: exec loader + libc integration
### Process ABI (kernel-defined contract; user requirements)
Exec support is a **later phase** (see roadmap) — statically defined
`K_PROCESS_DEFINE()` processes come first, including **prelinked-blob processes**
(prelinked ELF → binary blob linked into firmware, entry = numeric address), which
give code-in-own-partition process coverage with **no LLEXT and no filesystem**.
LLEXT-from-filesystem is strictly an *optional* dynamic image source layered on the
same kernel machinery. When exec lands, layering is:
- **The Zephyr kernel defines the process ABI framework**: the entry-point contract
and the **process memory layout**, including where the program-arguments/environment
block lives (Linux-style: a kernel-specified args/env block — argc, argv/envp
vectors + packed strings, ARG_MAX-bounded — placed at a defined location, e.g. top
of the initial stack). The kernel *places* this block at process start; it is
**owned by the application** thereafter — the kernel never reaches back into it.
- **crt0 is C-runtime territory, below POSIX**: the C library (minimal / picolibc /
common), not the POSIX module, provides the crt0 that implements the kernel's entry
contract — parses the args/env block per the ABI, initializes libc state, sets up
`environ`/argc/argv, calls `exit(main(argc, argv, envp))`. libc and POSIX both act
on the kernel-defined layout (shared, synchronized access to `environ` via libc's
storage — the POSIX module's getenv/setenv become consumers of that layout rather
than owners of a module-global).
- The build helper that produces executables (cmake wrapper over `add_llext_target()`
that links crt0 + static libc) is therefore Zephyr-side, not module-side.
### Executable format (when exec lands)
- **ET_REL relocatable objects** (`CONFIG_LLEXT_TYPE_ELF_RELOCATABLE`) — the only LLEXT
format common to all four platforms (`LLEXT_TYPE_ELF_OBJECT` is `!RISCV`). PIE/ET_DYN
deferred (multi-month loader effort).
- Apps write `int main(int argc, char **argv, char **envp)`; libc crt0 exports
`_start` per the kernel ABI; missing `_start` ⇒ `ENOEXEC`.
### execve flow (failure-safe ordering; sys layer — POSIX is not the right layer)
The load/commit split lives **below POSIX** as `sys_process_exec_load()` /
`sys_process_exec_commit()` in lib/os; the module's `execve()` only marshals and calls
them. Load (can fail cleanly: stage argv/envp → obtain image from the configured
source — static blob registry or, optionally, LLEXT fs loader → find entry; on
failure the old image is intact) then commit (point of no return):
1. Build new `k_mem_domain` + `llext_add_domain()` **before** killing siblings (so
`-ENOSPC` is still a clean failure). No `z_libc_partition` needed — executables
carry their own statically linked libc state in the image partitions.
2. `k_process_exec_begin(proc)` — terminate other threads in the group (POSIX).
3. **FD_CLOEXEC sweep**: new `zvfs_close_on_exec()` — first consumer of the stored flag.
4. Reset caught signal dispositions to SIG_DFL (keep IGN/mask/pending).
5. `llext_bringup()`; create **fresh main k_thread + stack** in the same k_process
(avoids in-place stack-rewind arch magic), start at trampoline → `_start(args)`,
K_USER drop when USERSPACE; old thread detaches old image (deferred
`llext_teardown/unload` on workqueue) and self-exits.
- Image outlives the entry thread: the **reaper** (wait/orphan auto-reap) tears down the
llext image — k_process needs an image-destructor hook (kernel interface requirement).
- `fexecve`: ~60-line module-side `llext_loader` adapter over zvfs read/lseek. No `#!`,
no exec-bit, no execvp sh-fallback in phase 1 (documented deviations).
- PATH search: `getenv("PATH")` falling back to `confstr(_CS_PATH)`.
### Symbol linkage — minimal surface (user requirement)
The export surface is **predominantly Zephyr kernel APIs**: enable
`LLEXT_EXPORT_SYMBOL_GROUP_SYSCALL` only — no libc symbol group, no POSIX-module
export unit. Executables **statically link their own libc + POSIX wrapper library**
(crt0 + libc.a + the module's thin `k_*()`-wrapping POSIX functions, partial-linked
into the ET_REL image), so only `k_*()` syscalls cross the process boundary. Benefits:
minimal ABI surface, per-process libc state for free (each image carries its own
`.data/.bss` — no shared `z_libc_partition` wart), and clean fit with the
POSIX-wraps-kernel separation. Cost: larger executables and partial-link-with-archive
build mechanics (resolve `libc.a` members into the relocatable — verify the
EDK/`add_llext_target` flow supports archive resolution; implementation detail to
prove in the exec phase). Compiler intrinsics (`__aeabi_*`, `memcpy`) must resolve
locally from the static link — audit during bring-up.
### Memory-protection fit
- qemu_x86 / qemu_cortex_a53 (MMU): fresh domain per process, full COW tier; verify
`llext_adjust_mmu_permissions()` in the per-process-domain flow (may need a small patch —
it is domain-global today).
- qemu_riscv64 (PMP): domain isolation feasible but tight; spawn tier.
- mps2/an385 (3 MPU slots): 4 llext partitions + libc **cannot fit** — phase 1 runs exec'd
processes with `POSIX_EXEC_ISOLATION_NONE` (correct semantics, no HW isolation; honest
single-image trust model). Phase 2: `llext-merged-mem-partitions.patch`
(TEXT+RODATA→RX, DATA+BSS→RW ⇒ 2 partitions) makes DOMAIN isolation fit.
- MPU tier's realistic pattern: fork-then-immediately-exec / future posix_spawn (phase 2).
### Common-libc patches (precedents: libc-common-tmpfile, libc-isoc-signal-handling)
- `libc-common-under-exit.patch`: common `_exit()` (+ `FUNC_ALIAS` `_Exit` — exists
nowhere today) → `k_process_exit(status)` under `CONFIG_PROCESS`, today's spin
otherwise; gates out minimal's **non-weak** `_exit` (picolibc/newlib are `__weak` —
override free); `_Exit` decl in minimal stdlib.h.
- `libc-common-exit.patch` — **DONE (pulled forward into M0, commit c105917b2)**:
exit/atexit promoted to common libc (`COMMON_LIBC_EXIT`, default y for minimal +
picolibc; `COMMON_LIBC_ATEXIT_MAX` default 32 per ISO C). Motivated by the M0
finding that the SDK's prebuilt picolibc has a STUB atexit (registration records
nothing; exit() runs only `__libc_fini_array`). newlib keeps `__call_exitprocs`.
Documented deviations: no stream flush at exit, fini-array functions don't run.
M1's remaining libc work: `libc-common-under-exit.patch` (`_exit`/`_Exit` →
`k_exit`) and clock().
- `libc-common-clock.patch`: promote minimal clock.c to common; per-process CPU time
under `CONFIG_PROCESS`, uptime-based otherwise (documented deviation); newlib `_times`
hook in libc-hooks.c (existing `#ifdef CONFIG_POSIX_MULTI_PROCESS` stub pattern).
- `assert()`: nothing needed (→ abort → COMMON_LIBC_ABORT/SIGABRT). Requirement on
kernel design: fatal-signal default action terminates the **process** with
WIFSIGNALED-compatible status.
- Boot: the static init process (pid 1) starts from a `SYS_INIT`, independent of the
image `main()`; no change to `bg_thread_main`'s main()-return behaviour is required.
(Process `exit()` belongs to process mains via crt0; coverage-dump ordering is
handled where the dump already lives.)
### Headers
- New module `include/zephyr/posix/sys/wait.h`: wait/waitpid/waitid, Linux-compatible W*
encoding, `WNOHANG/WUNTRACED/WCONTINUED/WEXITED/WNOWAIT/WSTOPPED`, `idtype_t`
(P_ALL/P_PID/P_PGID); `id_t` added to module sys/types.h. W* live in sys/wait.h only.
- `libc-sys-wait-h.patch`: **full-replacement** forwarder shims for picolibc/newlib
(toolchain sys/wait.h would collide on W* macros; matches libc-statvfs-utime pattern).
Minimal libc resolves the module header directly.
- `_POSIX_JOB_CONTROL`/`_POSIX_SPAWN` stay commented out (phase 2).
### Kconfig wiring (module)
`POSIX_MULTI_PROCESS` umbrella unchanged (getpid/times/sleep keep working with today's
deps). New sub-symbols: `POSIX_FORK` (depends PROCESS), `POSIX_WAIT` (depends PROCESS),
`POSIX_EXEC` (depends PROCESS && LLEXT && FILE_SYSTEM && LLEXT_TYPE_ELF_RELOCATABLE;
selects the two export-symbol groups), `choice POSIX_EXEC_ISOLATION` (DOMAIN|NONE).
Unconfigured functions exist and fail `ENOSYS` (documented subprofiling posture).
### Test topology
- **M1–M2 (pre-exec): statically defined processes are the coverage vehicle** — test
apps `K_PROCESS_DEFINE` one or more child processes whose entries exercise
exit-status, wait/waitpid/waitid, SIGCHLD, kill-by-pid, getpgid/getsid across real
process boundaries with zero loader machinery.
- **Prelinked-blob payloads (no LLEXT, no filesystem)**: the Zephyr-side executable
build helper prelinks tiny executables (crt0 + static libc) at the per-arch entry
convention, objcopys them to binary blobs linked into the test firmware, and
`K_PROCESS_DEFINE()` points at them — code-in-own-partition processes on all 4
targets with zero loader dependencies. (hello: checks argv/envp, exits with known
status; spinner: for kill/wait tests.)
- Optional LLEXT-from-filesystem scenarios (exec phase only): same payloads embedded
via `generate_inc_file_for_target()`, written to RAM littlefs `/bin/` by test setup.
- Scenarios: base (no FS/LLEXT, proves ENOSYS config), `.exec` (all 4 platforms; an385
with ISOLATION_NONE), `.exec_userspace` (MMU platforms + riscv64 best-effort),
`.fork` (COW on MMU only).
---
## Design: POSIX module layer
### Headers first (user requirement: declarations ironed out before linkage)
Header inventory for POSIX_MULTI_PROCESS (audited against POSIX.1-2017):
| Header | State | Action |
|---|---|---|
| `sys/types.h` | Present, near-complete | **Add guarded `id_t`** (missing entirely; required by spec and by waitid). **Add guarded `timer_t`** (spec requires it in sys/types.h; today only in posix_time.h:47, unguarded — add the standard `_TIMER_T_DECLARED`/`__timer_t_defined` guards in BOTH per the shared-symbol rule; sys/types.h is not the owner, posix_time.h keeps Doxygen). `trace_*` TODO stays (Trace group, not MULTI_PROCESS). |
| `sys/wait.h` | **Missing everywhere** | Create per posix-header-layout skill: IEEE declaration order, `idtype_t {P_ALL,P_PID,P_PGID}`, W* constants/macros, wait/waitid/waitpid declarations with Doxygen `@ingroup posix_option_group_multi_process` (owning header). **Every W* constant/macro derives from a Zephyr-native symbol** — `WNOHANG`/`WUNTRACED`/`WCONTINUED`/`WEXITED`/`WNOWAIT`/`WSTOPPED` map to `K_PROCESS_W*` flags and the status macros wrap the kernel `K_WSTATUS_*` bit layout (Zephyr-native encoding is authoritative; Linux-shaped layout is a convenience, not a contract — toolchain headers are replaced by shims, not accommodated). Linear includes, basic types only. Plus the picolibc/newlib full-replacement shim patch (`libc-sys-wait-h.patch`). |
| `unistd.h` | Complete | All 17 declarations already present. |
| `sys/times.h` | Complete | No change. |
| `stdlib.h`/`assert.h`/`time.h` | libc-owned | `_Exit` declaration added to minimal stdlib.h via the common-libc patch; nothing else missing. |
M0 lands the header work **first**, before any implementation linkage — declarations
compile-tested across all six libc variants before functions exist behind them.
### POSIX / kernel separation (user requirement)
POSIX functions are **thin wrappers** over kernel `k_*()` syscalls or lib-layer
`sys_*()` APIs — never bearers of kernel logic themselves (precedent: the series'
`sys_thread` layer in lib/os, which pthreads and C11 threads both wrap):
- `getpid/getppid/getpgid/getsid/setsid/_exit/fork/waitpid/waitid` → direct wrappers
over `k_process_*()` syscalls (+ W*-encoding / signo mapping, which is POSIX-ism and
stays module-side).
- **The exec core moves Zephyr-side as `sys_process_exec()` / `sys_process_spawn()` in
lib/os** (new patch, following the sys_thread precedent): composes LLEXT load,
domain build, argv/envp staging + crt0 ABI, `z_process_exec_begin/commit`, deferred
image teardown. Usable by non-POSIX apps; upstreamable on its own. The module's
`execve()` et al. keep only the POSIX-isms: path/PATH resolution (environ,
`confstr(_CS_PATH)`), varargs marshalling, errno mapping. (Supersedes the earlier
placement of the full exec flow in module `exec.c`; the fexecve zvfs-fd loader
adapter also moves into the sys layer.)
- `pthread_atfork` registry stays module-side (pure POSIX bookkeeping); its hooks run
around the `k_process_fork()` call in the module wrappers.
### Layout (one file per function, strict)
`multi_process.c` dissolves into `getpid.c` / `times.c` (preserving existing notices per
preserve-copyright). New files: `_exit.c`, `fork.c`, `execv.c execve.c execvp.c`,
`execl.c execle.c execlp.c`, `exec_common.c` (va_list→vector marshalling),
`getppid.c getpgrp.c getpgid.c getsid.c setsid.c`, `wait.c waitpid.c waitid.c`,
`multi_process_internal.h` (W* encode helpers). Shared:
`options/shared/atfork_common.c` (registry + `z_posix_atfork_run()`; shared/ compiles
unconditionally so fork.c links without POSIX_THREADS), `env_common.c` gains
`z_environ_init_from_vector()`, `posix_internal.h` gains `posix_this_pid()`.
New module `.c` = FPES notice; headers/tests/Kconfig = ZPC.
### Key decisions
- **W* encoding: the native Zephyr encoding (`K_WSTATUS_*`) is authoritative** — the
module's W* macros must agree with it by derivation. Where a libc's own sys/wait.h
encodings differ from Zephyr's for the same symbols, a **runtime mapping** is done —
with **compile-time equality checks creating a zero-cost fast path** when the
encodings coincide (the established pattern: `net/conversion.h`-style converters and
the module's `z_sig_set_equal()` fast path).
- **pid migration**: `posix_this_pid()` inline — `k_process_pid(k_process_current())`
under `CONFIG_PROCESS`, else the existing 42. Three users switch (getpid, signals
kill(), rtsig). Root process pid 1, ppid 0. `kill()` pid>0 branch becomes
`k_process_kill()` under CONFIG_PROCESS; legacy thread-pointer overload preserved
bit-exact when off.
- **pthread_atfork becomes real**: prepare (reverse order) → clone/fork syscall →
parent/child (registration order). Handlers become live with M4 COW fork.
**Registry is elastipool-backed, no malloc** (user decision — malloc in the atfork
path risks fork-time deadlocks): statically sized via
`CONFIG_POSIX_THREAD_ATFORK_MAX` (default 0) raised to the largest distributed
`POSIX_THREAD_ATFORK_MIN_*` contribution (SIGNAL_SET_SIZE_MIN aggregation pattern);
empty table → ENOMEM. No `MIN_POSIX` floor — POSIX specifies no minimum.
- **Docs posture (user decision)**: keep the `†` undefined-behaviour markers in the
option-group table until the group is actually conformant — no interim
"defined-fallback" table updates.
- **Coexistence with libc infrastructure (user requirement)**: registration arrays are
libc-owned and must be respected by all three layers. The contract: the **libc's
atexit registry is the single source of truth** for exit handlers — libc `exit()`
runs it (plus ISO C stream flush/tmpfile cleanup), then `_exit()` → `k_exit()`;
the kernel never runs handlers (`k_exit` is below libc); POSIX `_exit()`/`_Exit()`
bypass them by definition and fork/exec never duplicate or re-run them. The same
discipline applies to atfork: the registry lives in one place (module shared code),
and libc-internal atfork stubs (newlib/picolibc) must be checked for collision and
routed to it.
- **environ**: no fork hook needed — COW duplicates it for free. Exec side uses
`z_environ_init_from_vector` in the new image.
- **Kconfig (user requirement)**: the kernel exposes granular Zephyr-native symbols
(`PROCESS`, `PROCESS_STATIC`, `PROCESS_FORK_COW`,
`PROCESS_EXEC`, `ZVFS_PER_PROCESS_FDS`, …) and **`POSIX_MULTI_PROCESS` selects the
kernel symbols it requires wherever the platform supports them** (`select PROCESS`,
`select PROCESS_FORK if ARCH_HAS_THREAD_FORK && USERSPACE`, etc.). The POSIX layer
stays a **very thin wrapper** around the native syscalls. ENOSYS fallbacks remain
only where the substrate genuinely cannot exist on a platform (e.g. COW fork
without MMU); wait* → ECHILD when no children; identity fns fall back to
single-process identity only on `!PROCESS` builds of other option groups.
### Test design
- **Native suites first (user requirement)**: every native Zephyr component gets its
own dedicated testsuite in the Zephyr tree — `tests/kernel/process/static`
(K_PROCESS_DEFINE, reserved pids, membership), `tests/kernel/process/spawn`,
`tests/kernel/process/clone`, `tests/kernel/process/wait_exit`, signal-CHLD additions
to `tests/kernel/signal` — landing in the same patch as their implementation
(k-signal.patch precedent). **The native components must be working and tested
satisfactorily before the corresponding POSIX components are wired**; POSIX suites
are conformance re-checks over already-proven kernel behavior.
- POSIX side: one ZTEST per function for all 23 units; fallback sections assert the
defined ENOSYS/ECHILD behavior so the six existing variants stay green.
- **Exit-status observation**: pre-fork, a separate non-ztest console-harness app
(`harness: console`, ordered regex — exit() in a ztest binary kills the suite);
post-fork, in-suite via child exit status (`test_exit`: atexit ordering folded into
`_Exit(counter)`; `test__Exit`/`test__exit`: handlers skipped, WEXITSTATUS asserted).
- `test_fork` sections: ENOSYS assertion wherever COW is off (all non-MMU
targets); COW sections on MMU (memory isolation via exit status, fd offset
sharing, atfork child ran). posix_spawn coverage lives in the POSIX_SPAWN
group's own suite.
- `test_execve`: build child LLEXT at test build time, embed, before-hook writes to
ramfs; child checksums argv/envp and exits with it. ENOENT/ENOEXEC aspects.
- Scenarios: existing six unchanged; `.process` (M1+, all four bring-up platforms);
`.process.exec` (M3+). Fix the `muti_process` typo (grep CI yamls for stale
references).
### ISO C consistency (user requirement)
Process support is designed against **both** specifications, interoperating at the
level POSIX and ISO C jointly specify — not POSIX alone:
- ISO C exit sequence honored exactly: returning from `main()` ≡ `exit(main's
value)`; `exit()` runs atexit handlers LIFO (≥32 registrations), flushes/closes
open streams, removes `tmpfile()` files, then terminates; `_Exit()`/`abort()` skip
handlers; `EXIT_SUCCESS`/`EXIT_FAILURE` round-trip through wait status.
- POSIX↔ISO C interop points made explicit: `_exit()` ≡ `_Exit()`; `abort()` ⇒
SIGABRT ⇒ `WIFSIGNALED` in the parent; `CLOCKS_PER_SEC`/`clock()` (ISO C) vs
`times()`/`_SC_CLK_TCK` (POSIX) unit consistency; `assert()` ⇒ abort path.
- The ISO C pieces live in the **libc layer** (common libc patches), never in the
POSIX module — the module only adds POSIX-only semantics on top.
### Conformance/docs
- doc table cells → `yes` + a **Tiers and deviations** section (COW vs spawn per
platform class; ENOSYS/ECHILD without CONFIG_PROCESS;
WUNTRACED/WCONTINUED accepted-never-reported; no job control).
- `_SC_CHILD_MAX` from new `CONFIG_POSIX_CHILD_MAX` (floor 25); `_SC_JOB_CONTROL`
stays -1; `_POSIX_SPAWN`/`_POSIX_JOB_CONTROL` stay out through M4.
- Audit `times()` µs clock_t vs `_SC_CLK_TCK` in M1.
## Phased roadmap
**Commit-order rule (user requirement)**: the Zephyr-kernel-native implementations
come **first** in the patch series, each landing together with **near-100% kernel test
coverage** (`tests/kernel/process/...`, following the k-signal.patch
tests-in-same-patch precedent) — the thin POSIX wrappers and their suites follow.
Zephyr needs native syscalls for essentially all of the multi-process functionality,
including waiting on other processes, and those syscalls + tests are the upstream
deliverable in their own right.
**Toolchain track (bottom-up, in parallel — user requirement)**: alongside the
kernel-up work, explore the bridge from the compiler/toolchain end: what CRT pieces
(crt0/crti/crtn, specs/linker integration, arch entry conventions) would live in
gcc/llvm and in each libc (minimal, picolibc, newlib, common) to reach
kernel/toolchain independence — how picolibc's existing crt0 machinery and newlib's
crt0.o conventions map onto the Zephyr process ABI, what `-nostartfiles`/specs
adjustments the executable build helper needs, and where the ABI contract must bend
to meet stock toolchain behavior. Deliverable: a process-ABI document + a prototype
crt0 in at least one libc, converging with the kernel ABI "in the middle" before the
exec phase freezes the contract.
- **M0 — Headers + API design at all three layers** (declarations before any
implementation; user requirement): **headers first across kernel, ISO C, and
POSIX** — the full API-interoperability design is done up front, revisable later by
rewriting history:
- *Kernel*: draft `include/zephyr/kernel/process.h` complete — `struct k_process`,
`K_PROCESS_DEFINE()` (both entry forms), pointer `k_pid_t`, `k_spawn`/`k_clone`
arg structs and flags, `k_waitpid`/`k_waitid`/`k_exit`/`k_getpid`/…, `K_WSTATUS_*`
and `K_PROCESS_W*` symbols, hook section — compile-clean with stub/ENOSYS bodies.
- *libc*: `_Exit` declaration, atexit/exit contract notes, crt0 entry contract
sketch in the process-ABI doc.
- *POSIX module*: sys/types.h completion (id_t, guarded timer_t), new sys/wait.h
(deriving from the kernel symbols) + libc shims, unistd.h check —
compile-tested across all six libc variants.
Then: file split, all 13 new function files with fallback semantics, real
pthread_atfork registry, all 23 ZTESTs (fallback sections), exit console-harness
app, typo fix, docs UB→defined. Proves the whole declared surface on every
platform/libc before implementation starts.
- **M1 STATUS (in progress)**: M1a (membership, static start, exit/wait) + M1b
(k_spawn, positive-reap tests) DONE, folded into commit `6739f0706`; kernel patch
`k-process-exit-wait.patch`, `tests/kernel/process` 10/10 on qemu_cortex_m3
(kernel+userspace), qemu_x86, qemu_riscv64. Two kernel bugs found+fixed:
halt-path wake used sched-lock-unsafe `z_unpend_all` then `z_unpend_first_thread`
(recursive sched lock) → `z_sched_wake_thread_locked` + explicit 0 return value
(a bare unpend leaves pended -EAGAIN, read as timeout). Also a CI overflow in the
M0b waitpid test (`~(WNOHANG|WUNTRACED)` narrows to int on riscv64) — folded into
M0b. **M1c DONE (4ffcc97a2)**: POSIX_MULTI_PROCESS selects PROCESS; getpid/ppid/
pgid/sid/setsid/wait/waitpid/waitid wrap k_process; kill/sigqueue process-aware;
process syscalls gated CONFIG_PROCESS||__DOXYGEN__ moved to process.h (syscall header
ifdef-registered); CLD_* added to picolibc/newlib signal.h shims. **M1d DONE
(7413f7aef)**: _exit→k_exit via COMMON_LIBC_UNDERSCORE_EXIT + spawn-child-libc-exit
test (kernel/process 11/11). M1c/M1d bundle their patch regens (fold at pre-upstream
cleanup). **Remaining M1**: M1e SIGCHLD emission on zombify — needs k_sig_info
si_pid/si_status extension (new k-process-signal patch) + halt-path-safe deferral
(set per-process sigchld_pending under sched lock, submit a shared scan-work whose
handler re-validates ZOMBIE state under process_lock to avoid slot-reuse races);
M1f init-process start (SYS_INIT static pid 1).
- **M1 — Process identity + static processes + exit/wait** (kernel core; **user
hand-implements**): `k_process`, **`K_PROCESS_DEFINE()` iterable-section statics**
(the reserved swapper/init/kthreadd trio + app-defined processes), pointer
`k_pid_t` + slot-index numeric pids, k_exit/k_waitpid, k_sig_info si_pid/si_status,
init process start (SYS_INIT static pid 1). Module: posix_this_pid switchover,
_exit → k_exit, strict kill(), siginfo fill. Common-libc exit/atexit/_Exit/clock
patches land here. **Statically defined processes are the API-coverage vehicle**:
test apps define child processes at build time and exercise wait/waitpid/waitid,
SIGCHLD, kill, getpgid/getsid, exit-status across real process boundaries — no
loader, no fork. **M1 must verify all three protection configurations**: MMU
(qemu_x86, qemu_cortex_a53), MPU (mps2/an385), and **no-MMU-no-MPU** (a
`CONFIG_USERSPACE=n` build — kernel-mode processes; e.g. qemu_riscv64 or
qemu_cortex_m3 without userspace). `.process` scenario green on all of them.
- **M2 — `k_spawn()` + posix_spawn** (no arch context-copy work needed — spawn creates
a fresh process from a spec): dynamic process creation on **all** targets including
no-protection ones; posix_spawn() thin wrapper (POSIX_SPAWN group + suite,
`_POSIX_SPAWN` enabled); wait/exit round-trips against spawned children; W* encoding
finalized. All four platforms.
- **M2.5 — `sys_spawn()` + POSIX_SPAWN option group**: the pool-allocating spawn
tier and the full `<spawn.h>` surface, on all four platforms, no MMU required,
no arch asm.
- *sys layer (`lib/os/process.c`)*: `sys_spawn()` draws the child's thread +
stack from the system thread pools (`SYS_THREAD`), builds `k_spawn_args`,
calls `k_spawn()`; resources return to the pools at reap via a leader-release
hook (pool-allocated leader threads/stacks must not be recycled before
`k_waitpid()` reaps the process, or a reused thread object would collide with
the unreaped zombie's leader identity; shape: a `__weak
z_process_leader_release()` called at the reap point outside `process_lock`,
overridden in lib/os, gated by a pool-owned-leader process flag). Includes
fixing a pre-existing `sys_thread_create_common()` bug its ordering depends
on: the `k_is_in_user_syscall()` branch passes `K_NO_WAIT` instead of the
caller's `delay`, so paused creation silently starts the thread immediately.
`SYS_SPAWN_*` constants as needed for attribute support.
- *POSIX layer (`POSIX_SPAWN` group, own suite per suite-per-group)*:
`posix_spawn()`, `posix_spawnp()`, `posix_spawnattr_t` init/destroy +
flags/pgroup/sigmask/sigdefault/schedparam/schedpolicy get/set,
`posix_spawn_file_actions_t` init/destroy + addopen/addclose/adddup2,
`POSIX_SPAWN_{RESETIDS,SETPGROUP,SETSCHEDPARAM,SETSCHEDULER,SETSIGDEF,
SETSIGMASK}` derived from `K_SPAWN_*`/`SYS_SPAWN_*` counterparts;
`_POSIX_SPAWN` enabled. pgroup → `k_setpgid()` at spawn; sigmask/sigdefault →
child signal state before start; file_actions honored once per-process fds
exist (documented deviation before then).
- *Open question (Chris)*: what an executable **path** names before M3 exec.
Proposal: a build-time registry of prelinked images (iterable section,
name → entry point) that `sys_spawn()` consults — the same prelinked-blob
story M3 already prioritizes for the MPU tier; unknown paths → `ENOENT`.
- `k_clone()` stays a declared `-ENOSYS` surface reserved for M4 COW fork.
- **M3 — exec via LLEXT** (later phase, per user): kernel process-ABI framework
(args/env block layout), libc crt0, Zephyr-side executable build helper,
`sys_process_exec/spawn` in lib/os, module exec* wrappers + PATH resolution,
FD_CLOEXEC sweep, syscalls-only export surface. an385 with ISOLATION_NONE; MMU
qemus with domain isolation. Freeze the args/env ABI here. **On the MPU tier,
prelinked-blob processes remain the prioritized/supported path; dynamic LLEXT exec
on constrained MPU targets is documented best-effort** (RAM fragmentation from
relocatable loading). From M3 onward, add at least one **physical board per tier**
alongside qemu (board choice: user's call) — qemu masks timing/cache/TLB costs of
ASID swapping and MPU reprogramming that silicon will expose.
- **M4 — COW fork on MMU** (kernel; **user hand-implements**): module delta ≈ zero (one
IS_ENABLED gate flips — atfork child handlers now run). COW sections activate on
qemu_x86/a53.
- **M5 — Groups/sessions + SIGCHLD completeness**: setsid/getpgid multi-process
semantics, kill(0/-pgid), waitpid(0/-pgid), waitid(P_PGID), WNOWAIT corners, times()
cutime/cstime, deviations doc final. (M6 future: job control.)
Gates per milestone: suite-scoped `runci.sh -T tests/posix/multi_process` (+signals,
threads_base for M0/M1) then full `runci.sh`; doc build at M4.
### Module-layer risks
1. execvp PATH policy (env PATH else Kconfig default) — needs a call.
2. Header-conflict posture (user direction): what matters is **posix-next ↔
Zephyr-native definition compatibility**, not agreement with toolchain headers.
**Independent C library headers must be independent** — where a toolchain header
(newlib/picolibc sys/wait.h etc.) would conflict, inject the correct redefinitions
at the `zephyr/lib/libc/<libc>/include/` shim layer, as done previously
(libc-sys-select-h, libc-statvfs-utime patterns). posix-next then defines its
surface without worrying about excessive conflicts.
3. `.userspace` × `.process` matrix is the hardest kernel corner — claimed only from
M4 (COW) onward.
4. linux_compat variant: process tests filtered `not CONFIG_NATIVE_LIBC` (host fork out
of scope).
---
## Design: kernel `k_process` + fork tiers
**Governing frame (user emphasis): a thread group is mostly synonymous with a process
from the kernel perspective.** `struct k_process` IS the thread-group object — the
Linux model, where the pid is the thread-group id. Everything follows from this:
`k_getpid()` returns the group's id from any member thread; process-directed signals
(`k_sig_queue`) target the group (any eligible member delivers) while
`k_sig_queue_tid` targets one thread (the tgkill/kill distinction); `k_exit()`
terminates the whole group; `k_waitpid()` observes group termination; a thread created
by a member joins the group; per-process state (dispositions, fd table, resources) is
per-group state. No separate "process" concept exists apart from the group.
**Every thread group has a thread group leader** — the primary thread of the process
(`proc->group_leader`, a `struct k_thread *`): the initial thread at creation
(K_PROCESS_DEFINE initial thread, spawn's first thread, the clone child's thread),
re-anchored to the surviving thread on exec. Adopted leader-exit semantics (Linux
model, settled):
- **Leader exits early (pthread_exit-style)**: siblings are NOT killed and **no
promotion occurs** — the group persists, `getpid()` in siblings keeps returning the
group's pid, and group termination completes only when the last member exits (then
SIGCHLD → parent, reapable). Where Linux must keep a zombie leader `task_struct`
alive to anchor the TGID, Zephyr's design is structurally simpler: **the pid lives
in `struct k_process`, so the k_process object itself is the anchor** and the
leader's `k_thread` can be fully recycled; `group_leader` becomes NULL/stale-marked
until group exit.
- **Unhandled fatal fault in ANY member (leader or sibling)**: the entire group dies
immediately — the Zephyr analog of `do_group_exit()`: the fatal-error path
(`z_fatal_error`/`k_sys_fatal_error_handler`) invokes group exit with
`K_WSTATUS_SIGNALED(signo)` status (SEGV/BUS/ILL-class), aborting every member.
This is also the fault story for `test_*_fault` ZTESTs against processes.
### pid model (revised per user question: pointer handle is feasible and preferred)
- **`k_pid_t` = `struct k_process *`** — the idiomatic Zephyr handle, exactly as
`k_tid_t` is `struct k_thread *`. Kernel-native APIs (`k_waitpid`, `k_sig_queue`,
`k_getpgid`, …) take/return the pointer handle.
- **The numeric pid is derived from the pointer, not allocated** — a pointer is
itself numeric, and POSIX only requires `pid_t` to be a *signed integer type*. The
raw pointer value can't be the user-visible pid directly (kernel addresses with the
high bit set collide with the `waitpid`/`kill` sign encodings `0`/`-1`/`-pgid`, and
it would leak kernel addresses to user mode), so the numeric pid is the **slot
index**: `pid = (proc - base)` across the reserved trio + statics section + pool —
a small positive int, pure arithmetic in both directions
(`k_pid_of(proc)`/`k_process_by_pid(int)`), **no pid allocator or table at all**.
Slot reuse = ordinary pid reuse (an optional generation counter can detect staleness
if wanted). `K_PID_SWAPPER`=0, `K_PID_INIT`=1, `K_PID_KTHREADD`=2 name the reserved
numeric ids; the reserved handles are `&k_process_swapper` etc. The thin POSIX
wrappers translate handle↔index at the boundary.
- **Lifetime**: the zombie IS the `k_process`, so a parent's handle stays valid until
reap (`k_waitpid` returns); handles held past reap dangle when the slot recycles.
Mitigation: `K_OBJ_PROCESS` becomes a tracked kernel-object type — statics from
`K_PROCESS_DEFINE` are discovered by `gen_kobject_list` like every other static
kernel object, pool slots use `k_object_recycle`, and syscall-boundary validation
(`K_SYSCALL_OBJ`) rejects stale/foreign pointers. (Cost: perms-bitmap bytes per
process object — acceptable at CONFIG_PROCESS_MAX scale.)
- All existing `k_pid_t` users are in the local series/module — a coordinated flag-day
inside `k-process-core.patch` is safe and **compiler-enforced** (pointer→int flips the
type). `k_sig_queue()` becomes process-directed; new `k_sig_queue_tid()` carries
today's thread-directed semantics (pthread_kill); in-series callers
(alarm/msgq/aio/timer/cancel) are switched per the existing `TODO(k_process)` markers.
### struct k_process (kernel/process.c, include/zephyr/kernel/process.h — new)
pid/pgid/sid, parent + children dlist, thread-group dlist (`k_thread.process` +
`process_node` beside `mem_domain_info`), state (`K_PROCESS_ACTIVE` /
`K_PROCESS_EXITING` / `K_PROCESS_ZOMBIE` — all public symbols namespaced
`k_`/`K_`/`sys_`/`SYS_` per Zephyr convention), latched
`wstatus`, wait queue, `k_mem_domain *domain` (owned/borrowed/default), per-process
sparse sigaction table, utime/stime + cutime/cstime cycle accounting, `res[]` opaque
slots for upper layers (zvfs fd map/cwd).
**Static definition via macro + iterable section (the Zephyr idiom — user requirement):**
```c
/* Defines: struct k_process in STRUCT_SECTION_ITERABLE(k_process, name),
* K_THREAD_STACK_DEFINE(name##_stack, stack_size), the initial k_thread,
* and a partition-pointer array for the process's domain. */
K_PROCESS_DEFINE(name, stack_size, entry, p1, p2, p3, prio, options, delay,
...partitions);
/* Companions (reuse existing machinery): */
K_APPMEM_PARTITION_DEFINE(name_part); /* existing — process-private data */
K_APP_DMEM(name_part) int my_global; /* existing — tag globals into it */
K_PROCESS_ACCESS_GRANT(name, ...); /* forwards to K_THREAD_ACCESS_GRANT
on the initial thread */
```
- Parameter set mirrors `K_THREAD_DEFINE` (entry/args/stack/prio/options/delay — the
initial-thread spec; `K_USER` in `options` selects unprivileged execution) plus a
trailing partition list for the process's `k_mem_domain`. **`K_PROCESS_DEFINE()`
composes the existing macros internally** (`K_THREAD_STACK_DEFINE`, the static
thread-definition machinery, partition array construction) rather than reinventing
them — one user-facing macro, existing building blocks underneath.
- **Entry forms (user requirement — no LLEXT dependency)**: the entry may be
(a) a **resolved/named symbol** in a specific memory domain of the process (code in
the shared kernel image or in a process partition), or (b) a **bare numeric
address** inside the process's domain, where no symbol name is known. Form (b)
enables the key pattern: **prelink an ELF, convert it to a byte array via Zephyr's
cmake utilities** (`generate_inc_file_for_target()`-style include-file generation,
not bare objcopy artifacts) **included into the firmware image** (placed in a
process partition), and Zephyr loads it as a process using the `K_PROCESS_DEFINE()`
parameters — full process support with no LLEXT, no filesystem, no runtime loader.
- **Same look and feel on MMU and non-MMU**, with under-the-hood differences: on MMU
platforms the common pattern is a **fixed per-arch virtual entry address** — adopt
the entry-point/text-base conventions Linux uses per architecture (e.g. the
arch-default ET_EXEC link addresses) so prelinked blobs are directly buildable with
stock toolchain defaults. **MMU-less systems do NOT use a common virtual entry**:
each process gets a **different physically addressed region** for its initial
address space + protection area (per-process prelink addresses; the build helper
assigns non-overlapping regions).
- **MMU-less memory-safety realities (user notes)**: `malloc` is optional, but
processes on MMU-less systems still share the global `k_malloc()` pool, so
cross-process corruption is possible — mitigate with per-process heaps (the
resource-ledger `k_heap`) where configured, and with **hardware MPU protection of
the running process** where slot budget allows. Where MPU regions are scarce,
explore **sharing and dynamically updating MPU regions based on the currently
running process** (Zephyr already reprograms the MPU from the thread's domain at
context switch — this extends that to a per-process app region). Swapping a process
out is a separate topic — explicitly out of scope for this plan.
- Private data comes from `K_APPMEM_PARTITION_DEFINE` + `K_APP_DMEM`/`K_APP_BMEM`
tagging (existing linker-section machinery); MMIO/peripheral access is just another
partition; kernel object rights use the existing grant/inherit model.
- **Image permission machinery is process-layer, not LLEXT-layer**: mapping a process
image's partitions with correct W^X (RX text / RW data) uses
`k_mem_update_flags()`/domain partitions directly and must work with LLEXT disabled;
LLEXT's `llext_adjust_mmu_permissions()` becomes just one client of it.
- Boot `SYS_INIT` walks `STRUCT_SECTION_FOREACH(k_process, ...)`: numeric pids follow
section order after the reserved trio, `k_mem_domain_init` with the partition list,
create the initial thread paused (`sys_thread_create_paused` precedent), add it to
domain + thread group, start after `delay` (K_THREAD_DEFINE autostart semantics).
- **Dynamic processes are managed by an elastipool (user decision)**: spawn/clone
children come from a `sys_elastipool` (the series' elastipool.patch API, already
used for the module's pooled objects) rather than a bare static array —
`CONFIG_PROCESS_MAX`-sized static region with the elastipool's growth semantics.
The elastipool is also how the number of active processes is managed, and process
allocation/destruction will need **fairly elaborate custom handlers** (domain
setup/teardown, kobject recycle, resource-ledger reclaim, zombie retention until
reap — the elastipool's alloc/free hooks are the seam for them).
Note for patch 1: numeric-pid-as-slot-index derivation must use the elastipool's
static region indexing; entries from any dynamic-growth region need an index from
the elastipool API (or a small spillover map) — verify what sys_elastipool exposes
and constrain accordingly.
- **`K_OBJ_PROCESS` is a new kernel object type** with the 4-character code **"PROC"**
(matching the 4-char code convention of other Zephyr kernel object types), wired
through `gen_kobject_list.py` exactly as the series' K_OBJ_FILE work did — statics
discovered at build time, pool entries via `k_object_recycle`, `K_SYSCALL_OBJ`
validation at the syscall boundary.
- **Three kernel-reserved processes defined by the kernel itself** (via an internal
no-initial-thread variant of the macro — their threads already exist and are adopted
at boot):
- `k_process_swapper` — **pid 0**, the swapper/idle process: exactly the per-CPU
idle threads (Zephyr already has one idle thread per CPU in SMP — the idle process
is the logical extension of that). Does nothing but sleep when no other task is
runnable. Dummy/boot threads also start here. (No POSIX conflict:
`kill(0)`/`waitpid(0)` mean "caller's pgrp", so pid 0 is never a valid explicit
target.)
- `k_process_init` — **pid 1**, the init process: parent of **all userspace threads
and processes**; its **own** initial thread and stack (it does NOT adopt
`z_main_thread` — see boot flow below); orphans reparent to it; it auto-reaps.
**Existence is NOT gated on USERSPACE**: pid 1 exists whenever `CONFIG_PROCESS`
is enabled, because it also parents "root"-owned processes running with
supervisor/superuser privileges (on `!USERSPACE` builds, all processes are
effectively root); USERSPACE affects isolation, never existence.
- `k_process_kthreadd` — **pid 2**, kthreadd: parent process of **all kernel
threads** (sysworkq, logging, driver/net threads, …).
- No pid allocator: numeric pids are slot indices derived from the handle (see pid
model) — 0–2 are the reserved trio, statics and pool slots follow in section order.
`K_PID_NONE` (if kept) is `NULL` at the handle level.
- `getppid()` of init returns 0 (swapper) — consistent with the earlier "root ppid 0".
**Boot flow under CONFIG_PROCESS (main() repurposed; resolves the system-thread
ownership question):**
1. Idle threads (one per CPU) belong to **swapper (pid 0)**. `z_main_thread` starts in
swapper for bring-up; kernel/system threads created during init levels (sysworkq,
logging, driver threads, …) are assigned to **kthreadd (pid 2)** — membership rule:
idle → swapper; kernel-mode threads created during boot or by kthreadd members →
kthreadd; everything else inherits the creator's process (userspace threads and
processes thus descend from init, pid 1). The exact K_USER-vs-inheritance edge for
supervisor-mode app threads in ISOLATION_NONE configs is a patch-1 review detail.
2. Image `main()` keeps today's semantics; no `bg_thread_main` change is required for
process support.
3. The statically defined **init process (pid 1)** starts from a `SYS_INIT` at boot,
independent of `main()`. Its initial thread runs the application entry — the
`K_PROCESS_DEFINE`-style entry of a kernel-supplied default init process
(weak/Kconfig-selectable symbol — ztest supplies it for `.process` test scenarios),
or exec of an init executable (`/sbin/init` via LLEXT) in exec-capable configs.
Exact default-entry naming is an implementation decision for patch 1/2 review.
4. When `!CONFIG_PROCESS`, today's semantics are unchanged (main() is the app).
### API naming (user decision)
Process syscalls use the **bare `k_` prefix with the typical POSIX names** — the
convention the series already set with `k_sig_queue`/`k_sig_timedwait`:
`k_getpid()`, `k_getppid()`, `k_getpgid()`, `k_getsid()`, `k_setsid()`,
`k_execve()`, `k_waitpid()` / `k_waitid()`, `k_exit()` (no collision with
the existing `k_thread_exit()` — distinct scope), plus the creation primitives
`k_spawn()` and `k_clone()` (Linux-inspired names rather than POSIX ones — see
Process-creation primitives). The **object** keeps its full name:
`struct k_process`, `K_PROCESS_DEFINE()`, `k_process_current()` (object accessor, no
POSIX analogue). Internal helpers stay `z_process_*`. Wherever this plan says
`k_process_<verb>()` for a POSIX-named operation, read the `k_<posix-name>()` form;
final spellings reviewed in patch 1. (`kill()` needs no new syscall — it wraps the
process-directed `k_sig_queue()`.)
### Lifecycle
- `k_exit(code)`: latch status, abort all other group threads, self-abort.
Per-thread hook `z_process_exit_thread()` in the existing halt epilogue
(kernel/sched.c:1395, beside `z_mem_domain_exit_thread`/perms-clear). Last thread out
zombifies: run `K_PROCESS_HOOK` exit callbacks (iterable section — fd close, cwd,
timer/msgq retarget), tear down owned address space, reparent children to init
(init auto-reaps zombie orphans), emit SIGCHLD (unless parent IGN → auto-reap), wake
waiters. **The `k_process` object IS the zombie** — all k_threads recycle normally;
pid/wstatus/times survive in the pool slot until reaped. No thread_state bit needed.
- `k_waitpid(selector, &reaped, &st, opts, timeout)`: the selector carries the POSIX
waitpid() encodings **natively in the handle domain** (user decision): a valid
handle = that child; `K_PID_ANY` ((k_pid_t)-1) = any child; `K_PID_MY_PGRP` (NULL) =
caller's process group; `K_PID_PGRP(leader)` (negated handle) = that group. No
WPGID flag. Constraint: sign-discrimination of negated handles requires process
objects at non-negative intptr_t addresses; otherwise validate-then-negate.
WNOHANG/WNOWAIT, pend on parent waitq, ECHILD when selection empty. Distinct from
`k_thread_rejoin` (thread results) — module must not conflate.
- `k_clone(args, args_size, &child)` models **clone3** (user decision, superseding the
earlier clone2 framing): extensible size-versioned `struct k_clone_args` (u64 flags
with Linux flag values, exit_signal, stack/stack_size; zero = defaults).
- kernel/process.c uses the shared `os` kernel log module (LOG_MODULE_DECLARE), not a
private one (user decision).
- Subsystem hook section `K_PROCESS_HOOK_DEFINE(fork, exec, exit)` keeps zvfs/LLEXT out
of the kernel core; `z_process_exec_begin/commit` are the exec contract.
### Signals
- `k_sig_info` gains `pid`/`status` (gated on CONFIG_PROCESS) + `K_SI_CLD_*` codes —
flows through the existing fifo/timedwait/SA_SIGINFO paths unchanged.
- Process-directed fifo entries (`proc` pointer alongside `tid`); first eligible
unmasked thread in the group receives (POSIX one-thread rule); zombie-process purge
mirrors the existing dead-thread purge.
- Global `signal_actions[]` moves behind `z_sig_actdb(proc)` — per-process sparse DB
(~28 B × DB size × processes), global retained for !PROCESS.
- STOP/CONT stay ignored; WUNTRACED → -ENOTSUP (job control out of scope).
### Process-creation primitives (user decision)
- **`k_spawn()` — first-class citizen for non-MMU targets** (MPU *and* targets with no
memory protection at all): creates a new process + initial thread directly from a
spec (image entry per the K_PROCESS_DEFINE entry forms, domain/partitions, args/env
block, initial-thread parameters) — no address-space duplication, **no return-twice
arch machinery needed** (fresh entry, no context copy), so it lands before any
`arch_thread_fork` work and does not require USERSPACE. POSIX `posix_spawn()` is a
**thin wrapper around `k_spawn()`** (it lives in the POSIX_SPAWN option group with
its own suite, per suite-per-group; `_POSIX_SPAWN` can be enabled when it lands —
earlier than previously planned). `k_spawn` is also the runtime backend for starting
statically defined processes.
- **`k_clone()` — the MMU-tier COW vehicle**, modeled on Linux's clone/clone3:
one general primitive taking a `struct k_clone_args`, whose only planned flavor
is the COW address-space copy (M4) — POSIX `fork()` calls `k_clone()` with
those flags directly. Declared `-ENOSYS` until then. Linux-shaped room to grow
retained (threads-as-clone-flavor is explicitly NOT a goal).
### fork tiers
- **Return-twice primitive** (needed only by k_clone's M4 COW flavor; not by
k_spawn/sys_spawn): `arch_thread_fork(child, parent)` per arch — builds child
context so its first schedule returns from the clone syscall with retval 0 at
the same user PC/SP. No setjmp tricks. No `returns_twice` annotation is needed:
each returner runs on its own address-space copy, so neither can touch the
other's frame or spill slots.
- **COW tier** (`CONFIG_PROCESS_FORK_COW`, x86 then arm64): **keep the global shared-VA
invariant; carve out a private window**. Reserve `CONFIG_PROCESS_VA_SIZE` (default
8 MB) in the global `virt_region_bitmap` at boot; the invariant becomes "shared
outside the process window". Page tables are already per-domain (x86 ptables/CR3,
arm64 TTBR0+ASID) — new arch APIs `arch_mem_domain_{map,unmap,protect,query}_private`
are thin non-propagating wrappers over existing internals (`ARCH_HAS_MEM_DOMAIN_
PRIVATE_MAP`). `struct k_process_mm` (per-process VA bitmap + mapping records) lives
in k_process, keeping `struct k_mem_domain` unchanged (upstream-friendly).
Page-frame refcount = parallel `uint16_t` array in kernel/mmu.c (the pattern the
page-frame comment prescribes); **refcount>1 ⇒ pinned** (simple, correct; reverse-map
later). COW fault handler `k_mem_cow_fault()` runs before demand paging on user write
faults. Fork walk: RO/shared → share+ref; RW private → write-protect both + COW-mark.
Initial constraint: **x86 !KPTI only**. COW-process thread stacks must be
window-mapped (Kconfig-enforced via the existing mem-mapped-stack machinery).
**Honest scope**: full fork isolation holds for private-window state (exec'd images,
window heaps/stacks); fork-without-exec from the init process shares image globals —
documented; fork+exec/posix_spawn is the supported story.
### Scheduling & per-core address-space switching (user concern — mostly already exists)
Requirement: threads from many different processes run concurrently; any thread is
loadable onto any of the N cores (unless pinned), with that thread's virtual-memory /
protection state applied before it resumes. Assessment against the tree:
- **The machinery exists in the context-switch path today.** The scheduler is
thread-granular and process-agnostic (consistent with thread-group ≈ process), and
each arch already applies the incoming thread's domain on switch: x86 loads the
domain's page tables into CR3 (`z_x86_swap_update_page_tables`,
arch/x86/core/userspace.c:40, called from both ia32 and intel64 switch paths);
arm64 swaps TTBR0 per-domain with ASIDs (`z_arm64_swap_ptables`) so cross-process
switches avoid full TLB flushes; ARM MPU reprograms regions from
`thread->mem_domain_info.mem_domain` at every context switch
(arch/arm/core/mpu/arm_core_mpu.c:238); RISC-V PMP likewise. Because `k_process`
wraps a `k_mem_domain` and threads carry their domain, per-core process switching
requires **no new scheduler work** — SMP included (each CPU switches independently;
threads of one process may run on several cores at once sharing the domain's
tables).
- **Real gaps to watch**: (1) SMP TLB shootdown — when COW/private-window mappings of
a domain change while sibling threads run on other cores, an IPI-based invalidate
is needed (bring-up is UP qemu; flagged in the COW risk list); (2) arm64 ASID
wraparound full-flush fallback; (3) x86 KPTI exclusion for the private window;
(4) MPU slot budget when adding a per-process app region to the per-switch
reprogramming set (the dynamic-MPU-region-sharing exploration above).
- **Swapping processes in/out of memory (paging a whole process) is a separate,
massive topic — explicitly out of scope**; the demand-paging substrate would be its
starting point if ever pursued.
### Per-process resources (user requirement: every non-kernel process owns its own)
Each non-kernel process (init and its descendants — not swapper/kthreadd, which use
the kernel's global state) owns:
- its **file descriptor table** (below),
- its **list of process-specific memory allocations and mappings** — generalized from
the COW-tier `k_process_mm`: a per-process resource ledger (mappings, heap/resource-
pool ownership, dynamically allocated kernel objects charged to the process) exists
for **all** processes under `CONFIG_PROCESS`, not just `PROCESS_FORK_COW`, so
process exit can reclaim everything the process created. **Mapped pages (MMU) are a
per-process resource, exactly like fds (user direction)**: `k_mem_map`-style mappings
are owned by the **thread-group leader** rather than any individual thread, tracked in
the process ledger, and inherently reclaimed when the leader exits (after the group's
child processes are killed) — the same lifecycle as the per-process fdtable. The COW
tier extends the same ledger with the private-VA window records; per-process heap =
a `k_heap` assigned at `K_PROCESS_DEFINE`/creation (the resource-pool model, promoted
from per-thread to per-process — retargets the `TODO(k_process)` pool-ownership marker),
- its cwd, environment block (app-owned per the ABI), and signal dispositions.
**Termination cleanup (user requirement)** — when a process terminates, its exit hook
must fully unwind the resources it owned:
- **Heap**: clear every allocation and return the backing memory to the kernel —
whether it was `k_mem_map`-mapped pages or a physical region granted from some
memory pool. Nothing the process allocated survives it.
- **Shared-memory objects** (POSIX `shm_*` etc.) are **separate from the heap**: they
are refcounted, with **exactly one reference per process** (not per mapping or per
thread). On termination each such object the process holds gets its refcount
decremented once, and the object is closed/destroyed only when no other process
still references it.
- **Design (user direction): make a shared-memory object a first-class kernel object
of a new "memory" type** (e.g. `K_OBJ_MEM` — Zephyr today has only `k_mem_slab`, a
slab allocator, and no generic memory-region kobject; `k_heap` isn't a kobject at
all). As a kobject it plugs straight into the existing grant/validate/refcount
machinery: a process is *granted* access to a memory object, access is checked at
the syscall boundary, and lifetime/reclaim ride the same per-process kobject
cleanup as everything else — the one-refcount-per-process rule is just the kobject
grant count. This also unifies "explicitly granted physical region" and
"shared-memory object" under one type.
### Per-process fds/cwd (user direction, revised)
Under `CONFIG_PROCESS` the **global fdtable is not compiled in at all** — the
`fd → fd_entry` handle is abstracted behind an `ALWAYS_INLINE static inline`
accessor in a zvfs internal header, so a `!PROCESS` build resolves to today's global
array (zero cost) and a `PROCESS` build resolves to `k_process_current()`'s
per-process fd table. Per-libc file-descriptor→`FILE *` glue may need updating to go
through the same accessor. CWD likewise becomes per-process. FD_CLOEXEC (already
stored in `fd_flags`) is finally honoured by the exec hook. The open-file-description
objects and their refcounts stay shared so `dup()` across a fork works.
### Sessions/groups
`k_process_{getpgid,getsid,setsid}` syscalls; setsid: -EPERM for group leader, else
sid=pgid=pid. fork inherits pgid/sid. `k_sig_queue(0/-pgid)` fans out. No setpgid/
terminals/job control.
### Kconfig
`CONFIG_PROCESS` does **NOT** require USERSPACE — identity/exit/wait/SIGCHLD/pgid work
for kernel-mode threads on every platform incl. native_sim (huge CI surface for the
early patches). Isolation (both fork tiers) requires USERSPACE. Zero overhead when off.
### Kernel patch series (appended after existing 116; **USER** = hand-implement)
| # | Patch | Risk | Who |
|---|-------|------|-----|
| 1 | `k-process-core.patch` (struct, pool, pid alloc, k_pid_t flag-day, hooks, getpid/getppid) | Med | **USER** core; fixups mechanical |
| 2 | `k-process-exit-wait.patch` (exit, zombify, reparent, wait) | Med-high (SMP halt ordering) | **USER** |
| 3 | `k-signal-process.patch` (si_pid/status, process-directed delivery, per-proc DB, SIGCHLD) | Med | **USER** |
| 4 | `k-process-pgrp-session.patch` | Low | mechanical |
| 5 | `k-spawn.patch` (first-class spawn: process+thread from spec, all targets, no USERSPACE requirement; backend for static-process start) | Med | **USER** review; largely mechanical over patch 1 |
| 5a-d | `arch-{x86,arm64}-thread-fork.patch` (for M4 k_clone COW; MMU arches only) | High | **USER** |
| 7 | `mmu-page-frame-refcount.patch` | Low-med | mechanical + USER review |
| 8 | `mmu-domain-private-map.patch` (VA window, k_process_mm, 4 arch APIs x86+arm64) | High | **USER** |
| 9 | `k-clone-cow.patch` (COW clone flavor, backs POSIX fork(): fork walk, COW fault, syscall) | Highest | **USER** |
| 10 | `zvfs-per-process-fd.patch` | Med | mechanical |
| 11 | `zvfs-per-process-cwd.patch` | Low | mechanical |
| 12 | `k-process-exec-hooks.patch` (exec_begin/commit contract) | Med | **USER** + exec coordination |
| 13 | `k-timer-msgq-process-retarget.patch` (the TODO(k_process) sites) | Low | mechanical |
After 1–4: processes/wait/SIGCHLD on every platform (kernel-mode too). After 5 +
M2.5 sys_spawn: spawn-ready on all four boards (exec-ready with M3). After 7–9:
COW fork on qemu_x86(!KPTI)/a53.
### Kernel risks
SMP zombify-vs-halt ordering (patch 2, review hardest); refcount>1-pinned memory
inflation; arm64 ASID wraparound full-flush fallback; KPTI exclusion; shared signal
fifo cross-process starvation; **kobject permissions move from the per-thread bitmap
to the process (user direction)** — grants/queries key on `k_process` instead of the
thread, so `CONFIG_MAX_THREAD_BYTES`=2 becomes a cap of **16 userspace *processes***
(not threads), with effectively unlimited threads per process all sharing the
process's permission bits; this is a significant change to how the kernel grants and
validates object access. Upstream buy-in for
the Kconfig.vm wording change + 4 arch APIs (private-window formulation minimizes
friction; k_mem_domain untouched).
Additional discipline items (external review, adopted):
- **`__syscall` annotation rigor**: every user-callable process API must carry the
`__syscall` signature + generated `z_vrfy_` boilerplate — a missing annotation
silently breaks the user-mode boundary and the minimal export surface; audit as
part of each kernel patch's review.
- **pthread↔k_thread mapping stability**: new `k_thread` fields (`process`,
`process_node`) must not disturb the module's existing thread-handle mapping
(`to_k_thread()` etc.) — accessor-based, so appending fields is safe, but verify no
layout/offset assumptions exist.
- **Coverage flush ordering**: gcov data must be flushed before process partitions
are torn down — with dynamically loaded images, the reaper must sync coverage
counters before `llext_teardown`/partition reclaim, or exec'd-code coverage is
lost (interacts with the coverage-dump-at-init-exit move).
---
## Cross-design reconciliations
1. **`_exit` ownership**: common libc owns `_exit`/`_Exit` (the
`libc-common-under-exit.patch`, `#ifdef CONFIG_PROCESS → k_process_exit`); the
module does NOT add `_exit.c` (unistd.h declaration only). Supersedes the module
agent's `_exit.c`.
2. **wstatus encoding — REVISED (kernel-native symbols requirement)**: the kernel
owns the encoding via **public** `K_WSTATUS_*` macros (Linux-compatible bit layout
so toolchain sys/wait.h macros agree) and `K_PROCESS_W*` wait flags; the module's
W*/WNOHANG/… are thin derivations of those kernel symbols. The one POSIX-ism left
in the wrapper: `waitpid()` re-encodes the signo field from kernel numbering to
POSIX numbering at return (bit layout identical, only the signal number mapped),
so user code's `WTERMSIG` yields POSIX signos.
3. **Kconfig**: the POSIX group stays the single user-facing unit
(`POSIX_MULTI_PROCESS`); kernel symbols (`PROCESS`, `PROCESS_FORK`,
`PROCESS_FORK_COW`) + promptless module helpers do the gating; the exec agent's
`POSIX_EXEC`-style symbols become promptless internals (select LLEXT export groups,
isolation choice).
4. **fork() on the MMU-less tier**: fork()=ENOSYS on MMU-less; `posix_spawn()`
(M2.5 `sys_spawn()` + POSIX_SPAWN group) is the MMU-less process-creation
story. Module test impact: `test_fork` asserts ENOSYS + COW sections on MMU;
POSIX_SPAWN gets its own suite.
5. **exec image lifetime**: k_process gets an image-destructor hook (folded into patch
12) so the reaper tears down the LLEXT image; exec agent's deferred-unload flow
depends on it.
6. **an385**: a spawned child sharing the parent's domain costs 0 extra MPU
regions; post-exec domain needs merged LLEXT partitions (phase-2 patch) —
phase 1 runs exec'd processes with ISOLATION_NONE.
7. **Exec core placement**: per the POSIX/kernel-separation requirement, the exec flow
lands as `sys_process_exec/spawn` in lib/os (new `sys-process-exec.patch`); the
module keeps only PATH resolution, varargs marshalling, and errno mapping. The
args/env memory layout is **kernel ABI**; **crt0 is provided by the C library**
(below POSIX); the executable build helper (cmake, crt0 + static libc link over
`add_llext_target`) is Zephyr-side. The POSIX module ships no crt0, no export
unit, no build helper.
## Ownership / working agreement (revised)
- **Claude drafts, Chris reviews**: Claude produces first drafts/proposals for ALL
parts, including the deep kernel/arch pieces previously marked hand-implement;
Chris actively reviews and will likely rewrite many parts over multiple drafts and
iterations. The patch table's **USER** markers now mean "highest review scrutiny",
not "Claude doesn't draft".
- **Pause at the end of each milestone/phase** to review what was done — likely via
GitHub code review on the module branch (`posix-multi-process`).
- **The plan is synced to `~/Desktop/posix-multi-process.md`** after each round of
plan edits.
## Verification
- Per milestone: `runci.sh -T tests/posix/multi_process` (+ `tests/posix/signals`,
`tests/posix/threads_base` when touched), then full `runci.sh`; doc build at M4.
- **CI matrix management**: the platform × libc × protection-tier × scenario matrix
is large — shard twister runs in CI **grouped by protection tier** (MMU / MPU /
no-protection) rather than by platform, to keep cache reuse high and local
`runci.sh` runs scoped; wire into the existing CI-container/shard setup from day
one of M1.
- New kernel patches get their own Zephyr-tree tests (`tests/kernel/process`) following
the k-signal.patch precedent (patch adds tests alongside implementation).
- Patch hygiene: `west patch clean && west update && west patch apply` after edits;
sha256sums via the patch-series-rebase workflow; smoke build on all four platforms.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment