Skip to content

Instantly share code, notes, and snippets.

@galligan
Last active August 21, 2026 22:17
Show Gist options
  • Select an option

  • Save galligan/86ba18f0839db3aad9bea0f7a85a2eb7 to your computer and use it in GitHub Desktop.

Select an option

Save galligan/86ba18f0839db3aad9bea0f7a85a2eb7 to your computer and use it in GitHub Desktop.
Codex app-server file-descriptor exhaustion on macOS: diagnosis, two-layer mitigation (raise limit + reap idle children), shell-specific gotchas, and troubleshooting

Codex app-server file-descriptor exhaustion on macOS

Codex / ChatGPT Desktop features start failing one at a time — Computer Use usually dies first — while open chats and already-connected MCP servers keep working. The cause is file descriptor exhaustion in the codex app-server process against macOS's default 256 soft limit, made worse by a known upstream leak where MCP stdio children are never reaped.

This is a containment guide. Upstream tracking issues are linked at the bottom; none are fixed as of 2026-08-21.


Symptoms

  • Computer Use / browser automation silently stops working; other features look fine

  • Skipped loading N skill(s) due to invalid SKILL.md files — with a different set of skills each launch (the files are not actually invalid)

  • Tool calls fail with what looks like a script bug rather than a resource problem:

    exec_command failed for `/bin/zsh -lc "..."`:
      CreateProcess { message: "Rejected(\"Failed to create unified exec process: Too many open files (os error 24)\")" }
    
  • In the app-server log:

    failed to read file: Too many open files (os error 24)
    failed to spawn process: Too many open files (os error 24)
    

The asymmetry is the tell: established pipes keep working, only new process creation fails. Computer Use spawns helpers on demand, so it is the canary — not the cause.


Confirm it is this

Resolve the app-server PID. comm must be exactly codex, otherwise you match the /bin/sh bootstrap wrapper whose arguments embed the same string:

APPSERVER=$(ps -axo pid=,comm=,args= | awk '$2=="codex" && index($0,"app-server"){print $1; exit}')
echo "app-server pid: $APPSERVER"
# fd count vs the limit
lsof -p "$APPSERVER" | wc -l
launchctl limit maxfiles          # default: 256  unlimited

# the leak signature: pipes ~= 3x child processes
lsof -p "$APPSERVER" | awk '$5=="PIPE"' | wc -l
ps -Ao ppid | awk -v p="$APPSERVER" '$1==p' | wc -l

A ratio near 3.0 (stdin/stdout/stderr per child) with dozens of long-lived children confirms it. Independent reports and my own measurement both landed on 3.10.

Count the children that have been alive a long time — these are the leaked ones:

ps -axo pid=,ppid=,etime=,comm= | awk -v p="$APPSERVER" '$2==p'

Root cause

Two independent problems stack:

  1. The default limit is low. launchctl limit maxfiles is 256 unlimited on macOS. Any launchd-started GUI app, and any SSH session, inherits a 256 soft limit. The app-server spawns dozens of children by design and never raises its own limit via setrlimit.

  2. Children are never reaped (upstream #26984). The parent never closes its side of the pipes, so the child's stdin never sees EOF — meaning any MCP shutdown path that depends on stdin EOF cannot fire. MCP server authors cannot fix this from the transport side. Reporters have found children idle for four days still holding pipes.

Because of (2), raising the limit alone does not hold. One reporter measured bursts of +74 fds/minute; a bigger budget just moves the wall.


Layer 1 — raise the soft limit

Do not raise it to a huge value. The fd limit is the only brake on process accumulation. With a very high limit the failure mode changes from a legible EMFILE to hundreds of node processes until memory pressure, which is harder to diagnose and worse to recover from. 1024 is the recommended target.

Which file actually runs?

The SSH bootstrap invokes your login shell non-interactively ($SHELL -c '...'). Which startup file gets sourced differs per shell, and this is where the fix silently fails:

Shell Sourced by a non-interactive -c? Put the snippet in
fish yes, config.fish ~/.config/fish/config.fish
zsh only ~/.zshenv (not .zshrc, not .zprofile) ~/.zshenv
bash nothing, unless BASH_ENV is set see note below

All three verified empirically on macOS 26 / Darwin 27.

fish

# Codex app-server fd headroom — workaround for openai/codex#26984
if set -q SSH_CONNECTION
    set -l __fd_soft (ulimit -S -n 2>/dev/null)
    if string match -qr '^[0-9]+$' -- "$__fd_soft"; and test "$__fd_soft" -lt 1024
        ulimit -S -n 1024
    end
end

zsh — must be ~/.zshenv

# Codex app-server fd headroom — workaround for openai/codex#26984
if [ -n "${SSH_CONNECTION:-}" ]; then
  __fd_soft=$(ulimit -S -n 2>/dev/null)
  case "$__fd_soft" in
    ''|*[!0-9]*) : ;;                                   # "unlimited" etc — leave alone
    *) [ "$__fd_soft" -lt 1024 ] && ulimit -S -n 1024 2>/dev/null ;;
  esac
  unset __fd_soft
fi

bash

bash -c sources no startup file. If bash is your login shell, either switch the relevant account to zsh/fish, or set BASH_ENV in ~/.ssh/environment (which requires PermitUserEnvironment yes in sshd_config — a server-side change with its own security tradeoffs). The LaunchDaemon below is the simpler path for bash users.

Why the SSH_CONNECTION guard and the -lt 1024 test matter

  • SSH_CONNECTION scopes the change to SSH sessions, where the 256 limit applies. Local interactive shells often already sit far higher.
  • -lt 1024 means it only ever raises. An unguarded ulimit -S -n 1024 would lower a shell already at 1048576, which can break file-hungry builds.
  • The numeric test avoids an error when ulimit returns the string unlimited.

GUI-launched app-servers — LaunchDaemon

The shell snippet only covers the SSH-bootstrapped app-server. A desktop app launching its own bundled app-server never goes through your shell. For those, raise the limit at boot.

Save as /Library/LaunchDaemons/limit.maxfiles.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>limit.maxfiles</string>
    <key>ProgramArguments</key>
    <array>
      <string>launchctl</string>
      <string>limit</string>
      <string>maxfiles</string>
      <string>1024</string>
      <string>200000</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>ServiceIPC</key>
    <false/>
  </dict>
</plist>
sudo install -o root -g wheel -m 644 limit.maxfiles.plist /Library/LaunchDaemons/limit.maxfiles.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/limit.maxfiles.plist
# older macOS: sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist
launchctl limit maxfiles        # expect: 1024  200000

Check your kernel caps first — a hard value above either of these may be clamped or rejected, which looks like success while doing nothing:

sysctl kern.maxfilesperproc kern.maxfiles

Existing processes keep their old limit. The app must be restarted to inherit the new one.

Reboot durability: verified. Cold-boot tested on macOS 26 / Darwin 27 — the LaunchDaemon loaded in the system domain before login, so launchctl limit maxfiles read 1024 on a fresh boot with no intervention, and every process started afterwards inherited it. Still worth confirming on your own machine after the first restart rather than assuming.

Expect the hard value to read back as unlimited. The plist above requests 1024 200000, but macOS reports maxfiles 1024 unlimited — it preserves the pre-existing unlimited hard cap. This is benign: unlimited is more permissive than what was requested, and the soft value is what constrains processes. Do not chase it.

A trap: do not wrap ~/.local/bin/codex

The upstream thread suggests a PATH wrapper. On a standalone install, ~/.local/bin/codex is a symlink managed by the updater (→ ~/.codex/packages/standalone/current/bin/codex). A wrapper placed there works until the next update silently replaces it, reverting the fix with no signal. Hijacking CODEX_INSTALL_DIR has the same class of problem — that variable is the updater's install target. Prefer the shell hook.


Layer 2 — reap idle children

Raising the limit alone did not hold for anyone in the upstream thread. This is what did.

Save as ~/.local/bin/codex-reap-idle-children.sh, chmod +x:

#!/bin/sh
# Reap idle Codex app-server child processes.
# Workaround for openai/codex#26984 (MCP stdio children are never reaped).
#
# Env overrides:
#   CODEX_REAP_FD_THRESHOLD  only act above this fd count   (default 700)
#   CODEX_REAP_MIN_AGE       min child age in seconds       (default 300)
#   CODEX_REAP_DRY_RUN       set to 1 to report and not kill
#   CODEX_REAP_EXCLUDE       ERE of comm/args to never kill (default protects
#                            Computer Use clients, which may hold live state)
set -u

FD_THRESHOLD=${CODEX_REAP_FD_THRESHOLD:-700}
MIN_AGE=${CODEX_REAP_MIN_AGE:-300}
DRY=${CODEX_REAP_DRY_RUN:-0}
EXCLUDE=${CODEX_REAP_EXCLUDE:-SkyComputerUse}
LOGFILE=${CODEX_REAP_LOG:-$HOME/.codex/app-server-control/reaper.log}

log() { printf '%s %s\n' "$(date '+%Y-%m-%dT%H:%M:%S')" "$*" >>"$LOGFILE" 2>/dev/null; }

# 1. Resolve the app-server PID. comm must be exactly "codex" so the /bin/sh
#    bootstrap wrapper (whose args embed the same string) is never matched.
APPSERVER=$(ps -axo pid=,comm=,args= | awk '
  $2=="codex" && index($0,"app-server") { print $1; exit }')
[ -n "${APPSERVER:-}" ] || { log "no app-server process found"; exit 0; }

# 2. Only act under pressure.
FDS=$(lsof -p "$APPSERVER" 2>/dev/null | wc -l | tr -d ' ')
[ "${FDS:-0}" -ge "$FD_THRESHOLD" ] || { [ "$DRY" = 1 ] && echo "fd=$FDS below threshold $FD_THRESHOLD, nothing to do"; exit 0; }

# 3. ONE ps snapshot. Per-child ps calls are too slow to finish in the interval.
CANDIDATES=$(ps -axo pid=,ppid=,etime=,comm= | awk -v parent="$APPSERVER" -v minage="$MIN_AGE" -v excl="$EXCLUDE" '
  { p[NR]=$1; pp[NR]=$2; et[NR]=$3; line[NR]=$0; isparent[$2]=1; n=NR }
  END {
    for (i=1; i<=n; i++) {
      if (pp[i] != parent) continue          # direct children only
      if (p[i] in isparent) continue         # has descendants -> may host live work
      if (excl != "" && line[i] ~ excl) continue   # protected by CODEX_REAP_EXCLUDE
      e=et[i]; d=0
      if (index(e,"-")) { split(e,a,"-"); d=a[1]+0; e=a[2] }
      m=split(e,t,":")
      s = (m==3) ? t[1]*3600+t[2]*60+t[3] : t[1]*60+t[2]
      s += d*86400
      if (s >= minage) print p[i]
    }
  }')

[ -n "$CANDIDATES" ] || { log "fd=$FDS no eligible children"; exit 0; }
COUNT=$(printf '%s\n' "$CANDIDATES" | wc -l | tr -d ' ')

if [ "$DRY" = 1 ]; then
  echo "appserver=$APPSERVER fd=$FDS eligible=$COUNT"
  printf '%s\n' "$CANDIDATES" | while read -r p; do
    [ -n "$p" ] && echo "  would TERM $p  $(ps -o etime=,comm= -p "$p" 2>/dev/null | tr -s ' ')"
  done
  exit 0
fi

# 4. xargs -n1: `kill "$pids"` on a whitespace-joined list silently no-ops in zsh.
printf '%s\n' "$CANDIDATES" | xargs -n1 kill -TERM 2>/dev/null

sleep 2
SURVIVED=0
for p in $CANDIDATES; do kill -0 "$p" 2>/dev/null && SURVIVED=$((SURVIVED+1)); done
FDS_AFTER=$(lsof -p "$APPSERVER" 2>/dev/null | wc -l | tr -d ' ')
log "appserver=$APPSERVER fd ${FDS}->${FDS_AFTER} reaped=$COUNT survived=$SURVIVED"

Always dry-run first:

CODEX_REAP_DRY_RUN=1 CODEX_REAP_FD_THRESHOLD=100 ~/.local/bin/codex-reap-idle-children.sh

Selection criteria — why each one

  • Direct children of the app-server only. Never walk the whole tree.
  • No descendants of their own. A child that is itself a parent may be hosting live work.
  • Age ≥ 300s. Note this is elapsed time since start, not true idle time — ps gives no idle metric. The fd threshold and the no-descendants rule are what make this safe; age alone is a weak proxy.
  • Exclude stateful helpers. Reaping is safe for stateless node_repl workers. It is not safe for a stateful MCP server; the default CODEX_REAP_EXCLUDE protects Computer Use clients.
  • One ps snapshot. Per-child ps calls are too slow to finish inside a 60s interval.

Two refinements worth adding

Write a status line on every run. Otherwise a quiet reaper.log is ambiguous between "nothing to do" and "the timer never fired":

STATUSFILE=${CODEX_REAP_STATUS:-$HOME/.codex/app-server-control/reaper.status}
status() { printf '%s %s\n' "$(date '+%Y-%m-%dT%H:%M:%S')" "$*" >"$STATUSFILE" 2>/dev/null; }

Call it on every exit path — below-threshold, no-candidates, and after reaping. One line, overwritten each time.

Refuse to reap when a competing app-server exists. If a bundled desktop app-server is running alongside the managed one, the topology is not consolidated and its children belong to live threads. Target only the managed daemon, and warn rather than reaping blindly:

BUNDLED=$(ps -Ao args | awk 'index($0,"ChatGPT.app/Contents/Resources/codex") && index($0,"app-server") && !index($0,"--listen stdio://") && !/awk/' | wc -l | tr -d ' ')
[ "$BUNDLED" -gt 0 ] && log "WARN $BUNDLED bundled app-server(s) present -- children NOT reaped"

launchd agent

Save as ~/Library/LaunchAgents/local.codex.reap.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>local.codex.reap</string>
    <key>ProgramArguments</key>
    <array>
      <string>/bin/sh</string>
      <string>--</string>
      <string>REPLACE_WITH_ABSOLUTE_PATH/codex-reap-idle-children.sh</string>
    </array>
    <key>StartInterval</key>
    <integer>60</integer>
    <key>RunAtLoad</key>
    <false/>
    <key>StandardErrorPath</key>
    <string>/tmp/codex-reap.err</string>
  </dict>
</plist>

launchd does not expand ~, so use an absolute path.

launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/local.codex.reap.plist
launchctl print gui/$(id -u)/local.codex.reap | head -20
tail -f ~/.codex/app-server-control/reaper.log

To remove: launchctl bootout gui/$(id -u)/local.codex.reap


Verifying the raised limit actually took

macOS does not let you read another process's RLIMIT_NOFILE. Two traps:

  • Do not use fd count. Once reaping runs, the count stays low, so the evidence never appears.
  • Use the highest fd number. The kernel allocates the lowest free descriptor, so a number above 256 proves the limit is above 256.
lsof -p "$APPSERVER" | awk 'NR>1{gsub(/[^0-9]/,"",$4); if($4+0>m) m=$4+0} END{print "highest fd:", m}'

The limit only applies to processes started after it was raised — restart the app-server.


Troubleshooting

The snippet has no effect. Wrong startup file — see the shell table above. zsh -c reads only ~/.zshenv. Verify your config is actually running inside the bootstrap by checking the app-server's own environment for a variable your config sets:

ps -E -p "$APPSERVER" | tr ' ' '\n' | grep -E 'SSH_CONNECTION|SHELL='

launchctl limit maxfiles still shows 256 after a reboot. The LaunchDaemon did not load. Check ownership (root:wheel, mode 644) and sudo launchctl print system/limit.maxfiles. Verify after a reboot, not just after bootstrap — that is the only proof of durability.

Hard limit appears ignored. If it reads back as unlimited, that is expected and fine (see above). If it reads back lower than requested, it exceeded kern.maxfilesperproc or kern.maxfiles and was clamped — lower it below both.

The reaper reports success but nothing dies. In zsh, kill -TERM "$pids" on a whitespace-joined list passes one invalid argument and, with stderr suppressed, appears to succeed. Use xargs -n1 kill -TERM and verify each PID with kill -0.

Grep finds nothing in the app-server log. If grep is aliased to ugrep, its default -I treats the ANSI-colored log as binary and reports zero matches. Use awk/tail, or grep -a.

Your verification script reports all-pass and you do not believe it. Prove the checker can fail before trusting a green result — feed it a known-bad value and confirm a FAIL appears. A real example from this work: a comparison written as [ got = want ] || [ got -ge want ] made every "must be 0" and "must be 1" check pass for any value at or above the target, so the two checks that detect a split-brain topology could never fire while the script reported "8 passed, 0 failed". Also give the checker a third state for conditions it cannot evaluate — scoring those as pass or fail is how false conclusions get reported.

Everything is fine but Computer Use is still dead. Restart the app-server — helpers that failed to spawn are not retried.

The topology/ownership checks pass but nothing is running. bundled app-servers = 0 and persistence owners = 1 are trivially true when the desktop app is not running. Confirm the app is actually up before treating either as a green signal.


Caveats

  • This is containment, not a fix. Children are still spawned per session and still never reaped by the daemon; the timer just deletes them.
  • Reaping assumes idle children are safe to kill. True for stateless workers, not for stateful MCP servers.
  • Consolidating multiple clients onto a single shared app-server makes this significantly worse: one process then carries every client's MCP servers, threads, and helpers against the same ceiling.

Upstream issues

Issue Title
#26984 MCP stdio servers leak pipe fds + orphan child processes → cumulative EMFILE
#39446 [macOS] Desktop main process holds ~243 FDs and hits the default 256 limit
#36755 Skill loader mislabels transient EMFILE as "invalid SKILL.md files"
#27662 Codex Desktop/app-server exhausts syspolicyd, causing global spctl EMFILE
#28071 Desktop repeatedly exhausts syspolicyd, cannot relaunch until reboot

The real fix belongs in the parent process: close pipes on child exit, reap children, and call setrlimit to raise its own soft limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment