Skip to content

Instantly share code, notes, and snippets.

@savarin
Last active June 27, 2026 22:01
Show Gist options
  • Select an option

  • Save savarin/324a740b2f334e3d0232a78bd6d807a8 to your computer and use it in GitHub Desktop.

Select an option

Save savarin/324a740b2f334e3d0232a78bd6d807a8 to your computer and use it in GitHub Desktop.
The Rewrite: Python Edition — Chapter 14 — Conclusion

← Back to Index

Chapter 14 — Conclusion

Intro

A rewrite is a theory about what caused the problems in the original system. If the theory is wrong — if you attribute the problems to the wrong things — the rewrite produces a cleaner codebase with the same failure modes, or different ones. The theory behind this rewrite is specific: v1's problems came from three accumulated misalignments between the system's type contracts and its runtime behavior, between its topology and its operational requirements, and between its single-agent assumptions and its multi-agent ambitions. The rewrite's answer was not to add features or fix bugs one at a time. It was to surface the misalignments explicitly — through a shape audit, a parity checklist, and a gap roadmap — and resolve them at the seam level before writing the implementation. This chapter draws out the three recurring design moves that appear across every part of the rewrite, and what they mean for the systems you will build next.

Three Moves, Applied Across Every Part

  SHAPE AUDIT (Ch 2, 13)          SINGLE SEAM (Ch 4, 8, 10, 12)
  ┌────────────────────┐          ┌──────────────────────────┐
  │ What does the type │          │ Each capability has ONE  │
  │ contract promise   │          │ owner, ONE interface,    │
  │ that the runtime   │          │ ONE point of change:     │
  │ does not deliver?  │          │                          │
  │                    │          │ SessionFactory (Ch 4)    │
  │ profile.tools: 0   │          │ ToolRegistry   (Ch 8)    │
  │   readers          │          │ narrow_for_    (Ch 10)   │
  │ iteration_budget:  │          │   subagent               │
  │   never checked    │          │ wire module    (Ch 12)   │
  │ instructions:      │          │                          │
  │   static, no hooks │          │ 7 keepalive sites → 1   │
  └────────────────────┘          └──────────────────────────┘
           │                                  │
           └──────────┬───────────────────────┘
                      │
                      ▼
          DURABLE ENVELOPE (Ch 5, 6, 7, 10)
          ┌────────────────────────────────┐
          │ The unit of work is self-      │
          │ contained across process       │
          │ boundaries:                    │
          │                                │
          │ Submission (Ch 5, 7)           │
          │   session_key + credentials    │
          │   + input + lease              │
          │                                │
          │ Journal (Ch 6)                 │
          │   4 phases → exact resume      │
          │   point after any crash        │
          │                                │
          │ Coordinate path (Ch 10)        │
          │   structural position →        │
          │   deterministic node ID        │
          └────────────────────────────────┘

Sections

1. The First Move: The Shape Audit

The shape audit asks one question with real teeth: what does the type contract promise that the runtime does not deliver? It is not a code review. It is a comparison between the declared seam — the interface — and the wired behavior — the implementation. Chapter 13 formalizes this as a gap list, but the move was introduced in Chapter 2, when the book read Anthropic's claude-code and OpenAI's codex-rs independently, across nine dimensions, before a line of v2 production code was written. The point of that exercise was not to copy either system. The point was that two independent auditors, working from different source trees in different languages, arrived at the same five greppable facts about the v2 runtime's current state. When two analyses agree without coordination, that is corroboration, not coincidence. The shape audit borrows its credibility from that cross-harness convergence.

The two diagnostic questions the audit applies are structural. First: is there a field that is declared but has zero readers? In AgentProfile, both tools and skills are present in the type. build_toolset reads only profile.subagents. The fields exist in the contract; the implementation ignores both. Chapter 2 flagged this as one of the five agreed-upon code facts; Chapter 13 names it G0 and ranks it first because it is the cheapest gap with the most unlock. Second: is there a constraint that is specified but never enforced? iteration_budget is declared in contracts.py at the budget field, read nowhere in the executor. RunResult has no stop_reason field, so callers have no way to know whether a turn stopped because the model chose to stop or because it hit the budget. Chapter 13 names this G1.

The discipline of the shape audit is not in finding the gaps — a careful read of the source always finds them. The discipline is in ranking them. G2 — static instructions with no per-turn context assembly — blocks seven distinct features: memory recall, per-turn skill injection, dynamic instruction override, mid-turn fallback, steering messages, context compaction hooks, and tool-surface customization. G5 — the detached multi-agent graph — reshapes the primitive from a synchronous tree into a peer graph with message passing. The ranking is the rewrite's theory about causality: fix the root, and the features that depend on it become cheap. Skip the root and patch the symptoms, and you get a cleaner codebase with the same feature ceilings. The audit's build order — G0 → G1 → G2 → G3 → G4 → (G5 if required) → G6 — is derived from that leverage ranking, not from editorial preference.

2. The Second Move: The Single Seam

Every time the rewrite drew a boundary, it tried to draw it in exactly one place. The principle sounds simple: a capability should have one owner, one interface, and one point of change. The implication is sharper than it sounds. When a capability is split across multiple owners, you can satisfy the invariant in one place while violating it in another. You cannot test the invariant in isolation, because there is no isolation — the invariant is distributed across the codebase, held together only by convention and careful reading.

Chapter 12's DROP list is the clearest evidence of what happens when this principle is absent. Keepalive logic in v1 lived in seven separate places: the adapter, the harness, the SDK layer, keepalive.py, _is_busy, _CY_ACTIVE_RUNS, the autostop pong. The underlying requirement — a long turn or the approval park does not get killed by idle-suspension — is real and non-negotiable. But the mechanism was spread because each component that touched the keep-alive concern added its own version of the fix. When the topology shifted to ECS, where the runtime is always-up and idle-suspension is not a threat, every one of those seven places needed to change. The requirement was met; the mechanism was dead weight. One seam, changed in one place, is a refactor. Seven places, changed in seven places, is a coordination problem that produces subtle divergences in the meantime.

The specific seams the rewrite drew:

narrow_for_subagent in context.py (Chapter 10) is the single seam for subagent isolation. When a parent agent delegates to a subagent, the subagent receives an AgentContext with chat: null — so messaging tools structurally do not exist for it — and denyAllApprovals — so it cannot drive the approval side-channel even if it tries. These invariants are enforced in narrow_for_subagent, not in the callers. There are callers in the workflow executor, in the direct delegation path, and in the reconciler's repair path. Each of them passes through narrow_for_subagent. None of them re-implements the isolation logic. That is what a single seam means in practice: the callers do not know how isolation works, only that it is enforced.

wire.py (Chapter 12) is the single seam for the runtime-to-sandbox contract. The runtime runs on ECS; the sandbox runs on a per-tenant Fly machine. Both ends import the same TypedDict definitions and string constants, so method names and parameter shapes stay in sync by convention: a developer who changes one side sees the shared module and knows to update the other. HelloParams, ExecParams, SandboxFsParams — the shared definitions reduce drift, though they do not eliminate it the way a discriminated-union constructor would (a wrong method name is still a valid string at the type level). The alternative is two separately maintained interface definitions that diverge silently. wire.py is a drift-reduction mechanism, not a compile-time guarantee — but in practice, a single shared module catches most of the errors that two separate definitions would miss.

load_workflow_source (Chapter 10) is the single seam for workflow compilation. Parse, hash, validate, and eager compile-check against a throwaway subprocess — all of this happens in load_workflow_source, before any agent() calls are dispatched, before any durable state is written. A syntax error on line 200 of a script fails immediately at load time, not after 199 successful lines of dispatched subagents and written records. The eager check is what makes the failure clean. Without it, you get partially-initialized durable state from a script whose control flow was never valid.

AgentContext (Chapters 3 and 4) is the single seam for capability injection. Every edge the agent can reach — session, sandbox, chat, model, store — is named in AgentContext. SessionFactory is the one place that assembles those edges into a running system; the comment in runtime.py says it explicitly: "The GLOBAL session factory — the SINGLE construction seam for every session origin." Every session origin — a human prompt to the root agent, a workflow node being dispatched, a subagent being delegated to — flows through SessionFactory. There is no second path. The invariant that every capability is injected through the declared interface is not maintained by convention; it is enforced by the type system, because there is no other way to construct a session.

3. The Third Move: The Durable Envelope

A durable envelope makes the unit of work self-contained across process boundaries. It carries everything the receiver needs to process it, including its own identity. It does not depend on ambient state on the sender's side. The reason this matters is simple: in-process state is not reliable across the boundaries your system actually crosses. Process restarts, machine migrations, network partitions — these all kill in-process state. The envelope is the move you make when you stop pretending that in-process state is reliable and start packaging the state into the message.

The Submission object (Chapters 5 and 7) is the clearest example. In v1, a turn was driven by a boolean flag and a live promise. If the process died, the promise died with it. The store had the last checkpoint, but the runtime had no record of where in the turn the process was when it died. A new process would start a fresh turn, not resume the interrupted one. The work was redone; tool calls that had side effects fired twice; the user might receive two responses. The Submission in v2 is the fix: it carries the session key, the per-tenant credentials, and the input, all in a single durable record. The lease mechanism — lease_owner, lease_expiry, retry_budget — is the distributed lock that v1's turn_in_flight boolean was not. When a process dies mid-turn, the lease expires, another process claims the submission, reads the journal, and resumes from the exact phase the previous process reached. The pair of submission and journal gives end-to-end durability: no prompt is silently dropped, no turn is silently abandoned.

The workflow node's coordinate path (Chapter 10) is a durable envelope for its identity. In a workflow that fans out to parallel subagents, each node needs a stable identifier that survives the run: an identifier that can be written to the store before the node executes and read back after a crash to resume. The coordinate path encodes the node's structural position in the execution tree — wf:deep-research@sha256:abc123:run-id-42:par-0/agent-0002 is fully self-describing. The shape par-0/agent-0002 says "the third agent inside the first parallel block." That position is deterministic: given the same script and the same run id, with_coordinate produces the same segment at the same call site every time. This is the property that makes the coordinate path a durable envelope rather than a runtime handle: the node's id is derived at call time from its structural position, not from when it happens to run. A resume can reconstruct the graph from the store without needing any in-process state from the original run.

The two trust boundaries in this system — the first drawn around tool execution in Chapter 9, the second drawn around agent-authored code in Chapter 11 — also express the durable envelope pattern, though in a different register. The SessionEnv that wraps each sandbox session derives its working directory from a deterministic hash of the session's storage key. Two concurrent agents never share a filesystem path, because the path is computed from the key, not from a runtime allocation. The sandboxed subprocess in Chapter 11 enforces a harder boundary: each run gets its own process, its own namespace, its own counter state. There is no shared closure between runs. The sandbox is a durable envelope in the sense that its state is fully internal — no run can observe or corrupt another run's state, because there is no shared ambient state between them.

4. What This Means for the Next Five Features

The three moves — shape audit, single seam, durable envelope — are not techniques invented for this rewrite. Interfaces-before-implementations, single points of change, self-contained messages: these are the standard moves of systems that need to survive at scale. What the rewrite made visible is that they were missing. v1 had capabilities split across multiple owners; contracts that promised fields the runtime ignored; work units that depended on ambient state that did not survive process restarts. The rewrite did not invent new solutions. It applied known patterns to a codebase that had accumulated the debt of not applying them.

Gap Roadmap: Seam Status

  G0 (tools/skills)    seam EXISTS    → wire impl         ░░ 1 afternoon
  G1 (budget/stop)     seam EXISTS    → wire impl         ░░ 1 afternoon
  G2 (context assembly)  NO SEAM     → draw seam + impl  ████ arch work
  G3 (event hooks)     seam EXISTS    → extend contract   ░░░ moderate
  G4 (code-mode)       NO SEAM       → new edge           █████ new system
  G5 (multi-agent)     WRONG SHAPE   → reshape primitive  ████████ major
  G6 (widenings)       seams EXIST   → extend as needed   ░ incremental

  ░ = additive (impl behind existing shape)
  █ = structural (new seam or reshaped primitive)

The question those three moves answer for the next five features is: where is the seam, and does it already exist? For G0 — wiring tools and skills from AgentProfile — the seam is already in place. AgentProfile declares the fields; build_toolset just needs a resolver that reads them. This is one afternoon of work. For G2 — per-turn context assembly — there is no seam yet. The per-turn hook does not exist; the assembly function has no defined signature; the callers have no interface to depend on. G2 is architectural work: you have to draw the seam before you can fill it. For G5 — the detached multi-agent graph — the seam that exists is synchronous. delegate is await-synchronous; a parent cannot proceed until the subagent completes. Reshaping that to peer-graph message passing changes RunAgent at the recursion level, which changes the executor, the Coordinator, the resume machinery, and the coordinate path's assumptions about tree depth. G5 is not a gap; it is a different system.

The shape audit tells you which category each feature falls into before you start implementing it. That is the value: not finding bugs, but classifying work. Features that fit an existing seam are pulls. Features that require a new seam are architectural additions. Features that reshape a primitive are rewrites. The classification is not editorial — it follows from reading the contracts, finding the seam, and asking whether the seam supports the new behavior without modification. A team that runs this check before each feature addition does not accumulate the kind of debt that makes a rewrite necessary in the first place. Chapter 2's methodology — reading other runtimes deliberately, comparing the same dimensions across multiple systems — is the check for the whole-system case. Chapter 13's gap roadmap is the check for the incremental-feature case. The discipline is the same in both.

5. The gap roadmap as a living document

Chapter 13 ends with the gap list (G0–G6) and a verdict: the shape is mostly right; the implementation has not yet caught up to it. That verdict is a specific claim about the distance between the contract and the behavior, not a general assurance that everything is fine. The gap roadmap is the document that makes that distance legible and trackable. When G0 is wired, the document updates: tools and skills have readers, build_toolset consumes them, the approval seam has an implementation. When a new gap is found — a new field with zero readers, a new constraint with no enforcement — it gets added. The document's job is to keep the codebase honest about what the contracts promise and what the code delivers.

The build order from the shape audit — G0 → G1 → G2 → G3 → G4 → (G5 if required) → G6 as needed — is a dependency graph, not a priority list. G0 unblocks G2 because per-turn context assembly depends on the skills seam existing; you cannot assemble skills per-turn if the skills field has no consumer. G2 unblocks G3 and G4 because the loop hooks (G3) and the code-mode reverse transport (G4) both require a per-turn assembly point to attach to. The ordering is the rewrite's theory about causality: close the root gap, and the dependent features become additive work rather than architectural work.

This is the same pattern Chapter 2 used when comparing claude-code and codex-rs. Both audits found the same five facts, ranked the same gaps, and recommended the same deferred item — G5, the detached multi-agent graph — because both recognized that changing the recursive tree into a peer graph is the one move that changes the primitive's fundamental shape. Deferring G5 is not procrastination; it is the correct reading of the dependency graph. The completion-injection seam already exists (deliver_trigger accepting role: 'signal'); only the producer is missing. When G5 becomes a hard product requirement, the cheap half — a background-run handle, a per-child stop signal, a durable spawn-edge store — can be shipped first. The peer-mailbox waits until it is required. A gap roadmap that distinguishes between "additive" and "reshapes the primitive" is the tool that makes this call correctly.


Conclusion

Three moves recur across every layer of this rewrite: the shape audit finds the gaps between contract and behavior; the single seam concentrates a capability at one point of change; the durable envelope makes the unit of work self-contained across the boundaries it actually crosses. None of these moves requires a specific framework or language. They are structural choices, and they are the choices this rewrite made legible by naming them.

Chapter 2 introduced the shape audit as a methodology: two independent analyses of different source trees, nine dimensions each, no shared notes, same five greppable conclusions. The convergence was the point — when claude-code and codex-rs, built independently for different models by different teams, both identify the same unwired fields and the same deferred primitive change, that is not opinion. That is the shape the problem space tends toward. Chapter 13 applied the same methodology inward: read the current contracts, compare to the current implementation, rank the gaps by leverage. Seven gaps — four additive (G0, G1, G3, G6), one structural seam (G2), one new edge (G4), one deferred primitive reshape (G5) — a build order, and a clear distinction between gaps that fit existing seams and gaps that would change the shape.

Chapter 4 named the SessionFactory the single construction seam — the one place where every capability is assembled into a running session. Chapter 10's narrow_for_subagent is the single seam for subagent isolation. Chapter 12's wire.py is the single seam for the runtime-to-sandbox contract. Chapter 10's load_workflow_source is the single seam for workflow compilation. In each case, the seam is the move that converts a distributed invariant — held together by convention across multiple files — into a structural guarantee held by one interface. The DROP list in Chapter 12 is the record of what it costs when that move is not made: keepalive logic in seven places, each place a potential site for invariant violation, all seven needing to change when the topology shifts.

Chapter 5 described the Submission as the unit of durable work: a record in the store before the turn begins, a lease that survives process death, a journal that names the exact phase where the runtime was when it stopped. Chapters 6 and 7 traced what that durability enables: classify_submission_state as a pure function over the stored record, seven resume modes, zero silent abandonments. Chapter 10's coordinate path encoded the workflow node's identity as its structural position — not a runtime allocation, not an ambient counter, but a deterministic segment derived from the script's shape and the run id. The coordinate path means a workflow can be resumed by a process that has never seen the original run, from a record that is fully self-describing.

The v2 runtime is not finished. The gap roadmap is honest about that. iteration_budget is declared and unread. stop_reason does not exist on RunResult. The per-turn context assembly seam does not yet have a signature. The code-mode reverse transport — the path from inside the sandbox back into the agent loop — is structurally missing. These are not oversights; they are the next chapter of work, with a build order derived from the cross-harness leverage ranking rather than from guessing.

What the rewrite accomplished is not completeness. It is the condition under which completeness is tractable. Before the rewrite, the contracts promised fields that the runtime ignored, capabilities were distributed across owners with no single point of change, and work units depended on in-process state that did not survive restarts. Adding features to that system meant touching multiple owners, reasoning about distributed invariants, and discovering at deploy time that the work depended on ambient state that was already gone. After the rewrite, AgentContext names every capability at the seam level, SessionFactory is the one place those capabilities are assembled, and Submission plus the journal give the executor a complete picture of where any unit of work stands, regardless of which process is asking.

The DDIA ending model applies here: the real insight from a systems project is rarely the specific code you wrote. It is the mental model you built that lets you reason about the next change without reading every file. For the turn loop: drive-to-idle, journal the phases, the inner loop is unchanged between v1 and v2. For the boundary: one seam, one owner, the invariant lives in one place. For the unit of work: package everything the receiver needs into the envelope, derive identity from structure rather than allocating it at runtime. Those three moves are not specific to agent runtimes. They are the moves any system needs when it crosses process boundaries, runs at multiple tenants, and cannot afford to lose work quietly.

The shape is now in the right place. That is what this rewrite delivered — not a finished system, but a system whose incompleteness is legible. The gap roadmap names what remains. The build order says what to close first. The seam structure means that when you close G0, G1, G2 in order, each fix is additive: the contract already declares the right shape, and the implementation catches up to it. You do not need a third rewrite. You need to wire the declared fields, enforce the declared constraints, and draw the one new seam that G2 requires. The work is tractable because the shape tells you where to start.


Files read for this chapter:

  • docs/the-rewrite/chapters/02-reading-other-peoples-runtimes.md — cross-harness methodology, five convergent code facts, nine-dimension audit framework, corroboration as evidence.
  • docs/the-rewrite/chapters/03-contracts.mdAgentContext, five v2 seams, contracts as design tools, the ratio of seams to implementations.
  • docs/the-rewrite/chapters/04-the-single-construction-seam.mdSessionFactory as the single construction seam, create_runtime wiring, the invariant the single seam enforces.
  • docs/the-rewrite/chapters/05-the-turn.mddrive_run, AgentSubmission, v1 boolean/promise vs v2 journal phases, the inner loop as the rewrite's unchanged core.
  • docs/the-rewrite/chapters/07-resume-and-repair.mdclassify_submission_state, seven resume modes, the lease mechanism as distributed lock, pure-function classifier design.
  • docs/the-rewrite/chapters/10-workflows.md — coordinate path structure, load_workflow_source eager compile-check, parallel barrier semantics, WorkflowDef fresh-isolate guarantee.
  • docs/the-rewrite/chapters/10-workflows.mdnarrow_for_subagent, chat: null, denyAllApprovals, subagent isolation via construction seam.
  • docs/the-rewrite/chapters/11-the-isolate.md — two-layer isolation (subprocess + Fly machine), DoS node budget, scope bridge, determinism enforcement.
  • docs/the-rewrite/chapters/12-from-runtime-to-fleet.md — DROP list, keepalive in seven v1 places, wire.py as single seam, ECS always-up topology, FlySandboxEnv.
  • docs/the-rewrite/chapters/13-the-shape-gaps.md — G0–G6 gap definitions, leverage ranking, build order, gap roadmap as living document.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment