The v1 runtime worked. Turns completed, tools executed, users got replies. If you ran it in production, it did what it was supposed to do. But if you tried to write a test for the turn loop in isolation — to drive just the loop, with a fake model and no platform connection — you couldn't. The platform and the loop were fused. Every improvement to the turn hit the same wall. That's not a bug. That's a ceiling.
This chapter is about how that ceiling forms, what it looks like in code, and why the only way past it is to change the shape rather than patch the contents.
A turn is the unit of agent work: one user message in, zero or more tool calls, one assistant reply out. Simple to describe. Surprisingly hard to bound.
The model doesn't stop after one tool call. It receives the tool result, decides whether to call another tool, calls it, receives that result, and continues — until it reaches a stopping point it considers natural, or until the context window fills, or until something aborts the run. The runtime's job is to drive that inner loop to idle. Not to manage it tool-by-tool, not to supervise each step, but to start it and wait for it to stop.
The turn boundary is where you checkpoint. Once the model is idle — once the loop has settled — you write the new messages to durable storage. If the process crashes before that write, the turn can be replayed. If the write succeeds, the turn is committed and the session is ready for the next message. The checkpoint is the only durable event in the turn lifecycle. Everything between start() and checkpoint() is in-flight.
Everything else — keepalive pings, approval gating, platform rendering, compaction — sits around the turn, not inside it. The rewrite starts by making that distinction clean.
In v1, Session is the composition root for a turn. It holds the message buffer, drives the pi agent loop, handles compaction, manages platform connection state, and responds to keepalive signals. All in one class.
session/
├── session.ts — Session class: the composition root
├── reconcile.ts — turn repair (interrupted tool batches, partial streams)
├── stream-chunks.ts — in-flight stream chunk persistence
└── index.ts — public re-exports
This isn't a design mistake. It's how things accumulate. The session had to do everything because there was nowhere else to put it. The first time you needed keepalive, you added it to Session. The first time you needed repair logic, it went in there too. Each addition was correct in context. The problem is that, over time, the session's state machine became the only place where turn logic lived.
Session (v1)
├── messages: MessageBuffer
├── platform: FlyConnection
├── compactor: Compactor
├── keepalive: IntervalHandle
└── drive(): async void
├── prompt / continue
├── waitForIdle()
├── checkpoint()
├── isContextOverflow? → compact + continue
└── shouldCompact? → compact
The platform and the loop share a lifecycle. To drive the loop, you need a Session. To have a Session, you need a FlyConnection. You can't test the loop without the platform. You can't run a subagent without pulling in the platform — or without building a fake Session that carries all the same machinery the real one does.
This is the structural problem. Not that the code is messy. The code is actually quite organized. The problem is that the concepts are coupled. The turn loop — the smallest meaningful unit of agent work — is not a thing you can hold in your hand and hand to someone. It lives inside the session, and the session lives inside the platform.
The first extraction was drive_run, pulled verbatim from Session into core/run-agent.ts. The function is small:
async def drive_run(loop: RunLoop, opts: DriveRunOptions) -> None:
await opts.start()
await loop.wait_for_idle()
await opts.checkpoint()
trailing = trailing_assistant(loop.state.messages)
if trailing and is_context_overflow(trailing, loop.state.model.context_window or 0):
# Reactive compaction: compact regardless of threshold, then continue.
outcome = await opts.compactor.compact(loop.state.messages, loop.state.model)
if outcome.summarized and not outcome.needs_new_session:
loop.state.messages = outcome.messages
await loop.continue_()
await loop.wait_for_idle()
await opts.checkpoint()
return
# Proactive threshold compaction after a settled turn.
if (
trailing
and trailing.stop_reason != "aborted"
and trailing.stop_reason != "error"
and opts.compactor.should_compact(loop.state.messages, loop.state.model)
):
outcome = await opts.compactor.compact(loop.state.messages, loop.state.model)
if outcome.summarized and not outcome.needs_new_session:
loop.state.messages = outcome.messages
# proactive compaction is best-effort — not re-checkpointed;
# on restart the settled turn re-compacts idempotently.
if opts.on_compacted is not None:
opts.on_compacted(len(outcome.messages))drive_run takes a RunLoop — a minimal Protocol with wait_for_idle() and continue_(). The pi agent satisfies this Protocol directly. The function adds exactly two things pi doesn't own: overflow recovery (compact-and-continue when the model hits a context limit) and proactive compaction (fold the transcript after a clean turn, before the window fills). Everything else stays in the caller.
The DriveRunOptions type makes the dependencies explicit:
from dataclasses import dataclass
from typing import Callable, Awaitable
@dataclass(frozen=True)
class DriveRunOptions:
start: Callable[[], Awaitable[None]]
compactor: Compactor
checkpoint: Callable[[], Awaitable[None]]
on_compacted: Callable[[int], None] | None = NoneNo FlyConnection. No keepalive. No platform. The function is a pure procedure over injected dependencies. You can call it with a real pi agent and get a real turn. You can call it with a fake RunLoop and a stub compactor and get a test. The same function handles root agent runs and subagent runs — the interface doesn't care which you're running.
This is the shape the rest of the rewrite is aimed at. Not drive_run specifically, but the principle it demonstrates: a turn is a pure function over an injected loop, a start callback, a compactor, and a checkpoint. Platform differences go in the injected edges. The loop itself is stable.
The file header is explicit about the intent:
drive_runis the settle-and-compact engine extracted verbatim from the Session, so the root (its persistent, subscribed piAgent) and a subagent (a fresh ephemeral loop) share ONE path: drive the loop to idle, checkpoint, then the Hermes-style overflow-recovery + proactive-threshold compaction pass.
One path for root and subagent. That's the goal. drive_run is the proof that it's achievable.
The ARCHITECTURE.md for apps/runtime-v1 makes a commitment that most architecture documents avoid: it says the system is mid-migration and prints two columns.
| Status | Where | |
|---|---|---|
| Target seam model | Recursive RunAgent primitive, AgentProfile → AgentContext binding, first-party Chat, RunSink/ProgressReducer, PersonDirectory. |
core/contracts.ts |
| Wired today | Runtime + one interactive Session per thread, Vercel Chat SDK adapters, drive_run extracted as the shared run engine. |
harness.ts, router.ts, session/, adapters/ |
This table is useful precisely because it's honest. Most architecture documents describe the target. This one describes both the target and where the system actually is, and marks the difference clearly. The orange nodes in the Mermaid diagrams — ApprovalGate, RunSink, PersonDirectory — are "declared but not yet fully wired." They exist as interface definitions in core/contracts.ts. They have stubs or placeholder implementations. But the edges aren't connected.
The bridge has already landed: drive_run in core/run-agent.ts is the shared run engine. The session is still wired — the Session class in session/session.ts still owns the loop and drives it. The gap between drive_run as a function and Session as a class is the rewrite.
What makes this useful as documentation is that you can read the gap directly. The target is contracts.ts. The current state is harness.ts + session/. The work is moving behavior from one to the other, seam by seam.
The PARITY_CHECKLIST for apps/runtime lists several items under "artifacts to DROP / re-home." These aren't features being cut — they're mechanisms that existed only because of the Fly VM autosuspend topology. Understanding them is understanding the ceiling.
Keepalive / _is_busy / _CY_ACTIVE_RUNS. Seven sections in the checklist assert this. The keepalive loop's job was to prevent a co-located Fly VM from idle-suspending mid-turn. When a turn runs for five minutes and the VM autosuspends after three, the turn dies. The keepalive loop prevented that by pinging the VM to signal activity. On the new harness, the infrastructure is always-up. Flue's lease-and-heartbeat mechanism covers the requirement without the mechanism. The checklist verdict: "DROP the mechanism."
fly-force-instance-id VM wake. Each tenant ran on a specific Fly VM. When a message arrived for that tenant, the gateway had to wake that specific VM before delivering the message. On a pool-based executor, you claim a worker from the pool. There's no per-tenant VM to wake.
Loopback api_server routine detour. The harness and sandbox shared a VM filesystem. Some internal calls looped back through a local API server rather than calling directly, because the call had to cross a process boundary that existed only because of the shared filesystem. When Submission carries per-tenant credentials natively, the detour dissolves.
SQLite session split-rotate. Session storage used SQLite on the VM's local filesystem, with a rotation scheme to manage file size. The requirement — durable transcript plus lineage plus full-text search — carries over. The mechanism doesn't. The new harness uses an append-only journal in Postgres.
The checklist is explicit: "Carry the requirement, drop the artifact." That's a useful decomposition. The requirement is real. The mechanism was an implementation of that requirement under a specific topology. When the topology changes, the mechanism may not map, but the requirement still holds.
What's instructive is that each of these mechanisms became load-bearing. The keepalive loop wasn't just a background task — it was threaded into the session's state machine. The VM wake was embedded in the transport layer. The SQLite rotation was the primary persistence path. You couldn't surgically remove any of them without restructuring the code they'd grown into.
Here's the v1 shape in abbreviated form:
v1 shape:
Platforms (Slack, CLI, Discord)
↓
Router (delivers to Session)
↓
Session ← owns pi loop ← owns compaction ← owns keepalive ← owns FlyConnection
↓
drive_run (extracted, but called from Session)
And the v2 target:
v2 target:
Platforms
↓
Router (delivers to Session, which is now thin)
↓
Session ← owns durable state only
↓
run_agent(profile, ctx, prompt, ctl) → RunResult
↓
drive_run(loop, opts) ← loop is injected ← platform is external
The difference is not in the functionality. Both shapes handle the same turn lifecycle. The difference is in what's inside what. In v1, the platform is inside the session, and the loop is inside the platform. In v2, the platform is outside the session, and the loop is injected into the engine.
That inversion is not achievable incrementally. You can't move the keepalive out of the session while the session still needs it to stay alive. You can't decouple the loop from the session while the session is the only thing that drives the loop. You can't thread RunAgent through root and subagent paths while the loop is welded to Session.
Each decoupling enables the next. Decouple the loop from the session → now you can test the loop. Decouple the session from the platform → now you can run the session in a different topology. Decouple the platform from the VM identity → now you can run the platform in a pool rather than a per-tenant VM. None of these is possible as a one-line change. None of them is possible while the others are still coupled. That's what "can't be reached by incremental change" means in practice.
You could make improvements. You could add tests around the edges. You could extract individual utilities. But you can't reach the target shape from the v1 shape without changing the coupling structure, and changing the coupling structure means changing everything that depends on it at once.
Rewrites justified by frustration usually fail. "This is a mess" is not a design. What makes the v1 → v2 rewrite different is that it has a specific, describable destination.
The target is a recursive RunAgent primitive — a function with this signature:
type RunAgent = Callable[
[AgentProfile, AgentContext, PromptInput, RunControl],
Awaitable[RunResult],
]Every agent — root, subagent, routine — runs through this function. Platform differences are injected into AgentContext. The loop is injected into drive_run. The approval gate, the sandbox, the model provider, the event sink — all injected. The function itself is pure over its inputs.
drive_run demonstrates this is possible. It's already extracted, already injectable, already used by both the root path and the subagent path. The session directory shows what still needs to come apart — session.ts still owns the persistent pi agent, the journal phases, the repair logic. The DROP items in the PARITY_CHECKLIST show which pieces to leave behind — keepalive, VM wake, SQLite rotation. What remains is the work of closing the gap.
That's not a migration plan. It's an argument about shape. The target shape is justified by the problems it solves: testable turns, reusable engines, platform-independent sessions. The current shape is justified by the problems it solved: VM-based deployment, per-tenant identity, filesystem-adjacent storage. The rewrite is the argument that the problems the target shape solves are more important than the problems the current shape solves.
drive_run is the first paragraph of that argument. The rest of Part I writes the rest.