Getting an agent to run on one machine is an engineering problem. Getting it to run reliably across a fleet — where any tenant might land on any node, where machines can be suspended and resumed, where the runtime process and the sandbox process are on different hosts — is an architectural problem. The rewrite had to decide which parts of v1's distribution machinery were genuine requirements and which were artifacts of a specific topology that no longer applies.
The distinction matters more than it might seem. In Chapter 1, we traced how v1's co-located VM topology accumulated complexity over time. The harness and sandbox shared a Fly VM; the VM could idle-suspend between turns; per-tenant machines needed to be explicitly woken. Each of those characteristics created requirements that forced v1 to build mechanisms — keepalive pings, VM wake calls, filesystem monkey-patches, a loopback API bypass — that exist only to work around the topology's constraints. When the topology changes, those mechanisms do not become requirements. They become liabilities.
This is the payoff of the abstraction work in Chapter 9. SandboxApi defined the sandbox boundary as a pure interface — exec, writeFile, readFile, stat, readdir, exists, mkdir, rm — with no topology assumptions baked in. The runtime calls SandboxApi without knowing whether the sandbox is local or remote, in-process or across a network. That contract-first design is what makes it possible to re-implement the sandbox as a remote Fly machine without touching the runtime. Chapter 9 built the seam; this chapter shows what plugs into it.
The rewrite's parity checklist makes this concrete with an explicit DROP section. Reading it backward tells you what v1 chose and why; reading it forward tells you what the rewrite does instead. This chapter follows that reading order: what drops and why, then what the new topology looks like end to end.
v1 ran the harness and sandbox as a single co-located process on a per-tenant Fly VM. The VM could be idle-suspended by Fly between turns — useful for economics, harmful for agent reliability. A 5.5-minute approval park, where the agent waits for a human to approve a tool call, looks identical to an idle machine from Fly's perspective: nothing is happening. So v1 built a keepalive mechanism. Seven sections in v1's codebase assert the _is_busy flag and the _CY_ACTIVE_RUNS counter; the VM pongs the autostop watchdog every N seconds while a turn is in flight. It is not complicated code, but it is code that exists purely to fight the co-location topology.
The rewrite drops this mechanism entirely. The new harness runs on ECS — always-up, never auto-suspending. The requirement that a long turn not get killed mid-flight is met for free by the executor's lease and heartbeat. The keepalive mechanism was not solving a hard problem; it was solving a topology-induced problem. Change the topology, the problem disappears.
v1: Distribution Baked Into the Runtime
┌─────────────────────────────────────────────┐
│ Runtime process │
│ ┌─────────┐ ┌──────────┐ ┌─────────────┐ │
│ │keepalive│ │ wake/ │ │ loopback │ │
│ │ 7 sites│ │ autostop│ │ credential │ │
│ └─────────┘ └──────────┘ │ injection │ │
│ ┌──────────────────────┐ └─────────────┘ │
│ │ Fly topology wired │ │
│ │ into sandbox, router,│ │
│ │ adapter, SDK layer │ │
│ └──────────────────────┘ │
└─────────────────────────────────────────────┘
Moving to ECS requires surgery in 7+ places
v2: Distribution as a Separate Layer
┌──────────────────────┐ ┌──────────────────┐
│ Runtime (ECS) │ │ Fly Machine │
│ │ │ (per tenant) │
│ SessionFactory │ │ │
│ Turn loop │ │ Control server │
│ Tools + Registry │ │ /workspace │
│ Journal + Resume │ │ │
│ │ │ │
│ SandboxApi ─────────┼────┼─▶ SandboxApi impl │
│ (interface) │ │ (exec + fs) │
└──────────────────────┘ └──────────────────┘
│ ▲
└────── flycast JSON-RPC ────┘
Moving topology = swap SandboxApi impl
The other DROP items follow the same pattern. fly-force-instance-id existed to wake a specific per-tenant VM when a session resumed — because v1 mapped each tenant to one machine, and that machine might be sleeping. The rewrite drops this because there is no per-tenant VM to wake: the executor claims a slot from the pool on first code-exec. The inbound media-cache global monkey-patch — which overwrote IMAGE/AUDIO/VIDEO/DOCUMENT/SCREENSHOT_CACHE_DIR on module globals — existed because harness and sandbox shared one VM filesystem; if you wanted the agent to see an attached file, you dropped it in the shared cache directory. The rewrite re-homes this: SandboxApi.writeFile puts the file into the executor workspace over the wire, and the module-global monkey-patch disappears. The loopback api_server routine detour existed as a per-tenant-VM API bypass, because the harness and the API server were on the same machine. The Submission object now carries per-tenant credentials natively, so the bypass is unnecessary. Finally, the SQLite split-rotate, v1's durable transcript store, re-homes to Flue's append-only journal in Postgres: the requirement — durable transcript, lineage, full-text search — carries over; the SQLite file on the shared VM filesystem does not.
The pattern across all five DROP items is identical: each mechanism is an artifact of a topology where harness and sandbox shared a process, a filesystem, and an idle-suspend timer. The rewrite separates those planes. Separate the planes, and the artifacts that bridged them become liabilities. The DROP list is not a list of things that were badly built. It is a list of things that were well-built for a topology that no longer exists.
Once you separate the harness from the sandbox, you need a wire between them. The rewrite's initial answer was a minimal JSON-RPC contract defined in a shared wire module, imported by both ends so method names and shapes stay in sync. (The current source has since migrated to ConnectRPC over HTTP/2 with a single Exec RPC — filesystem operations are synthesized as shell commands over exec in control-sandbox.ts. The wire module's types remain for shared constants, but the active data plane is the generated protobuf service. The design principles below — shared contract, single authenticated entry point, streaming demux — carry over unchanged.)
The original JSON-RPC contract:
from typing import TypedDict
# runtime → CS: prove the handshake secret before any exec/sandbox_fs is accepted
# method: 'hello'
class HelloParams(TypedDict):
token: str
class HelloResult(TypedDict):
ok: bool
# runtime → CS: run a shell command, streaming stdout/stderr back as notifications keyed by id
# method: 'exec'
class ExecParams(TypedDict):
command: str
timeout_ms: int
class ExecResult(TypedDict):
exit_code: int
# method: 'exec/stdout', 'exec/stderr'
class ExecChunkParams(TypedDict):
id: str
data: str
# runtime → CS: granular filesystem operations (stat/readdir/read/write/exists/mkdir/rm)
# method: 'sandbox_fs'
class SandboxFsParams(TypedDict):
op: str
path: str
class SandboxFsResult(TypedDict):
data: str | None
HELLO = 'hello'
EXEC = 'exec'
EXEC_STDOUT = 'exec/stdout'
EXEC_STDERR = 'exec/stderr'
SANDBOX_FS = 'sandbox_fs'
SANDBOX_ERR = {
'auth_rejected': -32001,
'not_authed': -32002,
'bad_request': -32003,
}Three primitives: authenticate, exec, and filesystem ops. That is the entire surface. The runtime dials in over flycast — Fly's private networking layer — and the control server listens. The wire carries only runtime-to-host requests and host-to-runtime notifications for streaming output. There is no ride-back channel: when agent code needs to make a proxied REST call, it hits the runtime's public endpoint with the machine token, not back up this link. The asymmetry is intentional.
runtime (ECS, always-up)
│
flycast (private Fly network)
│
control server (per-tenant Fly machine)
[hello → authenticate] [exec → stream stdout/stderr] [sandbox_fs → fs ops]
│
/workspace (tenant filesystem + shell)
This architecture resembles what Fly.io describes as per-user development environments: each tenant gets their own Fly Machine, starting from a shared base image, persisting state on a mounted volume, and reachable over private networking. The key property is that machines are not general-purpose servers — they are sandboxes with a specific interface. The runtime's public endpoint plus the flycast wire to the control server together form something closer to what practitioners call an agentic gateway: a layer that understands agent sessions, carries authentication, and is the single point through which sandbox access is controlled. An API gateway does auth and rate limiting. An agentic gateway knows about sessions and proxies tool calls through an approval layer. The wire contract is agent-cy's agentic gateway for the runtime-to-sandbox link — it knows about sessions (the hello token is session-scoped), carries authentication (the token gates all subsequent calls), and is the only path through which the sandbox can be reached from the runtime.
One property of this design deserves explicit attention: both ends import the same wire module. This is not a documentation convention — it is a drift-reduction mechanism. Sharing the module means method-name constants and payload shapes are co-defined in one place; a rename in the module propagates to both ends on the next import. Python's TypedDict + string constants do not make a wrong method name a type error the way a discriminated-union constructor would — but they ensure that the canonical name lives in exactly one file, so drift requires actively ignoring the shared constant rather than accidentally defining a second one.
The control server is a thin process on the Fly machine. Its job is minimal and its surface should stay that way: accept a hello with a token, authenticate the handshake, then accept exec and sandbox_fs requests from authenticated connections. The hello token gates everything that follows — an unauthenticated connection cannot exec. This is the first line of defense for tenant isolation: a machine can only be used by the runtime that knows its token.
Streaming output for exec comes back as exec/stdout and exec/stderr notifications, each keyed by a request id. This matters when execs run concurrently: if the agent is running a Python script and the harness is also checking whether a file exists, those two exec calls interleave on the same connection. The request id is how the runtime demultiplexes which chunks belong to which call. The control server decodes child process bytes with an incremental UTF-8 decoder so a chunk never splits a multi-byte sequence — a quiet correctness property that matters when agents write output in non-ASCII character sets or when Python's subprocess emits UTF-8 across buffer boundaries.
Control Server Protocol
Runtime Control Server (Fly)
────── ────────────────────
│ │
│──── hello(token) ────────────────▶│
│ │── verify token
│◀─── hello_ack ───────────────────│
│ │
│──── exec(cmd, cwd, timeout) ────▶│
│ │── spawn child
│◀─── exec/stdout(rid, chunk) ─────│ process
│◀─── exec/stdout(rid, chunk) ─────│
│◀─── exec/stderr(rid, chunk) ─────│
│◀─── exec/done(rid, exit_code) ───│
│ │
│──── sandbox_fs(read, path) ─────▶│
│◀─── fs_result(content) ──────────│
│ │
The control server does not orchestrate, schedule, or interpret. It executes shell commands and proxies filesystem operations. All logic — which command to run, how to interpret the result, whether to retry — lives in the runtime. This is the principle of minimal surface at the boundary: the thing that runs on potentially many machines and that you cannot easily update between deploys should be as dumb as possible. Anthropic's work on scaling managed agents describes this decomposition explicitly: a persistent harness manages conversation state and the turn loop, while the sandbox provides the execution environment. The decomposition means you can restart the harness without losing the sandbox's state, and update the harness's logic without touching the sandbox's filesystem. The control server is what makes that decomposition work at the network boundary. It is the machine's only inbound surface, and it knows nothing about agent sessions, turn state, or conversation context. It knows about shell commands and file paths. That is all it needs to know.
FlySandboxEnv is the runtime-side adapter that wraps a ControlServerClient and presents it as a SandboxApi. Every filesystem method becomes one or more exec calls over the wire:
import base64
import shlex
from pathlib import PurePosixPath
# read → cat (binary-safe: exec streams raw bytes, no base64 required)
async def read_file(self, path: str) -> str:
return (await self.read_file_buffer(path)).decode('utf-8')
# write → base64-encode content, decode it on the other end (one exec, no fs RPC needed)
async def write_file(self, path: str, content: str | bytes) -> None:
if isinstance(content, str):
content = content.encode('utf-8')
b64 = base64.b64encode(content).decode('ascii')
r = await self.exec(
f"mkdir -p -- {shlex.quote(str(PurePosixPath(path).parent))} && printf %s {shlex.quote(b64)} | base64 -d > {shlex.quote(path)}"
)
if r.exit_code != 0:
raise RuntimeError(f"write_file {path}: {r.stderr.strip() or f'exit {r.exit_code}'}")
# stat → "stat -L -c '%F|%s|%Y' -- '<path>'" then parse the three fields
# readdir → "ls -1A -- '<path>'" then split on newlines
# exists → "test -e '<path>'" then check exit codeThe design makes no attempt to be efficient for bulk operations. Each filesystem call is a round trip over flycast: stat, readdir, exists, mkdir, and rm all go out as exec calls and wait for a response. For most agent workloads — reading a config file, writing a script, checking whether a directory exists — this is fast enough. For bulk operations like reading a large directory tree, the latency compounds.
The reason to accept that latency is that the control server stays minimal. Adding a native readdir RPC to the wire would make the server faster for that operation but would grow the server's surface, add code to the machine-resident process, and create a versioning problem: old servers would not support the new RPC. The current design keeps the server-side surface at three primitives — hello, exec, sandbox_fs — all implemented by delegating to shell. Shell is already on the machine; the control server does not need to know about filesystems. You are trading per-call latency for deployability and simplicity at the boundary. That is a deliberate trade-off, not an oversight.
The exec method on FlySandboxEnv is also the live-output path. When the agent runs a Python script and the runtime wants to stream its stdout to the user in real time, FlySandboxEnv calls client.exec(...) with chunk callbacks rather than buffering everything to a ShellResult. The streaming call lives inside the FlySandboxEnv adapter — the runtime itself never reaches past SandboxApi to the ControlServerClient directly. FlySandboxEnv.exec with buffering is the convenience wrapper for commands where you need the full output before proceeding; the same adapter's streaming path is the primitive for live output. The boundary between buffered and streaming is exactly the boundary between "know the full output before proceeding" and "show the user output as it arrives." Both paths go through the SandboxApi seam, preserving the claim that the runtime doesn't know it's distributed.
Fleet management in the rewrite is handled by just commands that wrap Fly API calls. The operations are simple by design:
just setup-fly-app --rebuild # rebuild + push the agent image to the registry
just redeploy-fleet # swap image on every reachable machine
just redeploy-fleet --machine <id> # target one specific machineredeploy-fleet works by fetching each machine's current config, swapping only the image field, and posting it back. It is an image swap, not a restart. The machine's env, secrets, and mounts are preserved across the swap. This is the fleet management surface in the rewrite: runtime logic ships in the image; per-tenant workspace state persists across redeploys on the mounted volume. The two are explicitly separated so that deploying a new version of the agent logic does not wipe the tenant's working files.
This separation between image and workspace is the fleet-level analog of the wire's separation between runtime and sandbox. At the machine level, the image is what you deploy and the workspace is what persists. At the network level, the ECS runtime is what you update and the Fly machine's filesystem is what persists. Both separations express the same principle: logic and state should have independent lifecycles. You want to be able to push a bug fix to the agent image without taking down the tenant's workspace. You want to be able to restart the ECS runtime without losing the in-progress exec on the Fly machine.
There are two separate deployment pipelines for two separate layers. The ECS runtime deployment happens through your standard container deployment process: build a new image, push it to the registry, update the ECS task definition, roll the new tasks in. The Fly machine image swap happens through redeploy-fleet: the same image, applied to all reachable machines via the Fly API. The runtime's version and the machine's version can advance independently. A new runtime can talk to old machines (as long as the wire protocol is backward-compatible), and old runtimes can talk to new machines (ditto). The wire contract is the versioning boundary, which is why it lives in a shared module and why its method names are strings that both ends agree on at import time.
One question the architecture leaves partially unresolved: when a session resumes after a runtime restart, how does it re-attach to the same Fly machine? The control server holds no session state — it is stateless except for the running exec processes. If the runtime restarts mid-turn, the in-flight exec is lost. The workspace on the Fly machine is intact; the files are still there. But the runtime's connection to the control server, and the streaming output from the exec, are gone.
The current answer relies on the gateway side-channel. The executor's lease and heartbeat hold the approval park — the window where the agent waits for human approval. On harness restart, the runtime re-attaches to the outstanding execute_code RPC and resumes from the live card. This handles the common failure mode: a planned ECS deployment rolls the harness, the new instance re-attaches to the executor's lease, and the turn continues. What it does not handle is a mid-exec restart — where a Python script is actively running on the Fly machine, streaming output through the control server, and the ECS task restarts. The script may have already run to completion; the filesystem has the result; but the stdout stream that was being forwarded to the user is gone and cannot be reconstructed.
Full durable-suspend — where the executor's connection state itself is checkpointed and can be restored on a new runtime instance — is deferred. The gap is accepted and documented in the parity checklist. The cost of accepting it is that some mid-exec restarts will result in a lost turn. The cost of solving it is significant: you need to checkpoint the exec stream, store it durably, and restore the connection handshake on restart. That is a nontrivial distributed systems problem — essentially a write-ahead log for a streaming RPC connection. The rewrite makes a pragmatic call: the common case is handled; the edge case is not, and the checklist says so explicitly. Documentation of the gap is not an apology. It is a design decision written in a place where the next engineer who encounters a lost-turn bug knows where to look and what it would take to fix it.
Distribution is not deployment. Deployment ships a new version of the runtime process. Distribution is the architectural layer that decides how tenants map to machines, how the runtime talks to those machines, and what happens when either side fails.
The rewrite's answer is a clean separation: the runtime is always-up on ECS, the sandbox is a per-tenant Fly machine, and the wire between them is a minimal JSON-RPC contract with three primitives. The DROP list is the clearest statement of the topology shift — every item on it is a mechanism that v1 needed because harness and sandbox were co-located. Once you separate them, those mechanisms become liabilities. What carries over is not the mechanisms but the requirements underneath them: durable turns, tenant isolation, attachment semantics. Those requirements are real. The artifacts that previously met them are gone.
The remaining gap — session-to-machine affinity after a runtime restart — is the one place where the clean separation creates new complexity rather than eliminating it. A co-located architecture never had this problem: if the process died, everything died together, and restart semantics were simple. The distributed architecture introduces a new failure mode where the runtime can restart while the sandbox stays up, and re-attaching to the sandbox's in-flight exec is not yet solved. The rewrite documents the gap rather than hiding it. That is how you build a system that the next engineer can reason about: not by eliminating all complexity, but by knowing where the complexity lives and writing it down.