Building an Agent Runtime from First Principles
You've built a chatbot. Maybe you've wired up tool calling — a function the model can invoke, a schema it can fill. You've seen the demos: an agent that books flights, writes code, searches the web. The loop is simple: prompt, tool call, result, repeat.
Now make it survive a crash mid-turn.
Now make it run one instance per customer, each in its own VM, each with its own credentials that the agent never sees. Now make it orchestrate multi-step workflows where the agent authors the workflow script itself — and you have to run that script safely. Now make all of this resumable: kill the process at any point, restart it, and have it pick up exactly where it left off without re-executing side effects.
This is the gap between a tool-calling chatbot and an agent runtime. This book is about crossing it.
You're a software engineer who has built web services and deployed to the cloud. You've done OAuth, you've used Postgres, you've written Python. You might have built a Slack bot or integrated an LLM into a product. But you haven't designed a system that:
- Provisions isolated VMs per customer and tears them down when they're done
- Makes agent turns durable — surviving crashes without replaying side effects
- Runs agent-authored code in a sandboxed subprocess with DoS budgets and determinism guarantees
- Separates the runtime from its distribution so neither knows about the other
You want to understand the architectural patterns — not just what the code does, but why it's structured this way. You'd lead a study group on this codebase at the Recurse Center.
-
When to rewrite. Not because something is broken — because the shape you're building toward can't be reached by incremental change. How to gather evidence (shape audits of other runtimes) before committing to a design.
-
How to make agent turns durable. The 4-phase turn journal, the submission state classifier, and why resume-and-repair is harder than checkpointing — you must classify what went wrong and choose the correct repair path. Tools with external side effects must be idempotent, because the journal records tool intent, not tool completion.
-
Why tools should be factories, not functions. A tool's capabilities should be scoped to the session that created it. A subagent gets narrower tools not because someone filtered the list, but because the factory produced different tools from a narrower context.
-
How to run untrusted agent-authored code safely. Sandboxed subprocesses, DoS budgets, determinism enforcement, error info-flow control. Every constraint traces to a specific failure mode the runtime fears.
-
Why distribution is a separate architectural layer. The runtime should be unaware that it's distributed. v1's mistake was baking Fly VM topology into the runtime; v2 separates them so the runtime talks to contracts, not infrastructure.
Most writing about agent systems focuses on the prompt loop — the LLM call, the tool schema, the response parsing. That's the easy part. This book focuses on everything below: the durability layer that makes turns survivable, the trust boundaries that make tools safe, the isolation model that makes multi-tenancy possible.
The book follows a real rewrite: a 132-file runtime (v1), forked from an agent engine, archived and replaced with a ground-up v2. Every chapter examines one dimension where v1 and v2 made different choices, reads the code closely enough to see why, and extracts a reusable design move. The architectural decisions are real, the constraints are real, and the trade-offs are honest — including what v2 hasn't built yet. Code examples are presented in Python.
Three design moves recur throughout:
- The shape audit — gather external evidence about what's settled before committing.
- The single seam — reduce construction to one point so the whole system fits in your head.
- The durable envelope — wrap the unit of work in a journal so recovery is a classifier, not a rebuild.
If Designing Data-Intensive Applications taught you to think about data systems, this book teaches you to think about agent runtimes — with the same rigor, the same honesty about trade-offs, and the same insistence on understanding why this way.