You and your friend Jasper are building an AI agent. Jasper is the CTO of a small startup that sells marketing tools, and one Friday evening he decided the company needs a Slack bot that writes emails, analyzes experiments, and manages ad campaigns. You're the engineer. You've built chatbots before — you've wired up API keys and streamed responses and felt very clever about it. But you've never built something that runs in production, handles real users, and survives a server restart.
Jasper bounces into the co-working space on Monday. "OK, what have we got?"
You show him the chatbot. You send it a message, it replies. You send it another message, it replies. Jasper nods.
"Nice. Now let me show you everything that's going to go wrong."
He picks up a whiteboard marker.
"First things first," Jasper says. "Your chatbot does request-response. Message in, message out. But an agent does more than that."
He's right. You send the agent "analyze my latest campaign data" and the agent can't just respond with text. It needs to call your analytics API, wait for the result, maybe call another API to cross-reference it, build a summary, and then reply. That's three or four steps, not one.
"This is called a turn," Jasper explains. "It's everything that happens between the user sending a message and the agent finishing its response."
Here's what a turn looks like:
User sends a message
|
v
+------------+
| Call the |<---------------------------+
| model (LLM)| |
+-----+------+ |
| |
v |
+------------+ +-----------+ +-----+------+
| Model says | -> | Run the | --> | Feed |
| "call this"| | tool | | results |
| tool" | | | | back to |
+------------+ +-----------+ | model |
+------------+
|
v (when model has no more tool calls)
+------------+
| Reply to |
| user |
+------------+
The agent calls the model. The model says "I need to call the analytics API." You run the tool and give the result back to the model. The model says "now I need to call the comparison API." You run that tool too. Eventually the model has enough information and produces a text reply. That's one turn.
This loop — model, tool, result, repeat — is the engine of every agent framework. In some open-source agents, this loop alone is nearly four thousand lines of code. It handles parallel tool calls, iteration limits so the model doesn't loop forever, and various edge cases like the model calling a tool that doesn't exist.
"OK," you say. "So I need a loop. I can write a loop."
"Sure you can," Jasper says. "But what happens when the server crashes in the middle of it?"
You hadn't thought about this.
"Your agent is halfway through a turn," Jasper says. "It's already called two tools. It has results from both. It's about to call the model again to generate the final response. Then the server restarts. OOM kill, deployment, hardware failure — doesn't matter why. What happens to the work in progress?"
The answer, you realize, is: it's gone. The agent was holding the tool results in memory. Memory doesn't survive a restart. The user's message is still in the chat, but the work the agent did — the API calls, the results, the intermediate state — is all lost.
"Can't you just save the conversation to a database?" you ask.
"You can," Jasper says. "And you should. But think about what that actually saves you."
He draws it out. You save every message to a database — user messages, agent responses, tool calls, tool results. On restart, you reload the conversation history. The agent can see what was discussed. But it can't see where it was in the work. It knows it was asked to analyze campaign data. It might even know it called the analytics API. But it doesn't know whether it already got the result, whether it already called the second API, or whether it was about to send the reply.
"The conversation history tells the agent WHAT was discussed," Jasper says. "But not WHERE it was in the work."
This is the difference between saving conversations and saving execution state, and it matters more than you'd think. Imagine a turn where the agent sends an email to a customer, then logs the send in your database, then notifies the sales team. If the process crashes after sending the email but before logging it, and you "recover" by replaying the conversation, the agent might send the email again. The customer gets two emails. Not great.
What you actually want is checkpointing — writing down what you've done so you can pick up where you left off.
Here's an analogy. You're following a recipe. If you write each step on a whiteboard, someone cleaning the kitchen will erase it. But if you write each step in a notebook, the notebook survives. And if you write "step 3: DONE" next to each completed step, you can pick up at step 4 tomorrow without redoing anything.
That's what a durable execution system does. Every step in a workflow writes its result to a database before the workflow advances to the next step. If the process crashes and restarts, the workflow runs again from the top — but each step checks the database first. If a result already exists, return it without re-executing. If not, run the step for real.
Normal execution:
Step 1 (call API) --> write result to DB --> Step 2 (send email) --> write result to DB --> Done
Crash after step 1:
Step 1 (call API) --> write result to DB --> 💥 crash
Recovery:
Step 1 (call API) --> result exists in DB? YES --> skip, return stored result
Step 2 (send email) --> result exists in DB? NO --> run for real --> write result to DB --> Done
The recovery is automatic. No special recovery code. The workflow function is the recovery function — it runs the same way every time, and the database tells it which steps to skip.
This is how DBOS Transact works — a Python library that provides durable workflows backed entirely by Postgres. Every step's result is stored in a table called operation_outputs, keyed by workflow ID and step number. On startup, a recovery thread finds any workflows that were still running when the process died and re-invokes them. Completed steps replay from the database. Uncompleted steps run fresh.
There's one important catch. For this to work, the workflow function has to be deterministic between steps. The steps themselves can do anything — call APIs, generate random numbers, check the weather. But the orchestration — which steps to call and in what order — must be the same every time the function runs. Because on recovery, you're literally running the same function again and expecting it to reach the same sequence of steps. If step 3 suddenly becomes step 4 because of a random branch, the stored results won't match and everything falls apart.
Jasper looks at you. "Make sense?"
You take a large sip of coffee. "Yeah. I think so. Checkpoint every step, replay on crash, deterministic orchestration."
"Good. Now here's the next problem. What happens when the agent needs to do something that takes five minutes?"
Your agent is good at answering quick questions. But then the product team asks you to add a feature: "Generate a weekly performance report for each customer." This involves pulling data from three APIs, running some calculations, building a summary, and formatting it as a PDF. It takes two to three minutes.
"You can't do that inside a turn," Jasper says. "The user sends 'generate my report' and then waits three minutes for a response? No."
He's right. The user should get an immediate acknowledgment ("Working on it!") and then a notification when the report is ready. The report generation should happen in the background.
"So I need background jobs," you say.
"You need background jobs," Jasper agrees. "But think about what that means. You need to start the job, run it separately from the conversation, and get the result back to the conversation when it's done."
Some frameworks solve the first two parts. They have a background daemon — a separate thread that does work after each turn. But the daemon is hardcoded. It does one specific thing, like updating the agent's memory or consolidating its skills. There's no general-purpose mechanism for "start this arbitrary work, run it in the background, and come back when it's done."
With durable workflows, the solution is simple. The background job is just another workflow. You start it, it runs independently, and because it's a workflow, it's automatically durable — it survives crashes, checkpoints its steps, and recovers on restart.
But that raises an interesting question. The background job finishes. It has the result. Now what? How does the result get back to the right conversation?
Your agent has fifty active conversations with fifty different customers. Background job X finishes generating a report. The report needs to reach conversation Y, on Slack thread Z. Not conversation W. Not thread A. Conversation Y, thread Z.
"This is the routing problem," Jasper says. "And most agent frameworks punt on it."
He draws a mailroom on the whiteboard.
"Think of it like an office building. Every department has a mailbox. When a letter arrives, the mailroom reads the address and puts it in the right box. The sender doesn't need to know which floor the department is on. They just need to know the department's name."
Background job finishes
|
v
+----------------+ +-----------+
| Job calls | write to table | Database |
| send(conv_Y, | ---------------> | (row: |
| "report done"| | dest=Y, |
| topic="rpt") | | topic= |
+----------------+ | "rpt", |
| msg= |
| "done") |
+-----+-----+
|
Postgres LISTEN/NOTIFY |
wakes up the listener |
v
+-----------+
| Conv Y is |
| waiting: |
| recv( |
| "rpt") |
| |
| Gets the |
| message, |
| resumes |
+-----------+
This is pub/sub — publishing events and subscribing to them. The background job publishes a message ("the report is done"). The conversation subscribes to messages on a certain topic. When a message arrives, the conversation wakes up and processes it.
In some agent frameworks, there is no internal pub/sub at all. Each platform — Slack, Telegram, email — handles its own message routing. Slack uses SQS queues. Telegram uses long-polling. There's no way for one part of the system to send a message to another part. If a background job finishes on a worker thread, it has the result but no idea which Slack thread to post it to.
With DBOS, the pub/sub is backed by the same Postgres database that stores everything else. send(conversation_id, message, topic) writes a row to a notifications table. Inside the conversation's workflow, recv(topic) checks for matching rows. If none exist yet, it waits — using Postgres's built-in LISTEN/NOTIFY mechanism so it doesn't have to keep polling. When a matching row appears, recv returns the message and the conversation continues.
This has a few nice properties:
- Any workflow can message any other workflow. A background job can notify a conversation. A conversation can notify another conversation. A cron job can notify everyone.
- The messages survive crashes. They're rows in a database. If the process dies between the job completing and the notification being received, the notification is still there when the process comes back.
- No separate message broker. No Redis. No RabbitMQ. Just Postgres.
"But what if the notification arrives and nobody's listening?" you ask. "What if conversation Y hasn't called recv yet?"
"That's fine," Jasper says. "The notification is a row in a table. It sits there until someone reads it. When conversation Y calls recv, it checks the table. If the row is there, it returns immediately. If not, it waits. Either way works."
"And what if the process crashes between the job completing and the notification being sent?"
"The send is itself a checkpointed step. On recovery, it replays. The notification gets sent exactly once."
You take a bite of a sandwich that appeared from somewhere and think about this.
"So the checkpoint system... and the pub/sub system... are both just... tables in the same database?"
Jasper nods. "Keep that thought. But first — one more problem."
"You've got active conversations," Jasper says. "Mid-turn. Background jobs running. Notifications in flight. And you need to push a new version of the code."
This is the deployment problem. In a traditional web server, you can do a rolling restart — stop one instance, start a new one, repeat. Requests are stateless, so it doesn't matter which instance handles them. But your agent has stateful workflows. A conversation is mid-turn. A background job is on step 3 of 5. If you kill the process, that work is in flight.
Without durable execution, you have two choices. Wait for all active work to finish (which could take minutes), or kill it and accept the loss. Some frameworks offer graceful restart — a drain period where new messages are queued and active turns are allowed to finish. But if a turn is still running when the drain timeout expires, it's gone.
With durable execution, the deployment story is much simpler. You stop the old process. You start the new one. On startup, the new process queries the database for any workflows that were still running. It finds them and resumes them. Completed steps return their stored results. Uncompleted steps run the new code.
Old process:
Conv A: mid-turn (step 2 of 4)
Conv B: idle
Job X: running (step 3 of 5)
|
v
Drain signal --> stop accepting new messages
Wait for active work... timeout
Stop process.
New process (starts on same or different machine):
Query workflow_status WHERE status = 'PENDING'
Found: Conv A (step 2), Job X (step 3)
|
v
Conv A: replay steps 1-2 from DB, run steps 3-4 fresh
Job X: replay steps 1-3 from DB, run steps 4-5 fresh
Conv B: idle (nothing to recover)
The user never knows the server changed. The workflows pick up where they left off because every step was already checkpointed. You don't need a clever deployment strategy. You need a database.
Jasper puts down the marker. You look at the whiteboard. Six sections. Six problems. Turn loops, crashes, background jobs, message routing, deployments. You feel like you should be overwhelmed, but instead you notice something.
The checkpoint table that stores step results. The notifications table that routes messages between workflows. The workflow status table that tracks what's running and what's done. The recovery thread that re-invokes pending workflows on startup. All of it is backed by the same Postgres database.
No Redis for the pub/sub. No SQS for the job queue. No separate orchestration server for the workflows. No in-memory state that disappears on restart. Just Postgres.
"Why does this work?" you ask.
"Because Postgres already solves these problems," Jasper says. "Concurrent writes? Postgres handles it. Transactions? Postgres handles it. Crash recovery? Postgres handles it. Real-time notifications? LISTEN/NOTIFY — Postgres handles it. You don't need four systems. You need one system that does four things, and Postgres has been doing all four for decades."
He pauses.
"Look, this isn't the only way to do it. Plenty of companies run Redis for pub/sub and SQS for job queues and it works great. If you're operating at massive scale, you'll probably outgrow a single Postgres database. But when you're a small team building a production agent — when it's you and me above a bubble tea shop — using the database you already have for everything means fewer systems to operate and fewer things to go wrong at 3am."
You look at the whiteboard one more time. Six problems. One database. You can build this.
"Ready to write some code?" Jasper asks.
You close your laptop, which is still open to that tab you don't want him to see.
"Yeah. Let's do it."