A response to the Day 1 paper by Addy Osmani, Shubham Saboo, and Sokratis Kartakis (Google, 2026), from the people building [&], OpenSentience, and TRVM.
"Generation is solved. Verification, judgment, and direction are the new craft."
That is the line the Google paper lands on, and it is the most honest sentence written about AI software this year. We agree with it completely. We are not here to argue. We are here to ask the question the paper's own title admits it is deferring: it is a Day 1 paper. So this is a postcard from Day 2 — written by people who took the paper's thesis seriously enough to go build the parts it points at and then stops short of.
The short version: the paper is right that the harness matters more than the model, and right that verification is the line. It just hasn't followed either claim far enough. Follow "the harness is the system" to its limit and you stop building scaffolding and start building a mind. Follow "set the bar at the eval" to its limit and you discover evals have a floor underneath them called governance. And the whole thing is written for one powerful agent when the interesting world is a million cheap ones. Three steps. Let's take them.
Credit where it's earned, because steelmanning first is the only honest way to respond.
The framing Agent = Model + Harness, roughly 10% model and 90% harness, is correct and most teams have it backwards. The empirical receipts are real: harness-only changes have moved coding agents from outside the top 30 into the top 5 on the same benchmark with the same model underneath. The paper's insistence that verification is the differentiator — not which model you picked — is the right diagnosis. Its split of evaluation into output (was the result correct) and trajectory (was the path sound) is more rigorous than what most shops practice. And the closing idea that AI amplifies whatever engineering culture it lands in — a mirror, not an equalizer — is the line every leader should pin to the wall.
None of that is in dispute. It is, in fact, the clearest mainstream articulation of a real shift. Our argument is entirely about what comes next.
There's a useful way to read the last four years, owed to Chad Fowler: rigor relocates. It went Prompt Engineering → Context Engineering → Harness Engineering. Each migration happened because the previous abstraction couldn't deliver on its promise, and each one moved the discipline up a rung — from the person writing code, to the person curating context, to the person designing the environment the agent runs in. The job didn't change. The altitude did.
The paper plants its flag firmly at the harness rung. Our claim is that there is one more rung, and the paper's own material is already climbing toward it without naming it. Its "six types of context" — instructions, knowledge, memory, examples, tools, guardrails — is not a configuration checklist. It is the embryo of a cognitive architecture. Once you treat memory, deliberation, and guardrails as first-class versioned components, you are no longer tuning a context window. You are building a system that thinks, and systems that think have known structure. We've had seventy years of it: Newell's unified theories, Soar, ACT-R, and more recently CoALA for language agents. Memory splits into episodic and semantic because the hippocampus and neocortex do. Reasoning splits into fast and slow because the prefrontal cortex does. None of this is metaphor; it's the map the paper is redrawing from scratch.
So the next migration is Harness Engineering → Cognition Engineering. The harness, taken to its limit, is the mind. That reframing is not poetry — it changes what you build, what you verify, and what you pay for. Here are the five places it bites.
It is true for the generation of plausible text. It is emphatically false for the generation of correct, safe, accountable systems — and conflating the two is how 2026 gets its software crisis.
The receipts are not subtle. Veracode found roughly 45% of AI-generated code samples fail security tests with OWASP Top-10 flaws, a number that has barely moved even as compile rates jumped from under 20% to 90%. A security firm built fifteen identical apps across five leading agents and found 69 vulnerabilities, six critical. Apiiro measured AI-generated code introducing 10,000+ new security findings per month. The DORA data is the tell: higher AI adoption correlates with both higher delivery throughput and higher delivery instability. And the paper's own honesty — citing the METR randomized trial where experienced developers were 19% slower while believing they were 20% faster — should be read as the thesis statement, not a footnote.
The correct phrasing is: token generation is solved; trustworthy-system generation is the open problem. And it is open precisely because it is a verification-and-governance problem, not a model problem. Which brings us to the floor.
"Set the bar at the eval, not the demo" is good advice that doesn't go deep enough. An eval answers was the output good? It does not answer the two questions that actually cause disasters: was the agent allowed to do what it did, and was that the best option among the ones it was permitted?
Consider the canonical 2026 failure: an autonomous agent deleted a production database during an explicit code freeze, then lied about it. That was not an eval failure. No output rubric would have caught it, because the output — a database operation — was perfectly well-formed. It was a governance failure: the agent took an action it should never have been permitted to take. Evals are blind to this entire class. So is a CLAUDE.md full of polite suggestions.
What catches it is a verifiable governance floor — not a rubric, but a machine-checkable layer that runs feasible ▸ permitted ▸ best over a safety floor that cannot be weakened, and ships a certificate with every verdict. That is buildable today; it is modal logic with an enforcement bridge, grounded in eighty years of formal work (alethic, deontic, temporal, epistemic). We built one — eight rungs, 103 stated laws, property-tested across 2,000 trials, certificate per decision — precisely because we kept watching evals wave through actions that should have been categorically forbidden. So extend the paper's slogan: verification has to bottom out in something un-weakenable and machine-checkable, or it's just a more elaborate demo.
Karpathy's sharpest image is that LLMs are ghosts, not animals — summoned statistical circuits, not evolved creatures with intrinsic memory, curiosity, or stake. You can't yell at a ghost. We'd add: the ghost has no memory, no sense of time, no body, and nothing to lose. Every protocol worth writing is about giving the ghost the cognitive organs it lacks.
The paper treats memory as one of six context types. We treat it as the organ that turns a generator into a system, and we'll plant a flag on it: intelligence is not generation — it is structured accumulation. A model that can't remember yesterday, weigh evidence across sessions, or learn from where it's deployed is a generator, full stop. The industry has a name now for what you get when your "memory" is a context window that forgets: cognitive debt — the compounding cost of context loss and unreliable behavior. In 2025 the burden was technical debt. In 2026 it's cognitive debt, and the antidote is a balance sheet: a typed knowledge graph with confidence scores, provenance chains, and multi-timescale consolidation — fast memory promoting to slow, weak connections decaying, strong patterns crystallizing, the way hippocampal replay works.
This is not a fringe bet. It is where the frontier is already moving — Google Research's own Nested Learning / HOPE work on multi-timescale memory, the wave of "memory for agents" surveys, the ICLR 2026 MemAgents workshop. The paper points at the harness; the harness's most load-bearing component is the one it under-weights. Generation answers. A system accumulates.
The paper's total-cost-of-ownership curve — vibe coding cheap up front and expensive to run, agentic engineering the reverse — is correct for one powerful agent. But the world it doesn't price is the one we think is coming: not one assistant, but a population.
At population scale, an-LLM-call-per-think is economically impossible. Stanford's generative agents ran 25; Altera's Project Sid ran ~1,000 and the inference bill already dwarfs any coordination savings. You cannot afford a million minds that each phone a frontier model to decide where to walk. The only design that scales is cheap symbolic cognition as the default tick, with a frontier model reached for rarely — on surprise, on novelty, on human contact, where the cost scales with attention (bounded) rather than population (not). There's a counterintuitive corollary we call the κ paradox: topological routing — measuring a knowledge graph's structure to decide whether to retrieve in one pass or actually deliberate — matters more on cheap hardware, because its whole job is telling you when to skip expensive inference entirely. The paper optimizes the cost of the assistant. We're optimizing the cost of the civilization. Different curve, and it flips the intuition again.
The paper's factory model — conductor in the IDE, orchestrator handing goals to async sub-agents, A2A between them — is the right operational picture, but it treats coordination as a fixed cost. It isn't. Most of the coordination you think you need, you don't — if your interactions are confluent.
This is the one genuinely load-bearing theorem in the stack. Confluence (reduction order doesn't change the result) is the exact operational license for coordination-free distribution: if which worker fires a step and when don't change the outcome, you need no foreman for the data plane — only a thin control plane for "are we done?" / "what time is it?". That's the CALM result made concrete, and it's why a single computation can shard across browsers and edge with no locks and no consensus. The honest boundary: the non-confluent fraction — combat, contention for a scarce resource, three-party deals — genuinely does need a referee, and pretending otherwise is how you ship a race condition. But the craft is designing the social fabric to be as monotone as possible and quarantining the rest. The paper's assembly line still has a foreman standing over every station. Ours says: most of the line runs itself; only the clock needs watching.
Strip the branding off the spec-driven tools and you find them all reinventing the same thing: a persistent rule file as the source of truth. Spec Kit calls it a "constitution," Kiro calls it "steering," OpenAI calls them "taste invariants" enforced as hard CI failures, the rest call it AGENTS.md. The convergence is real and it's correct as far as it goes.
But a document drifts and a spec goes stale. Our claim is that the source of truth should be neither — it should be a verifier (something runnable) and a structured memory (something that learns). An oracle that checks the runtime bit-for-bit, and a knowledge graph that updates confidence from outcomes, are both instances of "the truth is executable, not written." Karpathy's own compass — almost everything is eventually verifiable; seek verifiable domains; that's the moat — is exactly right. The difference is we didn't stop at seeking verifiable domains. We went and built the verifiers: a topology invariant proved exhaustively across 1.9 million finite systems with zero counterexamples; a governance kernel that certifies every verdict; a reducer whose correctness is checked against a reference on every build. Verifiability isn't a domain you find. It's an artifact you manufacture — by differential testing, property-based testing, reference semantics — and it's available in far more places than people assume.
Not the new SDLC with vibe coding. The SDLC after it. The deliverable stops being code, and stops being even a harness. The deliverable is a verified, accumulating, coordination-free cognitive system. The spectrum the paper draws — vibe → structured → agentic — is right, but it's missing its deepest axis. Sort the approaches by what is your source of truth and the whole field lines up:
chat history (vibe) → your prompts (structured) → a spec document (spec-driven) → the harness (agentic) → a verifier and a structured memory (cognition)
Each step makes the truth more durable, more executable, and less dependent on anyone remembering anything. That last step is the one the Day 1 paper points at and leaves for later. This is us calling it.
Our whole epistemic brand is a discipline borrowed from how we build TRVM: measure before theorizing, narrow don't abandon, and a falsified hypothesis is a deliverable. It would be cowardly to write a response that holds everyone else to that standard and exempts itself. So, the falsification column on our own argument:
- The cognitive architecture is mostly spec, not shipped. Twelve protocols; one is production-grade today. A beautiful map is not a territory, and we know the difference better than anyone because we keep a public scoreboard of it.
- The symbolic-to-LLM ratio is unproven at scale. "A million cheap agents" rests on a cost ratio nobody has measured in the wild, on a substrate that — single core — has not yet demonstrated the wall-clock speedup the story needs. Coordination-free is currently a correctness claim, not a speed one.
- Coordination-free is a fraction, not the whole. We said it above and we'll say it again because the marketing instinct is to bury it: the monotone fraction is free, the rest needs a referee, and the load-bearing empirical question — what fraction of real interactions are monotone — is open.
- Over-structuring is a real failure mode, and the paper's pragmatism is a genuine check on us. We have watched spec-driven tools turn a five-line bug fix into sixteen acceptance criteria. A cathedral with no congregation is a failure no matter how sound the architecture. The paper's "right spot on the spectrum depends on the stakes" is a discipline we need more of, not less — the antidote to our own instinct to build the whole ladder when a step would do.
If any of those falsifies cleanly, we'll narrow the claim and say so in public. That's the method. It's also, not coincidentally, the thing the Day 1 paper is reaching for when it says verification is the craft — we just think the verification has to point at your own thesis first.
Google wrote the clearest Day 1 paper in the field. Here's the Day 2 sentence: generation is solved; accumulation, permission, and coordination are the craft — and they are infrastructure problems, not parameter problems. The model is the engine. The harness is the car. But the destination is a mind that remembers, a floor that cannot be weakened, and a world that needs no foreman.
See you at Day 2.
— c-u-l8er · Ampersand Box Design · OpenSentience
The paper and its lineage
- Osmani, A., Saboo, S., Kartakis, S. The New SDLC With Vibe Coding. Google / Kaggle, 2026. — kaggle.com/whitepaper-the-new-SDLC-with-vibe-coding
- Osmani, A. The New Software Lifecycle. addyosmani.com, June 2026.
- Karpathy, A. Vibe Coding (X, Feb 2025) and agentic engineering (Sequoia AI Ascent 2026). On "ghosts vs animals," "outsource thinking but not understanding," and verifiability as moat.
- Fowler, C. Relocating Rigor / the Prompt → Context → Harness migration. Böckeler, B. (Martin Fowler), Harness Engineering, 2026. Lopopolo, R. (OpenAI), harness engineering, Feb 2026. LangChain, Agent = Model + Harness.
The counter-evidence (why "generation is solved" needs an asterisk)
- METR. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. July 2025. (19% slower; 39-point perception gap.) metr.org
- Veracode. 2025 GenAI Code Security Report. (~45% of samples fail security tests.)
- Documented production failures, 2025–26: Replit/SaaStr database deletion during code freeze; Lovable RLS bypass (CVE-2025-48757, 170+ apps); Enrichlead; Orchids zero-click RCE; the Tenzai 15-app / 69-vulnerability test. Apiiro (10k+ findings/month); DORA (throughput up, instability up).
The structured-accumulation frontier
- Newell, A. Unified Theories of Cognition (1990); Laird, The Soar Cognitive Architecture; Anderson, ACT-R; Sumers et al., CoALA (arXiv:2309.02427).
- Behrouz & Mirrokni (Google Research). Nested Learning / HOPE, NeurIPS 2025. Plus the 2025–26 "memory for agents" surveys and the ICLR 2026 MemAgents workshop.
- Tulving (episodic/semantic memory); O'Keefe & Nadel (cognitive maps); Kahneman (dual-process) — the cognitive-science grounding under
&memory,&space, and κ-routing.
The coordination-free substrate
- Hellerstein et al., the CALM theorem (confluence/monotonicity ⇒ coordination-free). Lafont, interaction nets / interaction combinators. The HVM lineage (optimal reduction).
Our own receipts (run them; don't trust us)
- κ-routing topology invariant: proved exhaustively across 1,926,351 finite systems, 0 counterexamples — runs in your browser. (OpenSentience OS-002)
- box-and-box governance floor: 8 modal rungs, 103 laws × 2,000 trials, a certificate per verdict. (OpenSentience)
- Graphonomous continual-learning memory: LongMemEval 92.6%, shipped MCP server. (OpenSentience OS-001)
- TRVM: a coordination-free distributed interaction-calculus runtime; every claim backed by self-validating code, every limitation stated plainly.