A proposed GitHub comment on leanprover/comparator#58 ("Comparator generates different instance names for problem and solution files"), cross-linking #59 ("Proof irrelevance is not respected in statements").
The reviewer who assessed our previous draft recommended this approach over filing a new issue.
Session: 2026-07-01 04:40–08:21 UTC
Resume: claude --resume 6b8dec2f-08d2-4996-9624-b06eaee2071d
Slack thread: #reporting/steering-experiment
Built and validated a complete pipeline for converting ephemeral human corrections to Cy into persistent, measured improvements. Scanned 9 Slack channels, found 15 steering threads, triaged all 15, ran 8-round steering conversations with Cy, wrote eval cases, ran baseline + override comparisons, and graduated 2 SOUL rules with eval-backed evidence.
A systematic scan of every Slack channel our bot can read (9 channels), looking for threads where a human steers Cy toward better behavior through back-and-forth correction. "Steering" = human gives task → reviews output → provides corrective feedback → Cy produces improved version. One-shot requests, simple Q&A, and bug reports without resolution are excluded.
Date: 2026-06-30
Branch: ezzeri/eval-suite-canon-aligned (PR #18)
Runtime: Hermes
PR #18 does two things: (1) ships a SOUL.md change (ghostwriting_voice_control rule),
(2) builds the eval harness (criteria-count scoring, 52 evals). This scorecard separates
runs WITH the SOUL override from runs WITHOUT it, so we can see the actual clean Hermes baseline.
You and your friend Jasper are building an AI agent. Jasper is the CTO of a small startup that sells marketing tools, and one Friday evening he decided the company needs a Slack bot that writes emails, analyzes experiments, and manages ad campaigns. You're the engineer. You've built chatbots before — you've wired up API keys and streamed responses and felt very clever about it. But you've never built something that runs in production, handles real users, and survives a server restart.
Jasper bounces into the co-working space on Monday. "OK, what have we got?"
You show him the chatbot. You send it a message, it replies. You send it another message, it replies. Jasper nods.
"Nice. Now let me show you everything that's going to go wrong."
Your startup has an AI agent. It lives on Slack. Customers talk to it, and sometimes — usually — it talks back. It manages their ad campaigns, writes their email copy, generates their performance reports. You built it on an open-source agent framework called Hermes, because Hermes existed when you needed it and had a nice README.
Your CTO, Jess Jessington, has been muttering about "the framework" for weeks. She keeps finding things she wants to change and discovering that "change" means "fork the core and lose upstream updates forever." One morning she walks into standup and announces:
"We're moving to Mastra. I found it Thursday night browsing my Slop AI feed on Twitter."
By Friday she'd migrated the core agent loop. By Monday she was demoing dynamic workflows where the agent writes three poems in parallel, scores them, and picks the best one. Max Maxington, your co-founder, who has been trying to add a single cron job for three weeks, puts down his coffee.
A rewrite is a theory about what caused the problems in the original system. If the theory is wrong — if you attribute the problems to the wrong things — the rewrite produces a cleaner codebase with the same failure modes, or different ones. The theory behind this rewrite is specific: v1's problems came from three accumulated misalignments between the system's type contracts and its runtime behavior, between its topology and its operational requirements, and between its single-agent assumptions and its multi-agent ambitions. The rewrite's answer was not to add features or fix bugs one at a time. It was to surface the misalignments explicitly — through a shape audit, a parity checklist, and a gap roadmap — and resolve them at the seam level before writing the implementation. This chapter draws out the three recurring design moves that appear across every part of the rewrite, and what they mean for
A rewrite is not finished when the tests pass. It is finished when the shape of the new system makes the next five features trivial to add. That is a stricter criterion. It requires the data types, the seam boundaries, and the composition points to be in the right places before the features arrive — not retrofitted in afterward, when retrofitting means touching everything that was built on top of the wrong shape. The shape audit is the tool the rewrite uses to apply that criterion. Chapter 2 used it outward — reading claude-code and codex-rs before a line of v2 production code was written, establishing whether the primitive design was correct by checking what two independent teams converged on independently. This chapter turns the same lens inward: it cross-checks the v2 runtime's own type contracts against the implementation, finds the places where the shape is already correct, and names the gaps w
Getting an agent to run on one machine is an engineering problem. Getting it to run reliably across a fleet — where any tenant might land on any node, where machines can be suspended and resumed, where the runtime process and the sandbox process are on different hosts — is an architectural problem. The rewrite had to decide which parts of v1's distribution machinery were genuine requirements and which were artifacts of a specific topology that no longer applies.
The distinction matters more than it might seem. In Chapter 1, we traced how v1's co-located VM topology accumulated complexity over time. The harness and sandbox shared a Fly VM; the VM could idle-suspend between turns; per-tenant machines needed to be explicitly woken. Each of those characteristics created requirements that forced v1 to build mechanisms — keepalive pings, VM wake calls, filesystem monkey-patches, a loopback API bypass