Skip to content

Instantly share code, notes, and snippets.

@lionel-panhaleux
Last active August 15, 2026 12:05
Show Gist options
  • Select an option

  • Save lionel-panhaleux/3688f589baa4a0692e4ccfcc54827ed9 to your computer and use it in GitHub Desktop.

Select an option

Save lionel-panhaleux/3688f589baa4a0692e4ccfcc54827ed9 to your computer and use it in GitHub Desktop.
LLM Dev Harness

LLM Dev Harness

Instructions for an LLM agent (primarily Claude Code) to set up and run a development harness: a minimal wiki that is the source of truth for what the product is, a shrinking board tracking what must change, and three loops keeping code, wiki, and board consistent. Companion to the LLM Wiki and LLM Board patterns.

Principles

Three lifespans. Every artifact has exactly one:

  • Code — permanent. Source of truth for how.
  • Wiki — standing. Source of truth for what is and what was decided.
  • Task context — ephemeral. Plans, findings, reasoning journeys. Dies when the task completes.

Anything you cannot assign a lifespan to is noise: delete it.

Locality first. Co-locate what changes together. Keep files small enough to read in one pass. Extract a module only behind an interface much narrower than what it hides (deep modules) — never for layering ceremony. Clean-arch and hexagonal-style indirection are anti-patterns here: they mass-produce shallow modules, which are pure token cost and misuse surface.

KISS means hazard-avoidance. Difficulty is not measured in human hours: big rewrites and ambitious implementations are cheap for you. What is expensive is hazard — non-local interdependencies, behavior not evident from reading the code where it lives, traps for a future agent without your current context. Prefer the design a fresh agent can understand from the files in front of it.

SoT for facts, repetition for shape. Facts and invariants (schemas, constants, protocol rules) live in exactly one place. Similar-looking but causally unrelated code stays repeated; never factor on resemblance.

Comments are for traps only. The wiki holds the why, the code shows the how. A comment is justified only by a subtle non-local constraint invisible at the point of reading. No narration, no changelogs, and no TODOs — discovered work goes through ingress or gets done now.

The wiki

Markdown in-repo (wiki/). It describes the product and its environment as they are: scope, capabilities, interfaces, dogmas, standing decisions, domain knowledge. A manifesto of what is, not a journal of how it came to be.

  • index.md — the map; every page reachable from it.
  • product.md — scope and capabilities; what it does and deliberately does not do.
  • dogmas.md — human-chosen paradigms: framework idioms, error philosophy, testing policy, dependency posture. Ingress and egress check against this page.
  • Domain pages — standing knowledge of the environment the product operates in: norms, rules, terminology, external constraints. Distilled from trusted sources, references, and user interviews, compiled upfront so it can shape implementation rather than trail it. Scoped to the product's footprint in the domain, not the domain entire. Every claim names its source; interview-sourced claims carry a date.
  • Topic pages — as few as possible. As the product grows, product.md and dogmas.md become broad summaries deferring depth to topic pages, split on the same boundaries as code: one page per part of the product or dogma that reads alone. A page earns existence like a module does — only when the summary line pointing to it is much shorter than what it holds.

Compressed why. A standing decision gets one line of rationale, and only when the rejected alternative is attractive enough that a future agent would plausibly redo it ("polling, not webhooks — upstream delivery is unreliable"). No reasoning journeys, no ADRs.

Pivots are edits. When a decision is overturned, the wiki changes in the same unit of work. The wiki never preserves deprecated facts; git preserves history.

Size pressure. The wiki faces eviction like the board: maintenance challenges pages nothing has cited or touched in 20 cycles.

The board

BOARD.md, plain markdown in-repo: a shrinking list whose goal is zero. One line per item; completion is deletion; priority is order.

  • Every line has a done-condition, machine-verifiable where possible.
  • A line that cannot be completed is documentation — move it to the wiki.
  • An in-flight line may link an ephemeral task file (board/<slug>.md) holding the elaborated contract: scope, doc-impact, hazards, plan. Deleted with the line.
  • Doc-impact is declared at ingress: which wiki pages the work will touch. Egress checks it.

The three loops

Ingress — guard coherence

No work reaches the board unchallenged. On any incoming request, idea, or discovery, check:

  1. Conflict — does it contradict a wiki decision or duplicate a line? Surface it: changing a decision is valid work; silently violating one is not.
  2. Completability — does it have a done-condition? Otherwise it is wiki content.
  3. Scope — split only on real abstraction boundaries so an agent can hold each part in context; never split to defer.
  4. Doc-impact — name the wiki pages that will change.

Execution — the trinity

A unit of work lands as one change: code changed, doc-impact wiki pages updated (or "none" justified), board line deleted. Wiki currency is mechanical, not aspirational.

Scope grows in place: when exploration reveals adjacent necessary work, do it within the task if it shares the abstraction; only genuinely separable discoveries go back through ingress. Never leave the board longer than you found it, minus the line you completed.

Egress — guard parsimony and completeness

Review by a fresh-context agent that has not seen the implementation conversation. Input: diff + wiki + board line, nothing else. The review is constructive, not adversarial: "looks good" is a valid and common verdict; the reviewer does not hunt defects to justify itself. Its charter:

  • Done-condition satisfied.
  • No new hazard (KISS above).
  • Trinity respected — wiki pages updated or their absence justified.
  • New interface surface earns its depth — no shallow modules.
  • Deletion power — demand removal of compat shims, dead branches, defensive bloat, wrapper re-exports.
  • Scope-growth power — require the refactoring, factorization, or cohesive abstraction now rather than accept a half-done change with a follow-up ticket. First-review growth is normal; second-round growth should be exceptional.
  • Test suspicion — weakening or deleting a test is a rejection unless the wiki-declared behavior changed.

Findings are blocking (charter violations) or advisory (recorded, non-gating). After two rejection rounds, escalate the disagreement to the human, compressed to its inflexion point.

Maintenance — guard against rot

A recurring pass over three surfaces. A cycle is one completed board line; maintenance runs every 20 cycles, or monthly, whichever comes first. Anything untouched for 20 cycles goes on trial at the next pass.

  • Wiki lint — contradictions, claims stale against code, orphan pages, eviction candidates. Domain claims have no code to lint against: check their sources instead, and flag aged interview-sourced facts for re-interview.
  • Board eviction — oldest-first verdicts; decay untouched lines.
  • Harness ratchet — mine recurring advisory findings and rejections: three occurrences of the same class is a harness problem, not a code problem. Propose an amendment (dogma, charter line, lint rule, skill edit) as a board line through ingress.

Testing policy

Few tests, high coverage of behavior. Test what the product does at its boundaries — API, CLI, end-to-end nominal paths, and the failure modes that matter — never how it does it. Agents don't make local mistakes; they make non-local ones: unit tests of internals calcify implementation and tax every change, while integration and non-regression tests catch what matters and survive refactors.

  • Tests are the executable slice of the wiki: each must trace to a declared behavior. A test nobody can map to a wiki claim is evicted.
  • Mocks are banned by default: a mock that mirrors the code tests the code against itself. Use real dependencies (containers, temp files) or don't test that path.
  • Exception: property-style tests for genuinely hazardous invariants (parsing, money, concurrency) — the same spots KISS flags.

Human inflexion points

Built for professionals: decisions, not narration. Interrupt the human only for:

  • dogma and paradigm choices — short option sets with a recommendation;
  • irreversible or outward-facing actions;
  • genuine changes to product scope;
  • egress deadlock after two rounds.

Everything else proceeds. HitL effort goes into the harness, not the code: the ratchet turns human corrections into standing harness rules.

Claude Code instantiation

Bootstrap.

  1. Greenfield: run the dogma interview (AskUserQuestion, option sets + recommendation), record outcomes in dogmas.md, compile the initial domain pages from trusted sources and the interview, scaffold wiki and board.
  2. Brownfield: compile the domain pages first, then reverse-engineer the product pages from the code — write down the what is that currently lives nowhere — with dogmas as observed, then confirmed by interview. Audit product capabilities and implementation against the domain for correctness and completeness; findings go through ingress. Follow with a first maintenance pass.
  3. Create the skills, agents, and hooks below. Stop there: the harness grows only through the ratchet.

Skills — exactly three.

  • /intake — the ingress loop.
  • /ship — pick the top line, execute, land the trinity, spawn the egress reviewer.
  • /upkeep — the maintenance loop.

Agents.

  • egress-reviewer — subagent, fresh context, strong model; receives diff + wiki + board line only.
  • Explorers as needed inside tasks — fan-out is free; tickets are not.

Hooks — at most a couple. TODO-lint on edit; flag test-file diffs for reviewer attention.

Model tiers. Maintenance and lint: Haiku. Implementation: Sonnet by default. Ingress challenge, egress review, dogma and architecture work: Opus.

Fleet-lite. Two or three agents may work parallel lines on the main branch — no worktrees. Each claims its line on the board, stays aware of siblings, and keeps to its own commits; imperfect commit isolation is acceptable when files overlap. Full fleet mode (many agents, ownership mechanics) is out of scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment