A methodology for generating correct code through SPEC-driven, quality-gated, self-healing LLM pipelines.
TRUST5 is a method for reliably generating correct code from natural language requests. It is not a tool, library, or framework — it is an approach that any code generation system can adopt. The method combines five core disciplines:
- SPEC-driven generation — every implementation starts from a written specification, not a one-shot prompt.
- Bounded validate/repair loops — code is validated automatically, and failures trigger automated repair, not manual intervention.
- Quality gates — generated code passes through five weighted quality pillars before acceptance.
- Never-give-up execution — when repair exhausts, reimplement; when reimplementation exhausts, only then terminate.
- Oracle Problem mitigation — vacuous tests are detected and rejected.
The name TRUST5 encodes the five quality pillars that code must satisfy before the method accepts it as correct.
Before any code is written, the method requires a written specification. This is not optional — code generation without a SPEC is guesswork.
The SPEC contains:
- Requirement description in natural language
- Module decomposition with explicit file ownership and interface contracts
- EARS-tagged acceptance criteria (see §1.1)
- Setup commands to prepare the environment
- Test, lint, and coverage commands with quality thresholds
The planner agent that produces the SPEC has read-only access — it cannot write code. Its only job is to produce a verifiable specification.
Requirements are tagged using the EARS framework:
| Tag | Meaning | When It Applies |
|---|---|---|
| UBIQ | Ubiquitous | Always true, no trigger condition |
| EVENT | Event-driven | Triggered by a specific event |
| STATE | State-based | True while the system is in a state |
| UNWNT | Unwanted | Response to unwanted/failure conditions |
| OPTNL | Optional | May or may not be implemented |
| COMPLX | Complex | Combines multiple conditions |
Every acceptance criterion is tagged with one of these. This makes requirements testable and prevents vague "the system should be fast" style criteria.
The method supports three development modes:
1. Write tests first (RED phase)
2. Implement source code (GREEN phase)
3. Validate — all tests must pass
4. If tests fail, repair the source code (never the tests)
5. Require test-first verification (all tests written before any source)
6. Enforce minimum coverage per commit
7. No coverage exemptions allowed
The test writer agent produces test files. The implementer agent then writes source code to satisfy those tests. The implementer never sees the SPEC — only the tests.
1. Before modifying existing code, write characterization tests
that capture current behavior
2. PRESERVE step: run characterization tests to verify understanding
3. IMPROVE step: make changes while characterization tests still pass
4. Detect behavior snapshot regressions
DDD is for working with existing codebases. The PRESERVE step prevents the generator from accidentally breaking existing behavior.
1. New code: enforces one coverage threshold
2. Modified legacy code: enforces a separate (typically lower) threshold
3. Different standards for new vs. modified code
When the SPEC decomposes the work into multiple modules, they execute in parallel. Three rules ensure correctness:
Rule 1 — Strict File Ownership: Every source file belongs to exactly one module. No two modules may write to the same file. Conflicts are detected before execution starts — the pipeline fails with a clear message rather than allowing concurrent writes.
Rule 2 — Interface Contracts: Each module declares its public API signatures in the SPEC. Parallel modules agree on these contracts instead of each imagining its own. This is how module A can call module B's functions while both are being implemented simultaneously.
Rule 3 — Dependency Ordering: If module A depends on module B, B is fully validated before A begins. The dependency graph is checked for cycles before execution — a cycle would deadlock the parallel pipeline.
After all modules pass individually, an integration step runs everything together to catch cross-module issues.
After code is generated (whether test-first or spec-first), it enters the validate/repair loop:
┌──────────────┐
│ Validate │
│ ┌──────────┐│
│ │ Syntax ││
│ │ Lint ││
│ │ Test ││
│ └──────────┘│
└──────┬───────┘
│
┌─────────┴─────────┐
▼ ▼
(pass) (fail)
│ │
│ ┌──────▼───────┐
│ │ Repair │
│ │ (LLM fixes │
│ │ code) │
│ └──────┬───────┘
│ │ jump_to("validate")
│ │
│ (bounded by limits)
│
▼
Next stage
Validation runs in order. Failure at any stage triggers repair:
- Syntax check — Language-specific compiler/parser (e.g.,
compileallfor Python,go vetfor Go,tsc --noEmitfor TypeScript). - Lint check — Read-only linter (e.g.,
ruff,clippy,eslint). Test-file lint errors are filtered out — the method considers test code held to a different standard. - Test execution — Full test suite.
The repair agent has absolute prohibitions:
- Never modify tests — Tests define the specification. Fix the source.
- Never install packages — All dependencies must be available.
- Never manually test — Only automated validation counts.
- Never repeat the same fix — Escalate after two identical failures.
When the current fix strategy fails, the repair escalates:
Level 1: Fix the specific line causing the error
Level 2: Rewrite the entire function with a different algorithm
Level 3: Check project configuration (imports, paths, manifests)
Level 4: Restructure the module (split/merge files, circular imports)
Level 5: Rewrite the entire file from scratch based solely on test expectations
The repairer re-reads the full test file at each escalation level — its previous understanding may be wrong.
| Layer | Max | Behavior on Exhaustion |
|---|---|---|
| Repair attempts | 5 | Reimplement the module from scratch |
| Reimplementations | 3 | Terminate the pipeline |
| Total DAG jumps | 50 | Absolute ceiling (all jump types) |
| Quality gate retries | 3 | Accept partial result + warning |
These bounds prevent infinite loops while giving the system enough runway to succeed.
After all tests pass, the code must pass the TRUST 5 quality gates before it is accepted.
T — Tested (30%) — Are we sure the code works?
R — Readable (15%) — Can a human read this code?
U — Understandable (15%) — Is it documented and reasonable?
S — Secured (25%) — Does it have known vulnerabilities?
T — Trackable (15%) — Is the project structured and traceable?
Each pillar is scored 0.0 to 1.0.
Tested (30%) asks:
- Do all tests pass?
- Are there zero type errors in test output?
- Does coverage meet the threshold?
- Do test functions contain meaningful assertions (not vacuous tests)?
Readable (15%) asks:
- Do lint commands pass?
- Are format and style consistent?
Understandable (15%) asks:
- Are warnings below the threshold?
- Are files reasonably sized (max 500 lines)?
- Do source files have module-level documentation?
Secured (25%) asks:
- Does the security scanner find any HIGH/CRITICAL issues?
- Are MEDIUM issues surfaced as warnings?
- If the scanner cannot run (tool not installed), is this surfaced?
Trackable (15%) asks:
- Do filenames contain no spaces?
- Do test files exist alongside source files?
- Does the last commit use Conventional Commits format?
The quality gate passes when ALL of these are true:
- Weighted score ≥
pass_score_threshold(default 0.85) - Total errors from all pillars = 0
- Completeness passes (required project files exist, no garbled files)
The weighted score calculation:
score = Σ(pillar_score × pillar_weight) / Σ(active_weights)
If a pillar's tool cannot run (e.g., bandit not installed), the pillar
is marked unverified and excluded from the weighted average rather
than scoring 0. This prevents environment gaps from failing good code.
| Score | Status | Meaning |
|---|---|---|
| ≥ 0.85 | PASS | Acceptable |
| 0.50-0.84 | WARNING | Acceptable with concerns |
| < 0.50 | CRITICAL | Blocking — must improve |
The method detects when the repair loop is making no progress:
If between consecutive quality reports:
- score is identical AND
- total_errors is identical AND
- total_warnings is identical
Then the pipeline is stagnant.
Stagnation triggers escalation to a fundamentally different approach (e.g., full reimplementation instead of more repair attempts).
The Oracle Problem: tests that pass without actually testing anything (vacuous tests). An LLM generating tests may produce assertions that look real but never fail — the generator "knows" the answer and writes tests to match.
The method mitigates this in two ways:
Every test function is checked for meaningful assertions:
- Python: AST-walk each test function; check for
assertstatements,assertXxxmethod calls, andpytest.raisescontext managers. - Other languages: Regex-based per-file assertion counts using
language-specific assertion patterns (
expect(),assert.Equal(),assert!(), etc.).
Thresholds:
- Density ≥ 0.8: Acceptable
- Density ≥ 0.5: Warning (some tests may be vacuous)
- Density < 0.5: Error (many tests are likely vacuous)
When mutation testing is enabled, the method injects faults into the source code and checks whether the test suite detects them.
- Kill rate ≥ 0.95: Acceptable
- Kill rate ≥ 0.80: Warning
- Kill rate < 0.80: Error — tests failed to detect injected faults
The method distinguishes between three failure modes and handles each differently:
| Failure Mode | Response |
|---|---|
| Test failures | Repair up to 5 attempts |
| Repair exhausted | Reimplement the module from scratch (up to 3) |
| All attempts fail | Pipeline terminates with clear error report |
The combination of multiple repair attempts and multiple reimplementation attempts means the system has a total of up to 15 attempts per module before giving up.
failed_continue is a distinct concept: the pipeline continues to
downstream stages even when a stage fails, so partial results are
delivered. Only terminal halts everything.
The method is designed to survive process termination:
- All state is persisted to SQLite after every action
- Event sourcing enables replay of any point in the pipeline
- The
resumeentry point finds the last terminated pipeline and restarts from the failure point - All accumulated context (repair attempts, test files, module ownership) is preserved across restarts
This means a 2-hour pipeline that crashes at 1h59m loses no progress.
- Write a SPEC before generating code
- Tag acceptance criteria with EARS
- Validate syntax → lint → test in order
- Fix source code, never tests
- Escalate repair strategy when stuck
- Check for vacuous tests via assertion density
- Persist all state for crash recovery
- Enforce file ownership in parallel execution
- Generate code without a SPEC
- Accept code that fails any TRUST 5 pillar at CRITICAL level
- Modify tests during repair
- Install packages during repair
- Repeat the same fix twice — escalate
- Allow infinite loops — enforce bounds at every level
- Accept tests without meaningful assertions
The TRUST5 method is appropriate when:
- Correctness matters more than speed — the overhead of planning, validation, repair, and quality gates means this is not the fastest way to generate code, but it is more reliable.
- The cost of failure is high — production code, financial systems, security-sensitive components.
- You need auditable correctness — the SPEC, test results, and quality reports provide a paper trail.
- You are generating code for an existing codebase — DDD mode with characterization tests prevents regressions.
It is less appropriate for:
- One-shot throwaway scripts
- Exploration and prototyping
- When you want the fastest possible codegen without validation overhead