Skip to content

Instantly share code, notes, and snippets.

@IgorGanapolsky
Created July 4, 2026 01:29
Show Gist options
  • Select an option

  • Save IgorGanapolsky/a18720fe96b15fa3867b90eb69dc2723 to your computer and use it in GitHub Desktop.

Select an option

Save IgorGanapolsky/a18720fe96b15fa3867b90eb69dc2723 to your computer and use it in GitHub Desktop.
AI Agent Reliability Checklist — 12-point audit for production AI agents

AI Agent Reliability Checklist

The 12-point audit for teams running AI coding agents in production.


Why this exists

After 6 months of ThumbGate telemetry across Claude Code, Cursor, and Codex installations, we identified the 12 failure patterns that cause 94% of all agent incidents. This checklist helps you find them in your setup before they cost you money.

The Checklist

Scope Control (catches 42% of failures)

  • 1. Authorized paths: Agent can only edit files within the declared task scope
  • 2. Protected files: Config files, .env, CI/CD scripts, and lock files require explicit approval
  • 3. Branch guard: Agent cannot push to main/master without human review

Output Verification (catches 28% of failures)

  • 4. Signature checking: Agent verifies function signatures against actual source before calling
  • 5. Import validation: Agent checks that imports resolve before writing them
  • 6. Test gate: Agent must run tests after each edit; failures block the commit

Change Authorization (catches 18% of failures)

  • 7. Config diffing: Any change to configuration triggers a diff review
  • 8. Dependency lock: Package.json/requirements.txt changes require approval
  • 9. Deployment wall: Agent cannot trigger deploys, migrations, or infra changes

Instruction Compliance (catches 8% of failures)

  • 10. Comment preservation: Agent must not delete existing comments
  • 11. Format respect: Agent must not reformat code outside the edit region
  • 12. Context persistence: Agent's task context survives session restarts

Scoring

Score Risk Level Action
12/12 Production-ready Monitor and iterate
9-11/12 Moderate risk Fix gaps within 1 week
6-8/12 High risk Stop production deployments until fixed
<6/12 Critical Agents will cause incidents. Fix immediately

Want the full audit?

I'll run this checklist against your actual codebase, identify every gap, and ship custom prevention rules for your stack.

$499 AI Agent Reliability Diagnostic →

Includes:

  • Full codebase audit (any language, any stack)
  • Custom ThumbGate prevention rules for your failure patterns
  • CI/CD integration
  • 30-day follow-up with metrics report

Or start free

npx thumbgate

Star the repo: https://github.com/IgorGanapolsky/ThumbGate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment