Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save dewdad/119b9007d250e5824c57a8f24ea0e75d to your computer and use it in GitHub Desktop.

Select an option

Save dewdad/119b9007d250e5824c57a8f24ea0e75d to your computer and use it in GitHub Desktop.
Enhancement Evaluation Protocol for AI Agents — A structured adversarial method to evaluate proposed dependencies before accepting them

Enhancement Evaluation Protocol for AI Agents

A reusable instruction for AI coding agents to critically evaluate proposed library additions, dependencies, or capabilities before accepting them into a project.

When to Use

Someone proposes adding a library, tool, or capability to an existing project. Before implementing, run this protocol.


Phase 1 — Understand the Current System

Before evaluating the addition, understand what already exists:

  1. Read the project's entry point, architecture, and key modules
  2. Identify the project's core promises (zero-config? fast? portable? minimal deps?)
  3. Identify what problems are already solved by existing mechanisms
  4. Note the dependency philosophy (pure stdlib? minimal? heavy?)

Phase 2 — Steel-Man Both Sides

Generate at minimum 4 arguments against and 4 arguments for, covering these axes:

Against (challenge vigorously):

  • Does it solve a problem users actually have, or a theoretical one?
  • Does it violate existing project promises (speed, portability, zero-config)?
  • Is there a lighter-weight alternative already available in existing deps?
  • What's the dependency/install cost vs. the marginal improvement?
  • Does the existing approach already handle 80%+ of cases adequately?
  • Is the motivation external validation (listings, badges) rather than user value?

For (advocate honestly):

  • What new capabilities does it unlock that are currently impossible?
  • What efficiency gains (cost, tokens, time) does it provide?
  • Does it create competitive differentiation?
  • Is the dependency concern manageable (optional import, headless variant)?
  • What user segments would materially benefit?

Phase 3 — Reframe the Decision

Ask the pivotal question:

"What is the actual bottleneck for user outcomes, and does this proposal address it?"

Map existing solutions to existing problems. Identify whether the proposal addresses an unsolved problem or merely optimizes an already-adequate solution.

Phase 4 — Evaluate Alternatives

Before accepting the proposed library, ask:

"Can the existing toolchain deliver 70%+ of the benefit at 0% new cost?"

Check whether:

  • An existing dependency already has the capability (e.g., ffmpeg filters vs. a dedicated library)
  • A flag/option on existing tools covers the use case
  • The "premium" library is only needed for edge cases that can be deferred

Phase 5 — Verdict Structure

Deliver a clear recommendation with:

  1. Decision: Accept / Reject / Accept-with-alternative
  2. Decisive factor: The single strongest argument that tipped the balance
  3. Recommended implementation: If the feature is worth having, specify the lightest path to ship it
  4. Escalation trigger: Under what future condition should you revisit (e.g., "if 3+ users request adaptive thresholds")
  5. Summary table: Compare the proposed approach vs. alternative on: deps, speed, install size, quality, sufficiency for the use case

Anti-Patterns

  • Don't let external validation (awesome-lists, badges) drive architecture
  • Don't conflate "the feature is valuable" with "this specific library is necessary"
  • Don't accept heavy deps when existing tools cover the use case adequately
  • Don't reject purely on principle — if the dependency is genuinely the only path, accept it
  • Don't approve just because the library is popular or well-maintained

Example: PySceneDetect for video scene detection

Proposal: Add PySceneDetect to a media-ingestion skill for AI agents.

Phase 3 reframe: The bottleneck for agent understanding is transcript availability and video length, not frame selection precision. Even-spaced frames are "good enough" for 90% of queries.

Phase 4 alternative: ffmpeg -vf "select='gt(scene,0.3)'" provides shot-boundary detection with zero new dependencies.

Verdict: Reject PySceneDetect. Ship the feature via ffmpeg's native scene filter (~30 LOC, 0 new deps). Revisit if users request adaptive threshold tuning.


License

MIT — Use freely in agent instructions, CLAUDE.md files, skill definitions, or system prompts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment