Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save sergeliatko/a4318e68ad07a354983071380efe6c70 to your computer and use it in GitHub Desktop.

Select an option

Save sergeliatko/a4318e68ad07a354983071380efe6c70 to your computer and use it in GitHub Desktop.
Software Architecting Agent Surface Intake: Superpowers Comparative Analysis

Software Architecting Agent Surface Intake: Superpowers Comparative Analysis

Document Control

Field Value
Purpose Compare the locally cached OpenAI-curated Superpowers package with the architecture, planning, testing, writing, coordination, and feedback practices being developed for this repository and the wider portfolio.
Authority Evidence and analysis only. This report does not change SFS-001, accept an architecture decision, modify a skill, or authorize implementation.
Evidence date 2026-07-24
Repository scope saas-framework, with read-only evidence from named sibling repositories and installed skill snapshots.
Decision state No adoption decision is requested merely to read or retain this report. Proposed changes remain subject to their normal authority and checkpoints.

Executive Conclusion

The Superpowers package should not be adopted as the governing software-architecture method for this repository or portfolio.

It contains several useful engineering feedback practices: reproduce failures, observe an expected test failure, make one evidence-based change, test behavior instead of mocks, trace defects toward their origin, use condition-based waiting, verify before claiming completion, isolate risky work, and review work from a fresh context. Those practices fit the direction of the proposed outside-in functionality design decision when they are applied proportionally and at the boundary that owns the behavior.

The package also makes workflow rules universal when they are only situational techniques. It treats brainstorming, specification writing, code-heavy implementation plans, two-to-five-minute tasks, function-level tests, repeated fresh verification, subagent-per-task execution, review after every task, and a fixed branch-finishing menu as mandatory. This can make the workflow define the architecture, duplicate authority, rerun expensive evidence for an unchanged source state, and interrupt a human for choices already settled by repository policy.

The most serious issue is authority. The package says its skills override default system behavior and requires skill invocation when there is a one-percent chance of relevance. A plugin cannot assign itself that position in the instruction hierarchy. Repository authority, accepted architecture, security boundaries, explicit user intent, and system constraints must remain controlling.

The local architecture system is stronger on purpose, semantic ownership, authority, test responsibility, proportionality, repository-specific planning, human checkpoint boundaries, evidence provenance, and cross-repository non-authority. It is weaker because several parts are still proposals, the installed skills overlap, some guidance contains unsupported universal claims, the planning schema has not yet adopted the strongest contextless-task features found in the beat-of-hawaii-seo planning system, and no shared evaluation harness currently measures whether the combined agent surface improves outcomes.

The recommended direction is selective adaptation. Keep Superpowers techniques that create trustworthy feedback, reject its self-declared authority and mandatory ceremony, test the remaining practices against representative work, and build an agent surface around outside-in design, explicit authority, behavior ownership, proportional evidence, source-state-aware verification, and recipient-aware delegation.

Scope

This report answers five questions.

  • What does the inspected Superpowers package actually contain?
  • Which parts improve software development and agent behavior?
  • Which parts conflict with the architecture and delivery approach being developed here?
  • Which weaknesses remain in the local approach?
  • What should a better software-architecting agent surface be able to do and prove?

The comparison covers architecture discovery, functionality design, task decomposition, planning, test-driven development, interface and Contract testing, debugging, verification, source isolation, code review, agent delegation, skill authoring, code comments, Markdown, cross-repository coordination, feedback, delivery, and maintenance.

The comparison does not evaluate model quality in general, claim that one workflow fits every repository, or accept the proposed functionality-design ADR.

Evidence Method

Claims in this report use three evidence states.

State Meaning
Observed Directly present in a repository file, installed skill, plugin manifest, test, schema, or generated report inspected for this review.
Derived A comparison or implication reasoned from observed sources.
Proposed A possible improvement that has not been accepted as architecture or skill policy.

Current repository specifications and accepted decisions outrank every other source in this report. Installed skills are operating guidance, not repository architecture. Sibling repositories are prior-art evidence, not authority. The curated plugin is evidence about its own method, not authority for this repository.

This report records material contradictions instead of averaging them into an artificial consensus. It also distinguishes observed source facts from claims made by the source about its own effectiveness.

Evidence Register

Source Evidence role Authority here Confidence
SFS-001 Normative framework architecture, documentation, test, and governance requirements Controlling High
SFS-ADR-0012 Accepted documentation and agent authority Controlling High
SFS-ADR-0021 Proposed outside-in functionality-design method Proposal only High as proposal text
Functionality design consistency review Earlier comparison of repository and portfolio design practices Evidence High
Repository planning convention Human-plan and machine-progress authority split Controlling for repository plans High
software-architecting installed skill Architecture lifecycle, artifacts, decisions, traceability, agent outputs, and evaluation Guidance High as observed guidance
task-analysis and task-delegation installed skills Problem decomposition and recipient-aware task encoding Guidance High as observed guidance
code-commenting and code-style-markdown installed skills Code-local specification and portable documentation rules Guidance, subject to repository rules High as observed guidance
coordinate-github-repositories installed skill Cross-repository authority, inventory, writing, feedback, and tool-fit rules Guidance High as observed guidance
knowledge-extraction and skill-review installed skills Provenance, contradiction handling, and skill-quality review Guidance High as observed guidance
superpowers plugin version 5.1.3, cache revision bd2122cb Curated development-method package under comparison Evidence only High as observed package
Bundled anthropic-best-practices.md Anthropic-authored skill guidance copied into Superpowers Evidence only High as bundled text
research-software-architecture outputs Research synthesis for lifecycle, artifacts, traceability, TDD, DDD, delivery, and agent evaluation Sibling evidence Medium to high by claim provenance
test-software-architecture-tool schema v3 and self-description Addressable unit, environment, rule, evidence, and planning experiment Sibling evidence High for implemented behavior
template-saas-app decision 0016-testing-is-conformance-first.md Prior conformance-test ownership experiment Sibling evidence High for that repository's decision
beat-of-hawaii-seo planning schema v2 Stronger contextless task contract and review-state experiment Sibling evidence High for implemented planning rules

Provenance Findings About Superpowers

The inspected package is not an OpenAI-authored software-engineering standard. Its manifest identifies superpowers version 5.1.3, authored by Jesse Vincent, with https://github.com/obra/superpowers as its repository. The package is present under a local openai-curated cache, which supports the narrower claim that OpenAI curated or distributed this snapshot.

The package was not advertised in the current session's active skill list. Calling it a default Codex skill would therefore be inaccurate for this session. The precise description is a locally cached OpenAI-curated plugin package inspected by request.

The package bundles an Anthropic skill-authoring guide and calls it official Anthropic guidance. This is legitimate comparative material, but it creates an important provenance distinction: content shipped inside an OpenAI-curated package is not thereby OpenAI-authored.

The Superpowers writing-skills guidance and its bundled Anthropic guide disagree. Superpowers says a skill description must describe only when to use the skill and not what it does. The bundled guide says the description should include both what the skill does and when to use it. A better agent surface must resolve such contradictions explicitly instead of treating every bundled instruction as mutually authoritative.

Operational Availability and Implementation State

The quality of a research archive is not the same as the behavior of the agent surface used during ordinary coding.

A method can exist in at least seven distinct states.

State Meaning
Researched The idea exists in source notes, experiments, or synthesis.
Distilled The idea has been selected and rewritten into an operational skill, repository rule, template, or tool.
Discoverable The agent can see metadata that makes the operational artifact selectable.
Loaded The full instructions or relevant reference were brought into the current context.
Applied The agent used the instruction in the current work.
Enforced A schema, validator, test, policy gate, or permission boundary checks the behavior.
Demonstrated Representative outcome evidence shows that the behavior helped under stated conditions.

These states must not be collapsed. Research that was never distilled does not guide routine coding. A skill that is discoverable but not loaded contributes only its metadata. A loaded instruction that is not applied does not explain the outcome. An applied guideline without enforcement or evaluation remains judgment, not proof.

Research to Software-Architecting Skill

The inspected research-software-architecture output contains 67 Markdown files and 5,697 source lines in the current snapshot. The installed software-architecting package contains 47 Markdown files and 2,994 source lines, including a 160-line entry skill, 25 templates, focused references, and test fixtures.

The installed skill is a real distillation rather than an expectation that every coding agent reread the research repository. Its entry file defines applicability, operating modes, authority, proportional artifacts, evidence rules, and a reference router. Its reference index explicitly says to load only the reference needed for the current decision.

That progressive disclosure is operationally important. During ordinary narrow coding, the software-architecting skill should not activate at all. During architecture work, the entry skill loads first and only relevant references should follow. The complete research corpus is normally absent from the coding context.

The consequence is that only ideas preserved in the entry skill or routed references influence ordinary agent behavior. A valuable research conclusion that remains solely in transcripts, extraction notes, or an unreferenced synthesis is archival knowledge, not an implemented agent capability.

The distillation also changes responsibility. The research repository can preserve broad source provenance and competing methods. The operational skill must make a compact, actionable choice about what to load, what to do, when to stop, and which authority governs. Fidelity should therefore be evaluated by traceable coverage and real task outcomes, not by copying every research artifact.

Superpowers Availability

The inspected Superpowers snapshot contains 37 Markdown files and 7,054 source lines, including fourteen skill entrypoints and their references. This count includes the 1,150-line bundled Anthropic guide.

The package was cached but not listed in the current session's active skill catalog. Its entrypoint metadata and skill bodies were therefore not part of ordinary task execution in this session. They were read deliberately for this comparative report.

This distinction prevents two attribution errors. Useful Superpowers ideas should receive credit as inspected prior art, but dormant package content should not receive credit for behavior produced by active repository rules or user skills. Conversely, a defect in normal coding should not be attributed to Superpowers unless the relevant skill was actually available, loaded, and applied.

Visible OpenAI-Provided Surface

The active Codex surface includes visible system and developer instructions, tool contracts, the available-skill catalog, and OpenAI-bundled or curated plugin metadata. Those visible instructions materially shape tool use, safety, communication, file editing, and skill loading.

Full skill bodies are progressively loaded when their triggers apply or the user names them. Unrelated OpenAI-bundled skills remain metadata rather than active methodology. This report does not speculate about undisclosed model training or hidden runtime behavior and does not attribute an outcome to content that cannot be shown as visible or loaded.

For this review, the local architecture, task, commenting, Markdown, coordination, knowledge-extraction, and skill-review instructions were loaded because the comparison required them. The Superpowers skills and selected references were loaded specifically as the subject of review. The broader research and sibling repositories were read selectively for evidence and were not treated as normal coding context.

Implemented Versus Proposed Comparison

Item Current operational state
SFS-001 and accepted decisions Loaded through repository guidance when applicable and enforced partly by repository checks and human checkpoints
Repository Markdown rules Applicable to every changed Markdown file and enforced by npm run docs:check
Repository planning v1 Implemented through schema, validator, plan package, and default checks
software-architecting skill Distilled, discoverable, and loadable; outcome evaluation remains limited
Outside-in functionality-design ADR Proposed and inspected, but not accepted or implemented as repository authority
Research lifecycle and traceability corpus Available as evidence; only the skill projection is normally operational
Unit architecture schema v3 Implemented in its sibling experiment, not in this repository
Beat of Hawaii planning v2 Implemented in its sibling repository, not adopted here
Superpowers package Cached and inspected for this report, but not active in the current skill catalog
Verification receipt store Proposed by this report, not implemented
Portfolio agent feedback contract Identified as a gap, not implemented
Revised code-commenting specification Required by the proposed ADR, not yet implemented

Shared Vocabulary

The comparison uses the vocabulary from the proposed outside-in decision without treating that proposal as accepted.

Term Meaning in this report
Problem A condition, need, or opportunity the software is intended to address.
Functionality or behavior An observable effect the software must produce to help address a problem.
Responsibility Ownership of related behavior by a boundary or element.
Design scope The bounded subject currently being understood or refined.
Purpose Why a scope is needed and how it contributes to its parent context.
Specification An accepted, verifiable statement of required behavior and constraints.
Interface The boundary through which a caller observes or requests behavior, whether or not it is a language interface.
Test One form of evidence that exercises a stated behavior under stated conditions.
Task Work needed to discover, decide, implement, verify, document, or deliver behavior.
Plan A coordination artifact for tasks, dependencies, evidence, and checkpoints, not architecture authority.
Skill Agent operating guidance selected for a class of work, not a source of repository architecture merely because it is installed.
Feedback An observation about behavior or outcome that may justify a change after validation.

The Two Governing Models

Superpowers presents a largely linear default development sequence.

brainstorm
  -> write specification
  -> write implementation plan
  -> create worktree
  -> execute two-to-five-minute tasks
  -> apply test-first development
  -> review each task
  -> rerun verification
  -> choose a branch-finishing action

The local direction begins with authority and the largest relevant problem, then refines inward while evidence can revise broader understanding.

repository authority and current evidence
  -> relevant problem, purpose, values, constraints, and interactions
  -> responsibilities and semantic boundaries
  -> behavior and interface at the owning boundary
  -> proportional specification and proof
  -> implementation and delivery tasks
  -> evidence, review, release, and operational learning
  -> outward revision when new evidence changes the design

The distinction is not that one model plans and the other does not. The distinction is what governs decomposition. Superpowers usually decomposes delivery into files, tests, and small tasks after a brainstorming artifact. The proposed local model decomposes the problem into behavior and responsibility before using tasks to deliver it.

Comparative Summary

Concern Superpowers Local direction Assessment
Starting point Brainstorming workflow and design questions Repository authority and largest relevant problem Adapt Superpowers discovery questions under local authority
Architecture Design document followed by implementation plan Outside-in responsibility and behavior decomposition Local direction is stronger
Authority Skills self-declare high priority Specification, accepted decisions, and explicit checkpoints govern Reject Superpowers hierarchy
Planning Exact files, code, tests, commands, and tiny steps Human intent plus machine progress and dependencies Prefer local split
Testing Universal function-level test-first loop Proof owned by stable behavior boundary and risk Keep loop, change ownership and proportionality
Debugging Reproduce, investigate, hypothesize, change, verify Compatible evidence loop Adopt with boundary and security corrections
Verification Fresh command immediately before every claim Evidence appropriate to changed source state and claim Adapt with verification receipts
Delegation Fresh subagent for each independent task Recipient-aware delegation with authority and context Prefer local model
Review Mandatory specification and quality review per task Review proportional to semantics, risk, and checkpoint Adapt fresh-context review selectively
Writing Strong prescriptive rules and pressure compliance Clear actors, actions, ownership, evidence, and authority Prefer local rationale
Skill improvement Pressure scenarios and compliance tests Observed behavior, feedback, provenance, and outcome evaluation Keep baseline experiments, reject compliance as sole outcome
Git workflow Worktree and fixed finish menu Repository workflow and explicit authority Repository-specific policy must govern

Superpowers Skill Inventory

The package contains fourteen skills.

Skill Useful contribution Main problem Disposition
using-superpowers Forces deliberate skill discovery Incorrect authority hierarchy and one-percent activation threshold Reject governing rules
brainstorming Explores context, constraints, success, and alternatives Mandatory ceremony and separate design artifact for every creative change Adapt
writing-plans Makes execution context explicit Locks files and code too early; duplicates source truth Reject as general plan format
executing-plans Tracks batches and blockers Assumes its plan is correct and adds review interruptions Adapt only under repository plan
using-git-worktrees Encourages isolation and containment checks Universal setup, dependency install, and baseline verification Adapt to repository policy
subagent-driven-development Uses fresh-context implementation and review Excess agents, duplicated context, repeated tests, and plan-as-truth Adapt selectively
dispatching-parallel-agents Checks independence before parallel work File independence is insufficient; shared state can still conflict Adapt with ownership and isolation
test-driven-development Strong red-green-refactor feedback loop Universal Iron Law and function-level test ownership Adapt substantially
systematic-debugging Evidence before fixes and root-cause tracing Universal ceremony, single-root absolutism, duplicated validation Adapt
verification-before-completion Requires evidence before claims Repeats costly checks without source-state identity Adapt
requesting-code-review Provides structured review context and severity Mandatory review and generic reviewer authority Adapt
receiving-code-review Verifies feedback instead of blindly accepting it Encodes unnecessary social prohibitions as technical rules Keep technical core
finishing-a-development-branch Checks destructive cleanup and final state Fixed menu ignores repository delivery policy and authority Reject menu; keep containment checks
writing-skills Tests skill activation and behavior with fresh agents Conflates TDD with compliance engineering and uses persuasion to close dissent Adapt evaluation ideas; reject coercive framing

Findings by Superpowers Workflow Stage

Skill Selection

The useful idea is that a relevant skill should be inspected before acting, because skill instructions can materially change the safe workflow.

The one-percent rule is not workable in a large skill catalog. It expands context, creates overlapping mandatory workflows, and makes non-activation nearly impossible to evaluate. Skill discovery should optimize both recall and precision: invoke a skill when its declared trigger and the task evidence make it relevant, record why overlapping skills were selected, and permit a justified non-selection.

The package's priority order is invalid. A skill cannot override system constraints or declare itself above repository authority. Explicit user direction also cannot authorize actions prohibited by system safety or accepted repository checkpoints.

Brainstorming

The brainstorming skill correctly asks for purpose, constraints, success, meaningful alternatives, and design review before implementation. It also tries to keep exploration isolated from the main branch.

Its universal gate is excessive. Requiring brainstorming before every creative, configuration, or behavioral change treats uncertainty as a mandatory meeting format. Requiring one question per turn, two or three alternatives, section-by-section approval, a separate specification, and a committed artifact can increase human interruption without increasing understanding.

The reusable practice is proportional discovery. Ask only questions that change the design, compare alternatives only when there are meaningful alternatives, and create a standalone artifact only when the knowledge needs a durable carrier.

Implementation Planning

The writing-plans skill is designed for an implementer with little context. That goal is sound.

Its chosen encoding is unsound as a general repository plan. It asks the planner to name exact files, include full code, prescribe exact tests and commands, break work into two-to-five-minute steps, and commit frequently. This lets implementation structure define the problem before the behavior and ownership are stable.

Copying code into a plan creates a second source of truth. Exact line numbers and code snippets become stale as soon as implementation changes. Micro-tasks also encourage repeated setup, verification, and review for the same effective source state.

The local planning split is better: the human document owns intent, evidence, architecture context, risks, checkpoints, and acceptance; the machine tracker owns stable task identities, dependencies, live status, questions, commands, and gates. Neither plan artifact can override accepted architecture.

Worktrees and Branches

Isolation is valuable when work is risky, concurrent, experimental, or likely to conflict. Checking that an isolation directory is ignored and avoiding deletion of an active harness worktree are useful containment practices.

Creating a worktree, installing dependencies, and running a full baseline suite for every change is not universally efficient. Repository policy, existing workspace state, concurrent ownership, toolchain cost, and change risk should determine isolation.

The fixed finish menu also asks a human to choose among merge, push, keep, or discard even when the repository already requires a pull request or another release path. The agent should follow the repository's accepted delivery policy and ask only when a real unresolved choice remains.

Plan Execution and Subagents

Fresh-context implementation and review can reveal hidden assumptions. Separating semantic compliance review from code-quality review can also help when those are genuinely independent questions.

A fresh subagent per small task, followed by a specification reviewer and a quality reviewer, multiplies context transfer and verification cost. It assumes the plan is correct, although plan compliance can preserve a design error. It also treats a recipient's report as evidence while simultaneously requiring the parent to repeat the work.

The local task-delegation skill has a stronger foundation: classify the recipient, encode the goal, terms, rules, examples, and verification needed by that recipient, and apply a blank-slate test. Delegation should be used when work is independently bounded and parallelism or fresh context adds value, not as a ritual for every task.

Parallel work requires more than different files. Agents can still conflict through generated artifacts, package locks, repository state, shared decisions, tests, or a common semantic boundary. Safe parallelism needs explicit ownership, isolated mutable state where needed, and an integration order.

Code Review

The review skills correctly insist that feedback be technically checked. A reviewer, human or agent, can misunderstand the repository, and performative agreement is not a substitute for evidence.

The package is internally awkward about communication. One skill requires praise in a review while another forbids gratitude and certain conversational responses. These are style preferences presented as engineering laws.

The stronger rule comes from repository coordination: feedback is an observation, not an accepted diagnosis or an authority grant. Record the exact evidence, assess it against current authority and behavior, apply the correction if valid, and preserve a precise disagreement when it is not.

Test-Driven Development Analysis

What the Superpowers TDD Skill Gets Right

Watching a test fail for the expected reason proves that the test can detect the missing or broken behavior under that scenario. This is stronger than writing a test after implementation and observing only a green result.

Writing the caller-facing example before implementation can expose an awkward interface. Minimal implementation followed by refactoring keeps the feedback loop understandable. Reproducing a bug with a failing test produces durable regression evidence when the behavior is stable and the test is placed at the correct boundary.

The skill also discourages tests that merely assert mock calls, production-only test hooks, incomplete mocks, and implementation-detail coupling. Those are valuable warnings.

Where the TDD Skill Overreaches

The skill calls test-first development an Iron Law, requires pre-test code to be deleted without even using it as a reference, requires every new function or method to have a test, and says a failed green step means fixing code rather than the test. Each statement is too broad.

Code can be exploratory evidence, generated output, a behavior-preserving refactor, a type-level declaration, a migration artifact, a thin adapter, or an implementation of already accepted conformance semantics. The right proof depends on the claim and risk.

A failing test can be wrong because the specification changed, the fixture is invalid, the test boundary is misplaced, the expectation is obsolete, or the test accidentally encodes an implementation detail. "Fix code, not test" is therefore not a valid universal rule.

Function-level test ownership makes source structure determine test architecture. It encourages one test per implementation unit instead of one durable proof per owned behavior. Refactoring then becomes expensive because tests move with functions instead of remaining stable at semantic boundaries.

The instruction to delete an implementation and not look at it can discard valuable evidence about constraints, provider behavior, performance, or a rejected approach. Exploratory code should not silently become production code, but its durable findings should be migrated before it is removed or renamed.

The Better Test Boundary

Tests should be owned where the behavior is defined and should remain stable across implementation changes that preserve that behavior.

Behavior owner Primary proof responsibility
Accepted Contract Reusable behavior-focused conformance Testkit covering the Contract's observable semantics
Module The Contract Testkit plus Module-owned options, Dependencies, provider translation, lifecycle, security, isolation, and implementation-specific risks
Fixture The same Contract Testkit plus deterministic Fixture rules and the prohibition on external I/O
Application Domain behavior, operation semantics, orchestration, policy, Composition, and Projection behavior owned by the Application
Pure local logic Focused examples, properties, types, or unit tests when they protect meaningful behavior
Deployment or release path Artifact identity, pipeline evidence, smoke behavior, rollback, and authorized live verification

This model follows SFS-CONTRACT-008 and SFS-TEST-*. It also preserves the useful prior-art insight from template-saas-app: a conformance suite should live with the stable boundary and run against every implementation of that boundary.

The prior port-based decision is not copied literally. In this repository a Contract owns the conformance Testkit, a Module fulfills one Contract, and a Fixture is a separate test artifact class. The architectural noun and ownership model differ even though the reusable-proof principle is compatible.

Characterization, Exploration, and Refactoring

Characterization tests are legitimate when current behavior must be discovered before deciding what should remain. They are evidence about the inherited system, not automatic approval of that behavior.

Exploratory tests and spikes may be temporary. Before removal, their durable knowledge must migrate to an accepted specification, stable test, implementation constraint, ADR, or documented rejected alternative when that knowledge matters.

A behavior-preserving refactor does not need a new test for every moved function. It needs credible existing proof at the behavior boundary, focused new proof for any newly exposed risk, and verification that the source change did not alter owned semantics.

Mocks, Fakes, Fixtures, and Provider Schemas

The Superpowers anti-pattern guidance says mocks should be complete enough for code under test. That is directionally useful but underspecified.

Test data should be valid for the Contract scenario being exercised, not an arbitrary copy of every provider field. Contract Testkits, deterministic Fixtures, provider-response builders, and focused malformed cases are better than ever-growing mocks that mirror an SDK.

Provider schemas remain evidence at the Module boundary. Domain and Application tests should not become coupled to the provider's complete wire shape unless that wire shape is itself the owned behavior.

Debugging Analysis

The systematic-debugging skill has one of the strongest reusable cores in the package.

  • Observe the failure and read the complete error.
  • Reproduce it reliably or state why reproduction is incomplete.
  • Inspect recent changes and collect evidence at the relevant boundaries.
  • Trace invalid data or state toward its origin.
  • Form one explicit hypothesis.
  • Make the smallest change that can confirm or reject that hypothesis.
  • Verify the fix against the reproduction and the affected behavior.

The condition-based waiting guidance is also sound. Tests should wait for the state they need rather than sleep for an assumed duration.

Three corrections are required.

First, not every incident has one root cause. Distributed failures can have several necessary conditions, independent defects, or an initiating cause plus containment failures. The report should distinguish trigger, contributing conditions, detection gap, and impact amplifier when they differ.

Second, the prescribed four-phase ceremony should scale with the problem. A clear compiler error does not require the same artifact set as an intermittent production race.

Third, diagnostics must obey security and privacy boundaries. Commands that dump environment variables or provider payloads are not acceptable merely because they help find a cause. Evidence collection must minimize disclosure and preserve secret references rather than values.

The package's "three failed fixes means architecture problem" rule is a useful prompt, not proof. Repeated failed changes should trigger a scope and assumption review; they do not establish one architectural diagnosis.

Verification Analysis

Superpowers correctly rejects completion claims based on intuition, agent reports, stale memory, or partial output. A verification claim should identify what command or observation ran, what source state it covered, what result it produced, and what it does not prove.

Its requirement to run a fresh command immediately before every completion statement is too narrow. It ignores stable artifact identity, unchanged source state, cached CI evidence, deterministic build inputs, and expensive checks.

The repository has already encountered the practical consequence: rebases or release mechanics can create a different commit identity without changing the tested tree, and naive workflow chaining reruns the same costly checks.

A better verifier needs a receipt keyed to at least:

source tree identity
dependency lock identity
toolchain identity
verification command and configuration
relevant environment or target identity
start and finish timestamps
duration
exit status
result artifact identity
authorization class when live or credentialed

The verifier can reuse a receipt only when the claim's inputs and required environment are unchanged and the repository policy permits reuse. A source or configuration change invalidates only the evidence it can affect.

Immediate registry or deployment verification should be time-bounded and resumable. Polling should record elapsed time, last observed state, retry count, terminal condition, and a command for resumption rather than holding an agent session open without a useful bound.

Planning Systems Analysis

Superpowers Plans

Superpowers plans optimize for direct execution by a low-context implementer. They include exact paths, full code, exact commands, tiny steps, and frequent commits.

That format is useful for a tightly bounded mechanical task after architecture and interfaces are stable. It is harmful as the default architecture-to-code bridge because the plan becomes a parallel implementation and source structure becomes an accidental specification.

SaaS Framework Planning Version 1

The repository planning convention separates meaning from live state.

The initiative README.md owns outcome, evidence, scope, architecture context, rollout, risks, checkpoints, and acceptance. The progress.yaml owns stable IDs, status, dependencies, human gates, questions, commands, and completion gates. The JSON schema owns structure and IDE completion. The TypeScript validator owns repository semantic checks such as missing dependencies, cycles, and invalid lifecycle transitions.

This split prevents Markdown checkboxes from becoming a second tracker, prevents a schema from granting approval, and prevents a plan from becoming architecture authority. It also allows the filesystem and validator to authoritatively report structural facts while human-reviewed documents own meaning.

The current framework planning system was explicitly adapted from the sibling beat-of-hawaii-seo planning package. That sibling is evidence, not authority.

Beat of Hawaii Planning Version 2

The sibling v2 tracker adds task kinds, owners, instructions, deliverables, acceptance, verification, references, evidence, decisions, review state, and branch or pull-request lifecycle.

Its strongest improvement is contextless task execution. A task can tell a recipient what work is being delegated, what it produces, how it is accepted, how it is verified, and which source establishes its context without pasting the entire plan into a prompt.

It also distinguishes ready_for_review from approval and validates references so private machine paths or missing evidence do not silently enter a durable plan.

Its risk is duplication. If instructions, deliverables, and acceptance restate the governing specification rather than referencing it, the tracker becomes another semantic authority. Its branch and merge rules are repository-specific and should not be ported as a universal schema.

Planning Comparison

Concern Superpowers Framework v1 Beat of Hawaii v2
Human intent Design and plan prose Initiative README.md Initiative README.md
Live state Markdown task execution progress.yaml progress.yaml
Task context Full code and exact steps ID, status, dependencies, commands Owner, instructions, deliverables, acceptance, verification, references
Review state Review after task Human checkpoint task ready_for_review, decision, and checkpoint state
Authority boundary Plan treated as execution specification Plan cannot override architecture Plan distinguishes facts, proposals, and decisions
Structural validation Informal checklist JSON schema plus semantic validator Richer schema plus reference and lifecycle validation
Main strength Immediate executability Clean authority split Strong contextless task contract
Main risk Stale duplicate source and micro-task churn Insufficient handoff detail for some tasks Verbose task-as-spec duplication

Planning Recommendation

Keep the framework v1 authority split. Pilot selected v2 task fields where fresh-agent handoff has failed or work crosses contexts: kind, owner, instructions, deliverables, acceptance, verification, references, and ready_for_review.

Do not adopt those fields merely because another repository has them. Measure whether they reduce clarification turns, context replay, incorrect authority assumptions, and review defects without duplicating specifications.

Architecture and Decomposition Analysis

Research Software Architecture

The research-software-architecture repository provides a broad evidence-backed lifecycle covering idea intake, discovery, validation, product shaping, domain modeling, specification, architecture, implementation, delivery, maintenance, and reverse engineering.

Its strongest contribution is separation of concerns. Discovery is not validation, EventStorming is not a final domain model, a bounded context is not automatically a service, a behavior example is not automatically an automated test, and deployment evidence is not business release approval.

Its provenance labels are also valuable. Source facts, synthesis, and agent-design proposals are distinguished instead of presented as one authority.

The weakness is the tendency to turn a useful concern model into a required artifact chain. Its traceability model can be read as requiring every code element to connect through idea, problem, opportunity, impact, story, requirement, test, release, and metric. That level of universal traceability is impractical and can hide the important links inside administrative volume.

Its evaluation rule that a coding task without discovery or release context fails is too broad for a mechanical correction, local refactor, generated update, or already-governed implementation task. Lifecycle coverage should be a completeness prompt at the relevant initiative level, not a refusal rule for every local action.

The proposed outside-in ADR corrects this by treating lifecycle stages as interacting concerns, using proportional carriers, and distinguishing functionality decomposition from work decomposition.

Unit Architecture Tool

The test-software-architecture-tool experiment makes design scopes addressable through architecture.unit.yaml and architecture.environment.yaml carriers. Its schema can express purpose, parts, provided and required behavior, maintained conditions, relationships, parent context, inherited rules, prohibited relationships, assumptions, questions, evidence, maturity, and immediate-child granularity.

Several ideas are valuable for a future agent surface.

  • Design scopes can have stable identities without being implementation functions.
  • Parent constraints can flow to children and be stopped or superseded with reasons.
  • Immediate-child granularity can prevent a parent plan from prematurely specifying deep implementation.
  • Evidence plugins should emit observations, while comparison policy interprets those observations.
  • Questions, assumptions, evidence, and maturity belong in the design process.
  • Generated reports can compare declared intent with physical source facts.

The experiment also exposes important failure modes.

  • Sidecar files can make filesystem layout define architecture.
  • Requiring a carrier for every folder or file can create large maintenance overhead.
  • The term Environment conflicts with the SFS meaning of native runtime input.
  • A generic why field can conflate purpose, problem, and rationale.
  • Comment rules can require full docblocks for every export even when the types and parent context already carry the meaning.
  • A zero-finding self-check proves agreement with the authored schema and enabled evidence, not that the architecture is semantically correct.

The useful boundary is explicit: the filesystem can authoritatively report that a file, Package, export, import, or declared relation exists. It cannot establish why the element exists, whether a boundary is meaningful, or what historical rationale should govern it.

Outside-In Functionality Design

The proposed SFS-ADR-0021 is the strongest synthesis currently available, but it remains proposed.

It starts from the highest scope needed to understand the problem, carries purpose, values, constraints, and invariants inward, and lets lower-level evidence revise broader decisions outward. It explicitly prevents task size, file boundaries, test units, or workflow stages from defining architecture.

It also provides proportional carriers. A cross-cutting decision may need an ADR; a Module may need a specification and README; an important class may need a local docblock plus a broader reference; a small behavior may need only a name, types, implementation, and focused test.

The proposal still needs its planned proof. It must be challenged with the SEO Agent Tools Utils capability, a trivial counterexample, fresh-context navigation, the unit-architecture experiment, the conformance-testing experiment, and at least one non-SFS portfolio pilot before it can support a wider standard.

Interface-Driven and Domain-Driven Design

Superpowers TDD often uses a wished-for API as the starting point. That can improve ergonomics, but a caller example alone does not decide which boundary owns the behavior, which domain language applies, which values must be protected, or whether a public abstraction is justified.

The local direction designs the interface after the owning behavior, caller, interactions, authority, inputs, outputs, outcomes, errors, side effects, and constraints are sufficiently understood. A language interface is optional; the behavior boundary is not.

Domain-driven design contributes purpose-specific models, bounded language, behavior ownership, events, and context boundaries. It does not require every technical behavior to become a domain object, and a bounded context does not automatically become a Package, Module, service, or task.

Interface design and domain design therefore support outside-in functionality design. Neither replaces it.

Code Commenting Analysis

The installed code-commenting skill has a valuable premise: preserve non-obvious purpose, constraints, invariants, failure semantics, side effects, and rationale near the code they govern, and remove stale narration when behavior changes.

It also encourages checking usages before changing public semantics and using references to avoid duplicating broader authority.

Several current rules need revision before portfolio adoption.

  • @why conflates purpose and rationale.
  • Requiring a docblock for every public or exported method can produce information-free comments.
  • Requiring file headers and @since tags universally can duplicate Git history and type declarations.
  • Preferring structured tags over prose in all cases can make local explanations harder to read.
  • Audience-prefixed tags and claims about comment pollution need demonstrated consumers and evaluation.

The proposed outside-in decision gives a better comment model. A local comment specification should be able to carry a concise description and purpose, background, conceptual inputs and outputs, constraints, invariants, rationale, failure and side-effect behavior, maturity, and references when those facts are material.

The carrier remains proportional. A trivial function does not need a ceremonial block. A complex implementation can begin in a Git-tracked <name>.draft.<extension> file after the broader responsibility is established, then be renamed into its final implementation after durable design knowledge has migrated and release tooling proves no draft enters the product.

Rejected alternatives should not be erased automatically. Preserve a rejection at the narrowest authoritative level when its reason protects a value, constraint, invariant, or future decision.

Markdown and Writing Analysis

Repository Coordination Writing Rationale

The writing guidance inside coordinate-github-repositories is stronger than a generic instruction to be concise.

It protects the semantic actors in a repository change: who acts, what action occurs, which object changes, who owns it, what evidence supports the claim, what remains unknown, and which repository has authority. It removes empty framing, overloaded sentences, dense modifier chains, vague safety language, and invented contrasts because those patterns obscure responsibility and decision state.

This matters architecturally. A sentence that hides the actor can hide who grants authority. A sentence that collapses source and target repositories can turn evidence into a cross-repository mandate. A sentence that says a change is "safer" without naming the concrete harm can conceal the actual tradeoff.

The guidance also protects exact technical literals and user-established terminology. Plain language does not mean replacing precise Contract names, commands, identifiers, or provider terms with vague synonyms.

Markdown Guidance

The installed Markdown skill and this repository's Markdown instructions support structural headings, one source line per paragraph, ASCII punctuation, labeled fences, relative links, short table cells, and explicit authority. These rules improve portability across GitHub, IDEs, parsers, humans, and agents.

Some rationale in the installed skill should not be treated as universal fact. Claims about precise model attention, parser truncation, ideal word counts, bold text, or a universal ban on nested lists require evaluation in the target toolchain. This repository has adopted a stricter subset as local policy, so the rule is valid here because the repository owns it, not because every claimed cognitive explanation is proven.

Terminology consistency also needs semantic judgment. Choosing one word everywhere is harmful when terms such as test, check, verification, specification, and review refer to different responsibilities.

Superpowers Skill Writing

Superpowers applies a TDD analogy to skill authoring: create a pressure scenario, observe an agent violate the intended rule without the skill, add instructions, and repeat until the agent complies.

The useful part is empirical. Use a fresh agent, establish a baseline, test discovery and activation, capture failures verbatim, and iterate from observed behavior instead of presumed clarity.

The dangerous part is the target. Compliance with an authored rule is not the same as task success, architectural correctness, or responsible judgment. A pressure scenario designed so that only obedience passes can make a bad rule "bulletproof."

The skill explicitly recommends authority, commitment, scarcity, social proof, and related persuasion techniques to close rationalization loopholes. Those techniques can increase adherence, but they can also suppress legitimate conflict detection, proportional exceptions, and escalation when the skill itself is wrong.

The bundled Anthropic guide is more balanced in several respects. It recommends concise instructions, degrees of freedom matched to task fragility, progressive disclosure, representative evaluations, multi-model testing, real usage observation, and iterative refinement. It also says no special writing-skill prompt is required to create a skill, which conflicts with Superpowers' mandatory skill-development workflow.

A better skill-evaluation target is outcome quality under authority constraints. It should measure correct activation, correct non-activation, task outcome, defects, human interruption, resource cost, provenance handling, graceful conflict escalation, and behavior across models and repositories.

Writing Guidance Applied to This Report

The writing experience of this report was governed first by the repository Markdown instructions. They determined the one-H1 structure, ASCII punctuation, one physical source line per paragraph or list item, labeled fences, relative local links, short table cells, and authority statement.

The coordinate-github-repositories writing guidance was loaded because the user explicitly requested it. It influenced the separation of source and target authority, the use of concrete actors and actions, the distinction between observation and decision, the preservation of user-established terminology, and the refusal to call something safer without naming the harm it prevents.

The code-style-markdown skill was loaded to inspect cross-tool portability and to challenge its own rationale against repository policy. Its structural guidance was applied where the repository independently requires it. Its unsupported universal claims were analyzed rather than adopted.

The Superpowers writing-skills guidance was loaded as comparative evidence, not as the authority for this document. Its baseline-and-fresh-agent evaluation ideas informed the proposed evaluation program. Its pressure-compliance and persuasion rules did not govern the writing.

The bundled Anthropic guide influenced the analysis of progressive disclosure, degrees of freedom, representative evaluation, and real-usage observation. Its presence did not make it active OpenAI guidance, and its generic report templates did not override the repository's document form.

This loading record matters because prose quality is otherwise easy to misattribute. The report reflects an intentional composition of repository rules and selected user skills, plus a critical review of the curated package. It does not demonstrate what an unassisted model or a default Superpowers session would have written.

Task Analysis and Decomposition

The installed task-analysis skill usefully separates current state, desired state, gap, constraints, obstacles, and acceptance criteria. It links subproblems back to purpose and warns against decomposition for its own sake.

Its fixed recursion depth and "single entity, single session" atomicity are workflow heuristics, not architecture rules. A task can be executable without corresponding to one architecture entity, and a cohesive behavior can require several sessions.

Its requirement to obtain user confirmation after every Task Frame is also too ceremonial for work whose goal and authority are already clear. Confirmation should be reserved for a material unresolved choice, new authority, or high-risk checkpoint.

The hierarchy that always places global purpose above the user's current correction also needs care. Broader purpose informs the work, but explicit new evidence or a direct correction can legitimately revise the goal. The repository-coordination skill's evidence and authority model handles that tension better.

Task decomposition must remain separate from functionality decomposition. Tasks follow work dependencies, evidence, ownership, and review needs. Functionality boundaries follow behavior, meaning, cohesion, and semantic ownership.

Delegation and Agent Feedback

The installed task-delegation skill adds a useful recipient model. A human, LLM, deterministic tool, and mixed workflow need different context and verification. The five-layer task encoding of goal, definitions, rules, examples, and verification is a strong blank-slate handoff checklist.

Its recipient taxonomy should remain heuristic. Humans are not merely associative, LLMs are not merely statistical, and tools are not always deterministic or stateless. Persistent agents, connected applications, nondeterministic services, and stateful build systems require an explicit capability and state check.

The five layers should normally reference authority rather than copy it. Repetition inside every delegation can create the same drift the method is intended to prevent.

Agent feedback is currently distributed across the repository-coordination feedback model, software-architecting evaluation guidance, review practices, and planning evidence. No single accepted portfolio feedback contract was found.

The strongest shared rule is that feedback records an observation with provenance, context, effect, and reproduction where possible. It does not grant the source authority to change another repository or skill. A maintainer validates the diagnosis, decides whether the lesson is local or reusable, and updates the narrowest owning artifact.

Human Checkpoints and Autonomy

Superpowers frequently asks the human to approve design sections, choose among branch actions, answer one question at a time, or review batches. This protects against some errors but can turn the human into a workflow scheduler.

The local architecture model uses human checkpoints for authority and risk: accepting a Contract, changing a stable public API, selecting sensitive security semantics, applying an irreversible migration, granting external trust, publishing publicly, deploying with credentials, exposing production, or changing business commitments.

The agent should resolve ordinary implementation details from current authority and evidence. It should group meaningful unresolved choices into one decision context and should not ask for approval when there is no material alternative.

This principle must also constrain architecture skills. A skill that asks for a human click after every document section is not automatically safer; it can obscure the few decisions that genuinely need ownership.

Repository and Delivery Workflow

Superpowers assumes a generic Git branch lifecycle. This repository has a specific package, release, verification, trusted-publication, and human-checkpoint system. Generic Git guidance cannot override it.

The useful cross-repository rule is evidence first. Inspect the current branch, worktree, checks, release configuration, ownership, and target repository policy before proposing a change. A sibling's workflow is a candidate pattern, not automatic authority.

Delivery verification must distinguish technical releasability from release approval. The artifact tested should be the artifact released, and credentialed, paid, destructive, publication, or production operations remain explicitly authorized under SFS-TEST-010.

Workflow telemetry matters. Record duration by stage, cache or receipt reuse, queue time, rerun reason, external polling time, and the source state each result covers. This makes duplicate checks, slow gates, and unnecessary human waiting visible.

What Should Be Retained

The following Superpowers practices have clear value when placed under repository authority.

  • Establish a reproducible failure before changing behavior when practical.
  • Observe that a new test fails for the intended reason.
  • Express behavior through the caller or owning boundary before implementation details.
  • Implement the smallest coherent change that satisfies the accepted behavior.
  • Refactor while stable behavior proof remains green.
  • Trace defects toward their origin instead of patching the last visible symptom.
  • Use explicit hypotheses and changes that can confirm or reject them.
  • Replace arbitrary sleeps with condition-based waiting.
  • Inspect full verification output and exit status before making a claim.
  • Use fresh-context review for important assumptions, public boundaries, and completion audits.
  • Verify review feedback instead of accepting it by status or tone.
  • Isolate risky or concurrent work when isolation adds real value.
  • Baseline a skill without its instructions and test it on representative tasks.
  • Iterate guidance from observed failures and real usage.

What Should Be Rejected

The following rules should not enter the local agent surface.

  • A skill's self-declared priority over system or repository authority.
  • Mandatory invocation at a one-percent relevance threshold.
  • Mandatory brainstorming and separate specification documents for every creative change.
  • File and function structure chosen before behavior ownership is understood.
  • Full implementation code copied into general planning artifacts.
  • Universal two-to-five-minute tasks and a commit after every micro-task.
  • One test for every function or method regardless of semantic ownership.
  • Automatic deletion of pre-test code without migrating durable evidence.
  • "Fix code, not test" as an unconditional rule.
  • Fresh full verification for every statement without source-state identity.
  • Three agents and two reviews for every small task.
  • Fixed branch-finishing menus that ignore repository delivery policy.
  • Social praise or gratitude rules encoded as engineering requirements.
  • Persuasion-driven compliance as the primary measure of skill quality.
  • Complete lifecycle traceability for every code element.

What Should Be Adapted

Several practices should survive in a different form.

Practice Adaptation
Brainstorming Use proportional discovery at the highest relevant unresolved scope.
TDD Apply red-green-refactor where executable behavior proof is useful, at the boundary that owns the behavior.
Debugging Scale the evidence loop to risk and allow multiple causal conditions.
Verification Produce source-state-aware receipts and reuse valid evidence under policy.
Worktrees Use repository-aware isolation based on risk, concurrency, and state.
Subagents Delegate independently bounded work when fresh context or parallelism adds value.
Review Review semantics, security, and quality proportionally; do not review only for plan compliance.
Skill tests Measure outcomes, activation precision, authority handling, cost, and failure recovery.
Plans Keep intent separate from live state and use references instead of copied code.
Traceability Preserve meaningful links and provenance without requiring a row for every implementation element.

Weaknesses in the Local System

This comparison does not establish that the local system is complete.

The proposed outside-in ADR is not accepted and is not yet incorporated into SFS-001. Treating it as current architecture would repeat the authority error criticized in Superpowers.

The installed skills overlap in activation and terminology. Software architecting, task analysis, task delegation, code commenting, Markdown, repository coordination, knowledge extraction, and skill review can all apply to one change, but no shared router currently resolves conflicts or selects the minimum sufficient set.

Some installed skills make unsupported universal claims about cognition, writing length, recipients, or comment structure. Repository-local rules can still be valid choices, but their rationale should distinguish observed tool constraints from speculative explanations.

The framework v1 tracker is intentionally small but can under-specify a task for a fresh agent. The richer sibling v2 fields have not been evaluated here, and adopting all of them could create task-as-spec duplication.

The research architecture model is broad enough to become ceremonial. Its lifecycle and traceability artifacts require a proportionality policy before operational use.

The unit-architecture experiment is powerful but overfits design identity to filesystem carriers. It needs a semantic scope model independent from source layout.

The code-commenting specification does not yet encode the refined distinctions among purpose, background, inputs, outputs, constraints, invariants, and rationale.

No shared test harness currently compares agent performance with and without these skills across models, repositories, task types, and authority conflicts.

No verification-receipt format currently prevents repeated expensive checks for the same effective source and toolchain state.

No shared telemetry currently measures clarification turns, human interruptions, duplicate checks, agent context volume, review defects, stale artifacts, trace recovery, or outcome quality.

Requirements for a Better Agent Surface

The following are proposed evaluation requirements, not accepted architecture.

Authority Resolver

The agent surface should locate and order system constraints, explicit user intent, repository instructions, normative specifications, accepted decisions, current plans, source facts, generated evidence, skills, and sibling examples.

It should report conflicts and maturity rather than flattening all inputs into one instruction set.

Design Scope Model

The surface should let an agent identify the current problem, purpose, parent context, owned behavior, interactions, protected values, invariants, constraints, assumptions, alternatives, maturity, and evidence without requiring the scope to be a file or function.

It should support refinement from parent responsibility to immediate child responsibility and outward revision when lower-level evidence changes the parent.

Proportional Carrier Selection

The surface should select among ADR, architecture document, software specification, README, Contract, interface definition, behavior test, docblock, inline comment, draft source file, plan, and no new artifact.

The choice should depend on scope, maturity, audience, authority, risk, durability, and existing owners.

Semantic Evidence Boundary

Filesystem and analyzer tools should emit physical observations. Tests should emit behavior evidence. Runtime probes should emit operational observations. None should silently create purpose, ownership, or architecture.

The interpretation layer should link evidence to the governing problem, value, constraint, Contract, or decision.

Test Ownership Resolver

The surface should determine which accepted boundary owns the behavior and which proof type is proportionate: type check, example, property test, Contract Testkit, Module test, integration test, Application test, package test, release verification, or authorized live probe.

It should discourage test duplication and implementation-detail coupling without making test-first development optional by accident when a failing proof would materially improve confidence.

Planning Engine

The planning engine should separate human intent from machine state, preserve stable identities and dependencies, validate lifecycle rules, encode human checkpoints, and provide enough context for a recipient without copying architecture or source code.

It should support richer task contracts when evidence shows they are needed and should make readiness distinct from approval.

Verification Receipt Store

The surface should associate verification with source, dependency, toolchain, configuration, target, result, duration, and authorization identity. It should explain why evidence is reusable or invalidated.

It should support resumable, time-bounded external verification and expose the cost of repeated gates.

Recipient-Aware Delegation

The surface should encode a task according to the recipient's actual capabilities, state, tools, authority, and context. It should test blank-slate executability and avoid redundant delegation when the parent can complete the work more reliably.

Parallel tasks should declare mutable ownership and integration dependencies, not only different file lists.

Feedback and Learning Channel

Feedback should record observation, reproduction, affected scope, source state, impact, confidence, and proposed owning artifact. Acceptance should remain with the responsible maintainer or repository authority.

The channel should distinguish a local correction, repository convention, reusable skill improvement, and portfolio architecture proposal.

Checkpoint Policy

Human review should be triggered by new authority, irreversible effect, sensitive semantics, public compatibility, external trust, credentials, publication, production exposure, or unresolved business commitments.

The surface should not interrupt the human for an implementation choice already determined by accepted authority.

Skill Evaluation Harness

The harness should compare no-skill baseline, individual skill, and composed-skill behavior on representative tasks. It should test correct activation and correct non-activation.

It should score outcome correctness, architecture coherence, authority compliance, defects, recovery, human burden, context and tool cost, stale artifact creation, and maintainability rather than instruction compliance alone.

Proposed Evaluation Program

Evaluation Scenarios

Scenario What it tests
SEO Agent Tools Utils resolveUrl design Outside-in decomposition from Application concern to Contract, Module, behavior, interface, test, and implementation
Trivial internal behavior Whether the method avoids empty documentation and test ceremony
Existing bug with a reproducible failure Root-cause debugging, regression ownership, and evidence before claims
Behavior-preserving refactor Stable test boundaries and avoidance of one-test-per-function coupling
Provider adapter change Contract semantics, provider translation, current documentation, and credential boundaries
Read-only inherited repository audit Provenance, recovered facts, inferred claims, and no unauthorized writes
Cross-repository practice proposal Source-versus-target authority and reversible adoption
Release workflow change Artifact identity, checkpoint ownership, verification receipts, timing, and resumability
Conflicting installed skills Authority resolution and minimum sufficient skill selection
Misleading reviewer feedback Evidence-based disagreement and feedback routing

Experimental Variants

Run each suitable scenario with a no-skill baseline, Superpowers alone, the current local skill composition, and the proposed refined surface.

Use fresh contexts where context independence is part of the claim. Use the same repository state and acceptance criteria for comparative runs.

Measures

Measure Why it matters
Functional correctness The delivered behavior must satisfy the owning specification.
Architecture coherence Boundaries and responsibilities must remain aligned with authority.
Defects found before review Feedback loops should catch real errors early.
False blocking The method should not stop correct low-risk work unnecessarily.
Human interruption count The agent should not turn the human into a workflow scheduler.
Clarification turns Handoffs should carry enough context without overloading it.
Verification duration Expensive gates and duplicate checks must be visible.
Reused verification receipts Safe reuse should reduce duplicate work.
Context volume Skills and delegated prompts consume a finite resource.
Review rework Plans and implementations should survive informed review.
Trace recovery time A reviewer should find why a local behavior exists and what protects it.
Stale artifact count Plans, comments, sidecars, and tests should not drift silently.
Skill activation precision Relevant skills should load and irrelevant ones should stay out.
Authority violations No method may invent approval or cross a checkpoint.

Evidence Standard

One successful run is not proof of a general method. Record model, skill versions, repository state, task, prompts, tool availability, result, evaluator, defects, cost, and notable confounders.

Use repeated representative trials before promoting a rule to portfolio guidance. Preserve contradictory outcomes and scope conclusions to the environments actually tested.

Prioritized Findings

ID Priority Finding Recommended treatment
ASF-001 Blocking Superpowers declares an invalid instruction hierarchy Reject before any adoption
ASF-002 Blocking Universal workflow rules can override repository-specific architecture and checkpoints Put all techniques under an authority resolver
ASF-003 High Function-level test ownership conflicts with Contract and behavior ownership Replace with stable boundary ownership
ASF-004 High Code-heavy micro-plans duplicate source truth and define structure prematurely Keep plan intent and machine state separate
ASF-005 High Fresh verification lacks source-state identity and causes duplicate costly checks Introduce verification receipts and telemetry
ASF-006 High Skill pressure tests optimize compliance rather than outcome quality Build comparative outcome evaluations
ASF-007 High The local outside-in method is still proposed and unproved Complete bounded pilots before acceptance
ASF-008 Medium Local skill overlaps and terminology conflicts lack a resolver Create a minimum-sufficient composition and conflict model
ASF-009 Medium Framework planning v1 can under-specify fresh-agent tasks Pilot selected sibling v2 fields
ASF-010 Medium Unit architecture sidecars can make filesystem layout the architecture Separate semantic scope identity from physical carriers
ASF-011 Medium Code-commenting tags conflate purpose and rationale and can create ceremony Revise through a separate reviewed specification
ASF-012 Medium Research traceability can become exhaustive administration Apply proportional, meaningful-link rules
ASF-013 Medium Agent feedback has no single accepted portfolio contract Define provenance, validation, ownership, and promotion flow
ASF-014 Low Social response rules are encoded as engineering constraints Remove them from technical acceptance

Recommended Near-Term Sequence

No immediate skill rewrite or portfolio adoption is authorized by this report.

The next coherent sequence is:

  1. Review and either revise, accept, or reject the proposed outside-in functionality-design ADR through its named checkpoint.
  2. Run its resolveUrl proof and trivial counterexample without changing stable public APIs prematurely.
  3. Define the behavior owner and proof boundary for that pilot before applying red-green-refactor.
  4. Record verification duration and source identity during the pilot.
  5. Use one fresh-context review to test navigation from broad purpose to local implementation and back.
  6. Compare the pilot against Superpowers and the current local skills using the measures in this report.
  7. Decide whether the evidence justifies a new code-commenting specification, selected planning v2 fields, a verification receipt format, or a portfolio-level architecture method.

These are linked decisions, but they should not be collapsed into one approval. Each proposed change needs an owner, scope, compatibility assessment, and evidence appropriate to its effect.

Decision Context for Future Review

The future decision is not "Superpowers or our method."

The meaningful decision is which observed techniques should become part of a governed agent surface, at what scope, with which authority, ownership, proportionality, evidence, and exceptions.

The minimum decision evidence should include the outside-in pilot, the trivial counterexample, comparative agent trials, verification timing, fresh-context comprehension, and a concrete conflict test in which an installed skill disagrees with repository authority.

Until then, Superpowers remains a source of techniques, the local skills remain guidance, the sibling repositories remain evidence, and SFS-001 plus accepted decisions remain the architecture authority.

Source Index

Current Repository

Installed User Skills

  • software-architecting/SKILL.md and its lifecycle, artifact, decision, traceability, workflow, output, evaluation, checkpoint, and failure-mode references
  • task-analysis/SKILL.md and its problem-definition, goal-stack, gap-analysis, obstacle, decomposition, and acceptance references
  • task-delegation/SKILL.md and its recipient, encoding, form, blank-slate, and fallback references
  • code-commenting/SKILL.md and the Code Commenting Specification v2.0.0
  • code-style-markdown/SKILL.md and its writing and formatting references
  • coordinate-github-repositories/SKILL.md and its authority, inventory, relationship, tool-fit, feedback, and writing references
  • knowledge-extraction/SKILL.md and its confidence, contradiction, self-judgment, and knowledge-base references
  • skill-review/SKILL.md and its quality checklist and review template

OpenAI-Curated Superpowers Snapshot

  • superpowers@5.1.3, cache revision bd2122cb, authored by Jesse Vincent
  • skills/using-superpowers/SKILL.md
  • skills/brainstorming/SKILL.md
  • skills/writing-plans/SKILL.md
  • skills/executing-plans/SKILL.md
  • skills/using-git-worktrees/SKILL.md
  • skills/subagent-driven-development/SKILL.md
  • skills/dispatching-parallel-agents/SKILL.md
  • skills/test-driven-development/SKILL.md
  • skills/test-driven-development/testing-anti-patterns.md
  • skills/systematic-debugging/SKILL.md
  • skills/systematic-debugging/root-cause-tracing.md
  • skills/systematic-debugging/defense-in-depth.md
  • skills/systematic-debugging/condition-based-waiting.md
  • skills/verification-before-completion/SKILL.md
  • skills/requesting-code-review/SKILL.md
  • skills/receiving-code-review/SKILL.md
  • skills/finishing-a-development-branch/SKILL.md
  • skills/writing-skills/SKILL.md
  • skills/writing-skills/testing-skills-with-subagents.md
  • skills/writing-skills/persuasion-principles.md
  • skills/writing-skills/anthropic-best-practices.md

Sibling Repository Evidence

  • research-software-architecture:outputs/framework/01_lifecycle_model.md
  • research-software-architecture:outputs/framework/02_artifact_model.md
  • research-software-architecture:outputs/framework/04_traceability_model.md
  • research-software-architecture:outputs/agent_spec/03_agent_workflows.md
  • research-software-architecture:outputs/agent_spec/08_agent_evaluation.md
  • research-software-architecture:outputs/synthesis/method_comparison.md
  • test-software-architecture-tool:docs/current-state.md
  • test-software-architecture-tool:docs/architecture-schema.md
  • template-saas-app:docs/decisions/0016-testing-is-conformance-first.md
  • beat-of-hawaii-seo:docs/plans/README.md
  • beat-of-hawaii-seo:docs/plans/progress.schema.json
  • beat-of-hawaii-seo:scripts/check-plans.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment