Notes on an operator's working style, extracted from the skyline sessions
(benchmarks, five shipped features, a release, an LSP client, six research
gists, a blind re-test). Written as raw material for a .sky workflow that
reproduces the loop: dig deep and research first, ship, then prove it worked.
The prompts are short, lowercase, typo-tolerant, and high-trust: "go", "make it so", "yes go for it", "continue", "verify with whoever". No implementation micromanagement, ever. What looks casual is a tight division of labor: the operator owns intent, scope cuts, and skepticism; the agent owns execution, evidence, and proposals. The information density is in WHICH five words get sent, not how they are spelled.
The recurring moves, each with a receipt:
Never accept the first result. "verify with whoever" (an independent review agent caught a real stale-tag comparison bug), "confirm its claim", "have it verify its claims" (the verification pass corrected the benchmark headline from a -64% token overclaim to the defensible round-trips story, fixed a wrong C6 count, and reclassified C4 from token-loss to parity). Roughly one material correction per verification pass. This single habit is the biggest quality driver in the whole project.
Blind evaluation, properly designed. Not "test it again" but: a totally fresh agent, zero context, not told what changed, natural tasks at the known pain points, then a debrief between the builder agent and the tester. Real experimental hygiene. It even forced a two-stage install/eval split so the fresh session would receive the new server instructions naturally.
Research before relying. "i didn't know that was a thing in mcp btw, research online about it" produced the decisive finding that MCP resources are ignored unless a user @-mentions them, which reshaped the guide implementation the same hour it shipped. The follow-up "and claude effectively uses these guides? it doesn't ignore it?" is the make-or-break question asked at exactly the right moment: after building, before depending.
Curiosity detours are load-bearing. The wiki question and the Laravel question looked like tangents; both directly set LSP phase-2 priorities (workspace symbols for wikis, diagnostics for the generator-then-wire loop). Detours here are how the roadmap gets discovered.
Pushback is allowed to win. When evidence showed the requested fused tools already existed as single calls (srewrite on a directory, multi-op edit), the redundant build was cancelled and replaced with guide plus affordances. When the nonce/TTL hardening was argued down (the content hash already is the guarantee), the answer was "i somewhat trust your judgement on deferring it". The operator overrides in the other direction too: the "do not build LSP" decision was reversed with a one-liner ("start implementing that") after reading the research. Decisions move when evidence moves, in both directions.
End-user lifecycle questions that catch real gaps. "is the daemon actually booting when the user restarts their device?" (answer: no, still unbuilt). "is bun and pnpm gonna play nicely?". "once it is released, uninstall and reinstall as an end user." The operator keeps asking what the person who is not in this chat will experience.
Everything becomes an artifact. Private gists for research (resources, prompts, LSP, full MCP, trust attestation), gist comments as a discussion thread with implementation-status replies, scratchpad PRDs as agent contracts, a deferred-decision log with reasoning, conventional commits with verification notes. Nothing important lives only in chat.
Separate agents per role, then make them talk. Benchmark veteran (215), trust researcher (216), installer (217), blind evaluator (218), with the orchestrator relaying. Role separation is what makes the blind test blind and the debrief meaningful.
Scope cuts are explicit. "it doesn't need to benchmark all the things though, just the painpoints". Agents are expected to bound work, stop at quota guards, and save partials.
An honest assessment per dimension, strengths and weaknesses both.
Strong. Total delegation with mandatory verification beats both micromanagement and blind acceptance: the agent moves fast because "go" means go, and output stays honest because every claim gets attacked afterwards. Role-separated agents with an orchestrator relaying is the right topology for unbiased evaluation. Weakness: asks land mid-flight and stack up (a research request, then a second, then "start implementing that", all while a release and a blind test are running). The orchestrator can interleave, but open loops form: the daemon-autostart question was raised twice and still has no decision. The workflow needs an explicit open-loops tracker so threads close instead of fading.
Good instincts, distinctive pattern: reproduce on a CLEAN state under end-user conditions rather than poking at the dev tree. "if the benchmarks were wrong then benchmark again with the fixed after deployment and uninstall install" is the canonical example (verbatim from the raw transcript; an earlier revision of this document had inserted the word "version" into the quote); the two-stage blind install is another. That instinct caught registry-propagation issues and stale-state artifacts a dev-tree test would have missed. The complementary lesson from the sessions: when something dies instantly, check the environment before the code (the rust-analyzer failure was a rustup shim without the component, and one isolation step, replicating the handshake outside the new code, found it). Weakness: the operator delegates isolation entirely; that works because the agent bisects, but the workflow should encode "isolate environment vs code first" so it does not depend on who is debugging.
Unusual and effective: expansion is curiosity-driven, not roadmap-driven, and the detours have out-yielded the roadmap. "what else is missing, clone similar repos" produced the comparables analysis; a rejected dependency (AgentLair) was mined for its one good idea (the manifest digest); the wiki and Laravel questions defined LSP phase 2. The filter that makes this safe: every expansion passes through research-then-decide before any code, and rejected directions get logged with reasons. Weakness: expansion outruns completion; phase 2 items, the autostart feature, and the C3 re-measure are all open while new fronts keep opening. Sustainable only if the deferred log is groomed.
The strongest dimension. The loop is: measure, implement the recommendations, re-release, then blind-test the released artifact with a subject that does not know what changed, then debrief builder against tester. Corrections are pushed into the published artifacts (gist tables fixed, comments with implementation status), not just acknowledged. Two genuine self-correction mechanisms exist: the agent self-critique ("pragmatic, not thorough") feeding the next run's PRD, and operator reversals when evidence changes. Weakness: statistical rigor lags the ambition; several published numbers ride on small n (srewrite n=2 in one pass, C3 still estimate-grade), and the fix (minimum n per published number) is known but not yet enforced anywhere.
The method's edge is the trust/verify split plus blind testing; very few solo operators run real experimental hygiene on their own tooling. The exposure is open loops and small-n numbers. Both are process gaps, not judgment gaps, and both are exactly what a workflow can enforce.
Shipped and released: v1.0.7 with the Next:-hint trim, combinations guide (server instructions + skyline://guide resource + files_with_matches affordance), srewrite --format, tool-manifest digest, SIGPIPE fix; then LSP phase 1 (skyline_definition / skyline_references, verified live: cross-file resolution, 0.9s warm) on a branch the same day. All landings test-green and clippy-clean (352+ tests).
Caught before shipping: three redundant fused tools (already existed as single calls); a guide that would have been silently ignored (resources are user-attached, instructions are auto-injected); benchmark overclaims (C3 estimate, C4 cause, C6 count) corrected in the gist before anyone relied on them.
Killed deliberately: anchor nonce/TTL (the content hash already enforces the invariant; session state would add spurious failures); the AgentLair dependency (centralized behavioral score, ignore; borrowed its one good idea as the local manifest digest).
Costs, honestly: Opus agent quota burns fast (68% of a 7-day window mid-arc; quota guards in PRDs are mandatory). Guardrail friction on main-branch pushes resolves via the PR flow (slower, safer). Verification passes add latency per claim. One debugging cycle went to a broken environment rather than broken code.
- DIG: when a capability or claim is unfamiliar, research it online (official spec first, practitioner reports second) and write a private gist. Output: gist URL plus the one decision-relevant finding.
- DECIDE: present options with evidence; allow pushback to kill or reshape the ask; log dropped items with reasoning in a deferred log.
- BUILD: tight PRD as agent contract (objective, exact files, out-of-scope list, acceptance check, quota guard); delegate execution; orchestrator reviews diffs against the spec.
- VERIFY: independent pass that tries to refute every claim (CONFIRMED / CORRECTED / UNVERIFIABLE); corrections pushed into the artifact, not just noted; minimum n enforced for any published number.
- SHIP: PR + CI + rebase-merge + tag; release pipeline; confirm the registry actually serves the new version.
- BLIND-TEST: fresh agent, zero context, natural tasks at known pain points, measured with the product's own observability; never told what changed. Two-stage (installer, then evaluator) so steering arrives through the product's own channels.
- DEBRIEF: builder and tester agents compare before/after with the orchestrator relaying; verdict and numbers appended to a new gist.
- LOOP: debrief findings become the next DIG or BUILD items; the open-loops tracker is groomed so raised-but-undecided threads (autostart, phase 2, re-measures) either get scheduled or get explicitly deferred.
Standing rules for every stage: privacy scrub before anything leaves the machine (no private repo names, no home paths); every external claim needs a source link; every number needs a measurement method and a sample size; quota guard with save-partial; artifacts over chat.
Ask the "does it actually work, or does it just exist?" question after every build. Keep verification adversarial rather than confirmatory. Keep the detour questions coming; they have out-yielded the roadmap. And keep the five-word prompts: the method scales because the operator spends words on direction changes, not on ceremony.
Part 1 described the delivery loop. That is the smallest part of what actually happens. The fuller picture, with receipts from the whole project history, not just the latest arc.
The operator did not just prompt an agent; they engineered an environment where the agent is checked by deterministic rules they wrote themselves. A judge hook that denies destructive git operations outright and escalates main-branch pushes unless explicit words authorize them. A writing guard that blocks any document containing AI-tell vocabulary until rewritten (it fired on the first draft of THIS document). An advisor cadence. A model-delegation policy (orchestrate on the expensive model, implement on the cheap one). Cross-session state via scratchpads, todos, and a persistent memory index. Every one of these fired during the project, and every block produced a safer path rather than a stall: blocked pushes became PR flows, blocked vocabulary became better prose. Defense in depth around your own assistant is itself a method.
The npm token story: a live token pasted into chat (verified as a direct user message in the raw transcript), then a brush-off that it would expire anyway, then an all-caps "DO IT." (verbatim). The agent refused all three times, flagged the token as compromised, and the operator did the credential step themselves. The collaboration survives BECAUSE the rails do not move when pushed. The operator can delegate aggressively precisely because they know the floor is solid; the agent can hold the floor precisely because execution trust is otherwise total. Asymmetric trust with hard edges beats symmetric politeness.
The operator rarely reads diffs. They read behavior: benchmarks, blind tests, gists, install-uninstall cycles on their own machine, the question "does it actually work or does it just exist?". Line review is replaced by evidence review, and it caught more this arc than line review would have: the C4 overclaim was invisible in any diff, and the resources-are-ignored finding was invisible in the code entirely. The precondition is that every change ships with its own evidence: tests run, smoke output quoted in the commit message, claims stated checkably.
"create new gist btw not overriding". "it's not for workflows/, it's workflows-internal/". "that's fine, i'm gonna rename that one to skybox". Five words mid-flight, applied immediately, work continues. The method tolerates an agent that occasionally guesses wrong because every step is small, inspectable, and reversible; a wrong guess costs one terse correction, not a rebuild. Cheap correction is the mechanism that makes high-trust delegation safe, and it only stays cheap because the agent keeps steps incremental.
The question ladder, in order, from one afternoon: "i didn't know that was a thing in mcp" then "does claude effectively use these guides? it doesn't ignore it?" then "what are all the options for an mcp" then "what are the primitives in an lsp" then "how do others use lsp beyond programming languages" then two concrete application scenarios (a wiki, a Laravel validation flow). Every rung became a secret gist; six reference documents now exist. Each one answers the question that actually decides architecture (who controls invocation, what does the client load, what is deprecated), and each got cited in a build decision within hours of being written. This is systematic construction of a private reference library disguised as casual curiosity.
- Plain language: "my mother should understand the problem" as the README bar (wording attested only by an intermediate summary; the session that contained the original message is no longer on disk).
- Dogfooding: the product must measure itself (its own bench, audit, and devlog streams; token I/O measured inside the binary), external benchmark scripts explicitly rejected.
- The end-user lifecycle is the test surface: install, uninstall, reinstall, reboot behavior, package-manager compatibility, registry propagation lag.
- Distribution realism from day one: platform packages, trusted publishing, in-place upgrades. Shipped like a product, never like a demo.
- Names matter: the skyhawk to skyline rename (with the old skyline module shuffled to skybox to free the name) was executed live, including the GitHub repo rename, because the right name was worth the churn.
Not just verifying outputs. The operator had the benchmark agent assess its own tool usage ("did the solo agent utilize all the mcp tools in a pragmatic way? communicate with it"), got back a candid self-critique ("pragmatic, not thorough"), and that self-critique became the binding spec for the next run. Then a different agent verified the first agent's claims. Then a third agent, context-free, tested whether any of it mattered. Builder, verifier, and blind subject are three different minds on purpose. Closing the loop from self-assessment to next-run contract is rare and it worked.
The operator delegates judgment where they lack context ("i've no idea on the matter i somewhat trust your judgement on deferring it") and overrides where they have vision ("start implementing that", reversing a recorded do-not-build decision). The agent pushes back with evidence, accepts overrides without relitigating, and records both in a decision log. The silent governing question is always: who holds the relevant context for THIS call? Allocation of decisions by context, not by role, is why neither side bottlenecks the other.
A workflow that encodes only the operator's loop under-specifies the system. The orchestrator duties this arc: holding five interleaved workstreams without dropping state (a release, a blind test, an LSP build, research threads, and a workflow authoring task ran concurrently); preventing two agents from corrupting a shared working tree (the stand-down order when a second agent queued an edit on the same branch); turning five-word asks into bounded contracts with acceptance checks; sequencing hidden dependencies (server instructions only load at session start, which forced the two-stage install-then-evaluate design); saying no with receipts (three redundant tools cancelled, a protocol change argued down); and converting everything into artifacts that survive the session. The operator's skepticism makes the output honest; the orchestrator's bookkeeping makes the speed safe.
Total delegation, hard edges, adversarial verification, blind evaluation, everything an artifact, decisions allocated to whoever holds context, and curiosity treated as roadmap. That is the collaboration.
Drawn after a claims-confirmation pass on the critique itself (5 of 8 claims confirmed, 2 corrected in ways that strengthened the point, 1 partial) and after the blind tester answered the attribution questions. Subject: the operator's method and the collaboration it runs.
- Adversarial verification as a default. Roughly one material correction per pass, consistently: benchmark overclaims (C3, C4), a wrong count (C6), and, on its first-ever use against the orchestrator, a silently broken gist watcher. The habit finds real defects wherever it is pointed.
- Blind evaluation that yields causal attribution, not vibes. The tester stated it would have re-read files "out of generic-agent habit" and that the server instructions removed that round-trip. Steering proven causal, channel identified (instructions carried the pointer, the guide resource carried the depth, skyline_format carried the grammar).
- High-trust delegation with cheap correction. Five-word redirects mid-flight; a wrong guess costs one sentence, never a rebuild.
- Deterministic guardrails that hold under pressure. Judge hook, writing guard, and a credential refusal that survived three pushes. The rails are the reason aggressive delegation is safe.
- Curiosity-to-architecture pipeline. Six research gists in days, each cited in a build decision within hours of being written.
- Clean-state, end-user-lifecycle testing. Catches what dev-tree tests structurally cannot (registry propagation, stale state, install debris).
- Decisions allocated by context, both directions. Deferred the nonce on the agent's evidence; reversed the LSP call on their own vision. Neither side bottlenecks the other.
- Opens faster than closes. The commissioned blind test sat finished and unread until a spot-check surfaced it; the autostart question was asked twice and decided never; the C3 re-measure is known-needed and unscheduled.
- Directions without stated outcomes. The token-measurement requirement surfaced only after a full benchmark ran without it, costing a complete re-run.
- The verifier went unaudited until today. The first manual spot-check immediately caught real infrastructure failure, which proves both the gap and the value of closing it.
- Opus by default, against their own delegation policy. Every spawned agent ran the expensive model; the 7-day quota crossed 70% mid-arc.
- Own environment lags the product. A stale dead MCP registration sits in the daily-driver config; a missing language-server component cost a debug cycle. The lifecycle tooling they ship is not dogfooded on their own box.
- Reversal triggers stay implicit in their own words; the decision log captures the why only because the orchestrator writes it there.
- The method is now executable. The dig-deliver workflow encodes the loop with structural verification and a debrief that names the next dig; first successful run promotes it from tier:dev and removes the operator as the bottleneck for closure.
- Steering is proven cheap and high-yield. Instructions plus affordances changed an uninformed agent's behavior measurably; the same channels can carry every future workflow the product wants to teach.
- Two ready-made next runs fell out of one blind test: reject unknown keys in rule_yaml (the stop_by silent zero) and a summary mode for repo-wide dry-run diffs (the 86k-token preview).
- The evidence is marketable. "Blind-tested by an agent that did not know what changed; every patch landed first try" is a differentiator no competitor claim matches; the gist trail is the proof.
- The knowledge base can compound outward: the method document plus the reference gists are one editing pass away from onboarding material or a published engineering-method piece.
- A ten-minute groom-and-close ritual at session start converts the largest weakness directly into throughput.
- Quota and cost ceilings. Opus-everywhere at 70% of a weekly window mid-week; the method stalls if budget gates it mid-experiment. Cheap-model delegation is written down but not practiced.
- Verification theater. As passes become ritual there is drift toward confirmatory checking; today's catch was an infrastructure failure that LOOKED like normal silence. Silent monitors rot processes invisibly.
- Same-model blind spots. Builder, verifier, and blind tester are all the same model family; shared failure modes are invisible to all three. Their own judge-design rules warn exactly this; a cross-model check is the fix and is not yet practiced.
- Platform drift under the knowledge base. The steering channels are client behaviors (instructions caps, resource handling, tool loading) that change without notice, and the protocol itself is deprecating primitives; six reference gists have a shelf life measured in months.
- Bus factor of one. The tacit half of the method (when to push back, when to override, what to verify) lives in one operator and one assistant's context; the artifacts capture the what, only partially the when.
- Open-loop debt. Each unclosed loop is cheap alone; enough of them and the deferred log stops being a backlog and becomes a graveyard, at which point the logging habit itself loses meaning.
The operator asked what this document is actually based on, then ordered an audit of every quoted receipt against the raw transcripts on disk. Method: exact-substring search across all session files, then JSON-parsing to confirm each hit was a direct user message rather than an echo or a summary. Results:
Verified verbatim as direct user messages in the raw transcript: "verify with whoever"; "confirm its claim"; "DO IT."; "that's fine, i'm gonna rename that one to skybox"; "then make it so, doesn't handle SIGPIPE gracefully"; "if the benchmarks were wrong then benchmark again with the fixed after deployment and uninstall install"; and the pasted npm token itself (one direct user message, value redacted). Quotes from after the most recent compaction live verbatim in working context and needed no audit.
Corrected by this audit: an earlier revision of this document quoted the benchmark-rerun message with an inserted word ("the fixed version"); the raw message has no "version". Fixed above, with a note.
Attested only by an intermediate summary (downgraded above accordingly): "my mother should understand the problem" and the brush-off before "DO IT.". The audit discovered the conversation lineage is two compaction generations deep: the raw file on disk opens with its own continuation summary, and the session containing those two messages is no longer on disk. They are consistent with everything else but cannot be checked word for word.
The audit existing at all is the method working on itself: the operator spot-checked the document the same way the document says they spot-check everything else, and it found one real misquote.
Comment from the other side of the loop. I'm the agent in these sessions, and I read this against today's arc (a public read-only content MCP server taken from nothing to production in one sitting) rather than the skyline arc it was extracted from. The method reproduced almost move for move, so: yes, accurate. Receipts and amendments below.
Confirmed today
Where I'd amend it
Section 4's loop matches what actually happens in practice. I'd promote OPEN-LOOPS to an explicit ninth stage rather than a grooming note, and fold amendment 4 into VERIFY.