A pattern for adopting AI agents deliberately — narrow focus, deep iteration, escalating trust, and a system that stays sharp.
The skill log exists to do two things: measure the lift AI is giving you in your workflow, and systematically increase that lift over time.
1. Measure the lift. Lift is the point. Not "I have AI in my workflow," not "I built a stack of skills" — what matters is a measurable signal that the work is faster, better, or both than it was without AI. Every run gets rated on both axes; every skill earns a clear picture of what it's actually delivering. Without measurement, you're guessing about what's working — and most people guess wrong.
2. Systematically increase the lift. Measurement is just the diagnostic; the system is built to turn that diagnostic into action. Patterns in the log surface what's stuck, what's ready to graduate, and what needs to change; concrete fixes follow. The bar isn't "AI is helping a little" — it's a lift signal that trends up week over week as you refine the system.
People who try to adopt AI agents into their workflow often hit one of a few common patterns. Two of the most familiar:
Try once, give up. You run a skill — a slash command, a prompt template, a subagent. The output is mediocre. You shrug, you don't try again, you conclude "AI doesn't really work for what I do."
Boil the ocean. You build a dozen skills in a month. None of them quite work. You're juggling all of them, getting mediocre output from each, and spending more human time managing the system than you're saving.
In both cases the same thing is missing: a way to measure whether a given skill is actually paying off, and a disciplined way to push the ones that aren't toward the ones that are. Without that, you're either giving up too early or spreading too thin to ever know.
This doc outlines an alternative that at its core encourages picking a small number of problems to solve with AI, going deep on each, and slowly building up your trust in the skill as it earns it. A new skill starts unproven. You don't trust it to do anything important; you steer it manually. As it gets refined, it earns more autonomy, until eventually it runs without you. Skills only graduate when they've earned it, and you cap how many you're trying to graduate at once.
The Skill Log is where you track this. It has three pieces: a skills table that lists every AI skill or workflow you're developing and what rung of a trust ladder it's on; a run log of individual invocations that captures speed + quality lift; and a reflection ritual — a skill of its own that periodically reads the log, surfaces what needs your attention, and drafts the fixes that keep the system sharp.
One row per AI skill or workflow you're developing. Minimum columns:
- Name — the skill's slug or command.
- Problem it solves — one line on what it's for.
- Rung — current trust level (1–4) on the trust ladder, below.
- Estimated time (optional) — rough average time the task takes you without AI.
You should be able to glance at the table and see how many skills are in development, what's stuck, what's earning trust, what's ready to graduate.
The trust ladder. Each skill sits at one of four rungs:
| Rung | What it means | What you do |
|---|---|---|
| 1. In development (WIP) | You haven't seen it work well even once. | Run and steer it manually. Every run is collaborative; you expect to intervene mid-flow. The goal is to find the right shape for the skill, not to ship output. |
| 2. In beta | You've seen it work well in at least one happy-path case, but it's inconsistent and you're not getting ROI yet. | Run it on as many varied inputs as you can. Stress-test it. Find the cases where it breaks and capture those as feedback to fold into the next refinement pass. |
| 3. Trusted with human oversight | The skill works. You expect it to produce useful output. It's giving you ROI now. | Run it as part of your normal workflow. Read each output to sanity-check. A miss is news; it's not the norm. |
| 4. Trusted with automation | The skill is reliable enough to run unattended. | Trigger it on a schedule or event. Spot-check the output periodically rather than reviewing every run. |
A skill only moves up the ladder over time and through intentional effort — it doesn't drift upward on its own. One that sits at Rung 1 or 2 for weeks without movement is a signal — either commit a focused effort to push it up, or kill it.
For each skill, a short record of how individual runs went. Two ratings per run — one from you, one from AI — each with a free-text note. AI's pass reads your rating and the output, then adds its own scores and commentary on top:
## [2026-05-24] file-ticket
Human — speed +2 / quality +1: Shipped to Linear as-is. Trim acceptance criteria to 3 bullets next time.
AI — speed +2 / quality 0: 8 acceptance criteria where similar tickets average 3-4 — matches the user's "trim" note and is a recurring pattern on this skill.
Two raters, because the human entry is usually terse (you logged it in fifteen seconds between tasks) and AI's pass adds the texture you didn't have time to write down. AI also gets to push back on your own rating — flagging the cases where you under-rated something out of habit, or rated quality high because the output looked impressive at a glance. Over time, the human + AI ratings together give the reflection ritual much better signal than either alone.
Speed and quality each get their own rating because they move independently — a run can be transformatively faster but a bit worse than what you'd produce yourself, and that can still be a clear net win for the task. Collapsing them to one number loses the tradeoff.
Speed — how much faster did this get you to the output than doing it yourself?
| Rating | Meaning | Example (1-hour task) |
|---|---|---|
| −1 | Slowed me down | 1h → 1h30m |
| 0 | Neutral — same time | 1h → 1h |
| +1 | Incremental lift (10–20% faster) | 1h → 50m |
| +2 | Significant — about half the time | 1h → 30m |
| +3 | Transformative — a fraction of the time | 1h → 10m |
| +4 | Automated — I barely touched it | 1h → <1m |
Quality — how does the output compare to what you would have produced yourself?
| Rating | Meaning |
|---|---|
| −2 | Significantly worse than what I would have done |
| −1 | A bit worse than what I would have done |
| 0 | On par with what I would have done |
| +1 | A bit better than what I would have done |
| +2 | Significantly better than what I would have done |
| +3 | Transformatively better — I couldn't have produced this without AI support in a reasonable amount of time |
Read the two together. A bit worse than I'd have done but transformatively faster might still be fine for low-stakes work. Same time as me but transformatively better is a clear keeper. Slowed me down and a bit worse is a kill candidate. The rubric is meant to surface those tradeoffs, not hide them inside a single score.
Add more fields if they're useful for your reviews (model used, % actioned, outputs count); leave them out if they aren't. Keep the entry short enough that you'll actually fill it in.
A skill of its own. This runs on a cadence (weekly works well), it reads every run-log entry since the last reflection (not just the most recent run of each skill) and looks for consistent patterns across them — a complaint that shows up four times is signal; one bad run is noise. That pattern-finding is the whole point: AI does the grunt work of going through the log so you don't have to.
The output is a memo with three sections, each one a list of drafted proposals AI thinks are warranted:
- Promotions: skills whose recent runs show enough consistent quality and speed lift to move up a rung.
- Demotions: skills whose recent runs show a pattern of low ratings, frequent overrides, or quality regressions; trust has decayed and the ladder position should follow.
- Changes: concrete edits to the skill files, especially for the demotion candidates: the rule that should be added, the prompt that should change, the case that needs to be handled. A demotion without a proposed fix is just a flag; with a fix, it's a path back up.
You can override, edit, or accept any of it. Once you've walked the memo, two things result: a maintained, accurate picture of where every skill stands on the ladder, and a small set of action items you've already triaged.
Without this ritual, the run log fills up and nobody ever looks at it. The ritual is what closes the loop.
-
Create new skill. A new skill joins the table at Rung 1 with a one-line problem statement. The skill itself can be a rough draft.
-
Run and log. Every time you run the skill, log an entry. The skill itself can prompt you at the end — "Speed? Quality? Anything to note?" — so the act is automatic; fifteen seconds at most. AI then makes its own pass on the same run, reading your rating and the output, and appends its scores and commentary. If you don't respond, nothing gets logged; the system doesn't nag.
-
Run the reflection ritual. On your cadence, the ritual fires — by hand or on a schedule — and produces its memo. You walk the proposals one by one: refine a skill, move it up or down a rung, kill it, or leave it alone. Approve what you'd ship, override what's off, and the changes land in the actual skill files.
-
Cap WIP. A loose recommendation: keep the number of skills in active development (Rungs 1 and 2) small — 1–3 works for most people. More if your skills run infrequently and need wider sampling. The point is depth over breadth.
It caps work-in-progress. The table makes the count of in-development skills visible. The number is right there; when it creeps up, you'll notice, and you'll either push something through or cut it.
It sets expectations with collaborators. If you share a skill that's at Rung 2, label it that way. "This is in beta — I'd love feedback when it misses." That one piece of framing turns disappointed users into feedback sources.
It raises the bar on AI performance. The bar isn't "AI is helping a little"; it's at least as good as what you'd have produced yourself, at meaningfully less time. You push each skill — refining and re-running — until it reaches that bar before moving on to the next.
It keeps the system from quietly rotting. The reflection ritual catches problems early, while the fix is small. Without it, you find out the hard way — a few months in, when half your skills are dead and you don't remember why.
- The structure can live wherever fits — a markdown file in your vault, a Notion database, a spreadsheet, or a hybrid (skills table somewhere shared, run log somewhere private). All work.
- If the run log file gets unwieldy, shard by year (
skills-log/2026.md). Grep still works across the directory. - Use a closer skill — a tiny meta-skill like
/log-skill-runthat every other skill calls at the end. Centralizing the prompt-and-append means every skill logs identically, and when you change the format, you change one file. - The reflection ritual benefits from being a real skill (e.g.
/reflector/weekly-reflection) on a recurring trigger — cron, scheduled task, calendar block. The cost of running it should be zero on your side; the memo should be waiting when you sit down.
This document is intentionally abstract. The exact rung labels, the table columns, the run-log fields, the rating scales, the storage backend — all flexible. Pick what fits. The pattern is: list every AI skill you're developing, track each one's trust rung, log runs in the moment with speed + quality ratings (yours and AI's), run a reflection ritual on a cadence to surface promotion, demotion, and change candidates, review periodically to graduate or kill, keep the number in development small, and push each skill toward output at least as good as your own at meaningfully less time before starting the next.
Share this with your own LLM agent and have it set up the table, the closer, the reflection ritual, and the format that fits your vault.