Skip to content

Instantly share code, notes, and snippets.

@headinthebox
Created August 21, 2026 03:34
Show Gist options
  • Select an option

  • Save headinthebox/347d17132c4096419d9fc08ca10d4d2b to your computer and use it in GitHub Desktop.

Select an option

Save headinthebox/347d17132c4096419d9fc08ca10d4d2b to your computer and use it in GitHub Desktop.
Do explicit style rules fix Claude's prose? A before/after measurement over 67k sentences of Claude Code transcripts

Do explicit style rules fix Claude's prose? A before/after measurement.

Context: anthropics/claude-code#77136

I added seven explicit style rules to my project instructions on 2026-08-13, then measured my own Claude Code session logs before and after.

Corpus. 183 Claude Code transcript files (the JSONL session logs), 2026-06-19 to 2026-08-21. Main-session assistant text blocks only — no subagent output, no thinking blocks. Code fences, tables, headings and bullets stripped before counting, because they are not prose and counting them produces garbage. 67,023 sentences, 1,462,457 words.

The rules. Sentences under 30 words. Data, not adjectives. No vague quantifiers. Cut any sentence that does not change a decision. Answer directly (yes / no / a number / "I don't know"). Subject, verb, object — name the actor. Define jargon on first use.


1. The effect

Rules landed 2026-08-13. A linter that reported compliance landed 2026-08-17.

period (by message timestamp) sentences mean w/s p90 over 30w named filler /1k scaffolding /1k
Jun 19-30 8,233 22.2 41 23% 0.33 0.20
Jul 1-31 33,611 21.5 41 22% 0.69 0.30
Aug 1-12 (PRE) 17,812 22.4 43 25% 0.87 0.37
Aug 13-21 (POST) 7,367 20.7 38 20% 0.63 0.38

Against the window immediately before: mean -7.6%, p90 -12%, share of sentences over the stated 30-word rule 25% -> 20%.

2. The rules moved what they named, and nothing else

Per term, raw regex over each period's full text, per 1,000 words.

NAMED in the rules (policed) pre post change
load-bearing 0.217 0.240 +11%
by construction 0.232 0.149 -36%
reads as 0.124 0.057 -54%
one level up/in 0.057 0.023 -60%
the shape 0.177 0.166 -6%
the tell 0.029 0.023 -22%
family total 0.836 0.658 -21%
UNNAMED scaffolding (not policed) pre post change
worth naming 0.044 0.069 +55%
worth saying 0.021 0.040 +90%
the crux 0.006 0.023 +262%
which is exactly 0.173 0.137 -20%
that is exactly 0.017 0.029 +70%
the whole point 0.019 0.011 -40%
worth keeping 0.072 0.069 -4%
family total 0.352 0.378 +7%

The policed family fell 21%. The unpoliced family rose 7%. load-bearing is on my own ban list and rose 11%.

Also measured, the "leads with what something isn't" construction the issue describes (X is not Y, it's Z): 0.194 -> 0.137 per 1k, -29%.

3. Two traps, both of which cost me a wrong answer

Do not gate the mean. A mean sentence length that blocks a commit is gamed by splitting sentences at commas. That scores better and reads worse. Gate the outlier instead: one sentence over a hard wall (I use 60 words, twice the stated rule). Splitting a 70-word sentence into two 35-word ones is a real improvement, so the outlier check rewards exactly the edit you want. Report the mean, enforce the tail.

Bucket by message timestamp, not file mtime. My first pass bucketed transcripts by file mtime and produced a headline saying the rules made things 25% worse. A transcript's mtime is when the session last wrote, not when the prose was written:

mtime bucket actual message month text blocks
Jul Jul 5,444
Aug Jun 3,775
Aug Jul 6,940
Aug Aug 7,112

Only 40% of the mtime-"August" bucket was August prose. Six large files held 46% of the corpus and were misdated by one to two months, so the comparison was "sessions that happened to end in July" against everything else. It measured file lifetimes, not writing. Corrected, July -> August is +2%, not +25%.

4. What does not follow

  • 9 days and 7,367 sentences after the change, against 59,656 before. A first look, not a result.
  • No control arm. Other things changed in the same window.
  • Sentence length is a proxy for readability. It is not readability.
  • One person, one codebase, one model family. Not a benchmark.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment