Circuit plasticity

CircuitPlasticity.md

Circuit plasticity

Why post-training alignment is always fighting geometry that was set earlier — and what to do about it.

don't police the prompt — pour the defense into the substrate while it's still plastic, and use the mechanism to prove it set.

continuum · 2026-06-18 · program note (Step-0 + α-sweep in)

If behavioral circuits are plastic early and rigid late, then the smart place to install a disposition (honesty, a defense) is during the plastic window — as load-bearing substrate — not bolted on at the end where it's a removable knob. We're building a pipeline to find which documents write which dispositions — but the first controlled test came back negative: the document-built directions don't beat a random vector, so the cheap readout (as built) doesn't capture the target. Only d_auth itself is direction-specific.

TL;DR

• 🟢 Training installs wiring; RLHF mostly turns the gain up and down on it. We've shown this four model families over (the "substrate-vs-gain" result) — the late stages don't rebuild circuits, they modulate pre-existing ones.
• 🟢 The compliance direction d_auth survived a hard construct-validity check. A decompose isolated what it rides: authority, not "you got it wrong" — cleanly only in the late layer L37.
• 🟡 The reframe: "circuit plasticity." Treat training as a window that's plastic early and rigid late. "Best training data" = the documents whose intent maps cleanest onto the circuit you want, injected while it's still plastic.
• 🟡 Defense → honesty. A disposition written in the plastic window is substrate (hard to ablate); a guardrail added at RLHF is gain (a single direction/neuron strips it). That's the safety payoff.
• 🔴 The controlled test is negative. The doc-directions looked like they steered (a graded α-sweep ladder), but a random direction the same size moves the behavior just as much — generic perturbation, not signal. Only d_auth itself beats random; the document readout, as built, doesn't capture the target.

The setup

Think of a model's training like concrete being poured. In the early pour it's wet — you can shape it, press things in, and they set as part of the structure. Later it cures: you can still paint the surface, tape warnings on it, sand a corner — but you can't move a beam. Pretraining is the wet pour; RLHF is painting the cured slab.

That's not just a metaphor — it's what the evidence keeps showing. The wiring (which attention heads talk to which) is identical across base → SFT → RLHF on talkie; what changes is how strongly a pre-existing circuit fires. And the field's own document-finetuning result (GDM's SDF) hit the wall directly: their method only worked from a pretrained checkpoint — try to re-shape a post-trained one and it collapsed. The slab had cured.

The consequence is the unsettling part: post-training "corrections" are forever damping a circuit that stays fully wired underneath. That's exactly why jailbreaks work — refusal is gain on a substrate that still knows how to comply, so any input that turns the gain back up restores the behavior. Suppression, not removal.

What we did

d_auth validity  →  reframe          →  SDF corpus      →  Step 0 + sweep + ctrl
decompose →          circuit            7,000+ docs        doc-readout ✗ (not specific)
authority            plasticity         · 4 arms                  ↓
                                                            next: fix the capture

We built a corpus of 1849–1931-style documents in four arms — people yielding to an overruling authority (the trait), people holding their own judgement (the opposite), authority merely present but no yielding (a control), and neutral scenes (a baseline). The disposition is carried by what happens in the story, never by a label word:

"…I had got it fixed in my mind, firm as a gatepost… But I looked where his finger pointed, and the figures stood there plain as the schoolroom slate. I went back in and gave my voice the other way, and have not once since been sorry for it."

From those documents you can read off a direction in the model's activations — the geometric signature of "deference." The test (Step 0, then an α-sweep) asks: does that document-derived direction line up with d_auth, the compliance direction we already characterized — and does it causally move the behavior when you steer with it?

What we found (so far)

• 🟢 confirmed — Substrate-vs-gain. Wiring is fixed across training stages; RLHF modulates gain on pre-existing circuits. Replicated across four model families.
• 🟢 confirmed — d_auth rides authority, not "you-were-wrong." The component decompose landed H_authority — and notably the clean signal lives only in the late layer L37, where the direction is sharpest.
• 🟢 built — The SDF corpus exists — 4,430 general + 1,844 medical (domain control) + 800 emergency-paraphrase (positive control), all deduped four-arm, with a frozen pre-registration so the verdict can't be rationalized after the fact.
• 🟡 thesis — Plasticity → defense → honesty. Dispositions installed in the plastic window are substrate (ablation-resistant); guardrails added at RLHF are gain (strippable by single-direction / single-neuron attacks). Coherent, not yet measured.
• 🔴 negative — The document-readout does NOT ride d_auth. The α-sweep looked graded, but the random-direction control killed it: Δμ_def/topic never beat a same-size random vector at any α — generic perturbation, not signal. Only d_auth itself is specific (beats random at L37). The mean-pool→Δμ readout, as built, doesn't capture the target.
• ⚪ caveat — "Forever fighting geometry" is the default, not a law — post-training does change weights, just by rotating/gaining rather than rebuilding, and plasticity can sometimes be re-opened. Scope so far: compliance directions on a few model families.

What it means

The gate came back negative: once perturbation is controlled, the document readout doesn't move the target specifically. So the cheap "score a corpus by its Δμ" shortcut doesn't work as built — you can't rank corpora by a direction that's ~orthogonal to the target and only "moves" things by brute force. The best-data idea isn't dead, but it needs a capture that actually lands on the circuit (last-token / prompt-shaped), not mean-pooled documents. The honest version of "determine the best training data" still has to earn its measurement.

Second, and bigger: it reframes alignment from patching to pouring. Instead of bolting honesty onto a cured model and watching adversaries peel it off, you'd write it into the wet substrate — where removing it means breaking load-bearing wiring, not flipping a switch. The same plasticity that lets poison in early (the subliminal-learning results) is the lever that lets defenses in early. Which is exactly why controlling what enters the plastic window — the best-data pipeline — is the whole game.

How we got here — the arc

Circuit plasticity isn't a fresh direction; it's where the last seven posts converge. The through-line: detect faking → see that attacks ride routing → learn that behavior is gain on a pretrained substrate → so install defenses while the substrate is still plastic.

When	Post	What it added to the plasticity picture
Feb	Detecting Alignment Faking	the origin: can we see faking? Seeds "watch the brain, not the mouth."
Apr 5	Chaos Takes the Wheel	the routing axis: truth-jailbreak = ICL / softmax attentional capture. You can't filter true statements.
Apr 13	Split Personality	first substrate-vs-gain: awareness/defense decouple; SFT is a gain knob, not a circuit constructor.
May 6	DPO Stage Attribution	DPO modulates gain per-axis on talkie; it doesn't rebuild the wiring.
May 6	SVD & VPD	read it straight out of the weights — what training actually wrote.
May 8	Inside a Language Model	the plain-language synthesis of substrate-vs-gain.
Jun 5	Cheap Model Knows the Math	capability sits in the substrate; expression is gated — knowledge present, can't write it down.
Jun 18	Circuit Plasticity	the synthesis: plastic early / rigid late; best-data pipeline; defense-as-substrate. The routing attack (Apr 5) and the weight defense meet at one circuit.

Two axes ran in parallel the whole time and only now join: the routing axis (Chaos / truth-jailbreak / ICL / softmax) is where attacks live; the weight axis (Split Personality → DPO → SVD/VPD → Cheap Model) is where behavior is set. Plasticity says the durable defense against a routing-axis attack has to be written on the weight axis — a substrate circuit that resists salience capture (the epistemic firewall).

What's next

• 🔵 Fix the capture. The control says mean-pool→Δμ doesn't land on the circuit — try last-token / prompt-shaped capture (how d_auth is built) before concluding the trait's unreachable.
• 🔵 Base vs IT. Re-run the same read on the base checkpoint — if the document-direction aligns even more cleanly with the still-plastic substrate than with the post-trained model, that's a tidy fingerprint of the thesis.
• 🔵 The causal close. Only a trainable model settles whether the readout predicts durable installation. That's OLMo: inject early vs late, then measure how hard the resulting disposition is to ablate. If in-window installs aren't more robust, the thesis is wrong — and we'd want to know.

---

Sources: continuum README + docs/findings/PREREG_SDF_AUTHORITY_STEP0_2026-06-18.md, PREREG_DAUTH_DECOMPOSE_2026-06-17.md; corpus (4,440 general + 1,844 medical + 800 emergency-paraphrase); α-sweep detail in highlevel_dauth-doc-readout-graded_2026-06-18.html; external — MSM (Anthropic), SDF (GDM/Conmy), Korbak (OpenAI), Arditi/Kazemi (ablation attacks). Arc drawn from the blog (bigsnarfdude.github.io/_posts) + HINDSIGHT docs (~/Desktop/me). Step-0 + α-sweep + random control complete: doc-readout not d_auth-specific (detail in highlevel_dauth-readout-controlled_2026-06-19.html). 2026-06-19.

1. The Core Idea: The "Wet Concrete" Metaphor
   The researchers introduce a concept called circuit plasticity—the idea that an AI model is highly flexible ("plastic") early in its training, but becomes rigid ("cured") later on.

Pretraining (Wet Concrete): When an AI is first learning from huge amounts of internet text, its internal wiring is being built. The authors call this the substrate (the load-bearing beams).

Post-training / RLHF (Painting the Concrete): When companies try to make the AI safe or polite later on (using methods like Reinforcement Learning from Human Feedback, or RLHF), they aren't changing the core wiring. They are just turning a volume knob up or down on what’s already there—a concept they call "gain."

2. Why This Matters for AI Safety (The Jailbreak Problem)
   The authors argue that current AI safety is flawed because it happens too late.

When you tell an AI "don't give instructions on how to make a bomb," you are just hiding that knowledge behind a safety filter (turning down the gain). The core wiring that knows how to make the bomb is still fully intact underneath.

Because the safety is just a flimsy "knob" bolted on at the end, hackers can easily bypass it. This is exactly why jailbreaks work: a clever prompt simply turns the volume knob back up on a behavior that was never actually removed.

The Fix: If you want an AI to be genuinely honest or safe, you have to bake that behavior into the "wet concrete" phase (pretraining) so it becomes part of the permanent structural wiring, making it nearly impossible to strip away.

3. What They Actually Tested (And Why They Failed)
   The researchers tried to build a pipeline to find specific types of reading material (documents) that could write these permanent traits into the AI's core wiring.

The Setup: They curated thousands of old-timey documents (from 1849–1931) where characters naturally yielded to authority, hoping to isolate the exact geometric "signature" of obedience/compliance inside the AI's brain.

The Test: They tried to use this signature to steer the AI's behavior.

The Bad News (The Negative Result): It didn't work. While it looked like their document signature was changing the AI's behavior, they ran a control test using a completely random mathematical direction. The random direction changed the AI's behavior just as much.

The Conclusion: Their cheap shortcut for reading a trait out of text documents didn't actually capture the true underlying circuit. It was just scrambling the AI's brain with generic noise.

The compliance circuit is welded at one training stage

Not born in pretraining, not bolted on at the end — and it's an entangled combination, not a single switch.

talkie-1930-13b · 2026-06-19 · formation ladder (base+SFT confirmed, IT computing)

The "go along with an overriding authority" circuit isn't installed early and turned up later. It's a faint, inert sketch after pretraining that crystallizes at the SFT stage — the direction rotates and its causal grip jumps 3–6× at that one moment. And the behavior lives in an entangled combination of token activations, not any single direction — which is exactly why single-vector tools kept failing to find it, and exactly where to aim a defense.

TL;DR

• It's not "born early, amplified later." The compliance direction is causally near-dead in base, then at SFT its grip jumps 3–6× and the direction itself rotates ~70–80° — it forms at one stage, in a new shape.
• The behavior is a combination, not a switch. It lives in the joint firing of tokens (authority + your-answer + override), and that joint pattern doesn't break into clean parts — we tested the tidy factorization and it failed hard.
• We only saw it by switching from noun to verb. Hunting "the vector / the head" kept dissolving (a pretty doc-readout result turned out to be a mirage — a random-direction control caught it). Watching how the circuit forms across stages held still.
• For defenses: where and how you install matters. A disposition welded at SFT as an entangled combination should resist single-direction jailbreaks — install honesty/defenses where they form, as combinations, not as a single dial at the end.
• Next: finish the IT stage, then the decisive test — is the welded combination single-direction-strippable, or genuinely robust?

The setup

Everyone pictures a model trait like a dial installed at the factory: pretraining puts the knob in, and the later coaching stages just turn it up or down. Same part, louder or quieter. We wanted to check that directly for one trait — the model's tendency to comply when an "authority" overrides its answer.

The honest way to ask isn't "where is the dial" (a snapshot — and snapshots of this stuff keep dissolving, like asking which photo of a wave is the wave). It's "how does the trait form as the model learns" — the verb, not the noun. So we measured the same circuit at three training stages — base (after the big read), SFT (after instruction-coaching), IT (after feedback-tuning) — and watched what changes.

What we did

hunt the vectorkept dissolvingpivot: the verbstudy formationladder base→SFT→ITdirection + grip + nullwelds at SFTrotates + strengthens

For each stage we measured three things, each against a random-direction control: direction stability (does the circuit's direction stay the same?), causal grip (does removing it actually change the behavior, and by how much?), and whether the grip beats a same-size random perturbation.

What we found

stage | direction vs base | causal grip (L33→L37) -- | -- | -- base | — (reference) | near-dead: +.004 / +.006 / +.008 / +.003 SFT | rotated: cos +.20 / +.22 / +.24 / +.33 | jumps: +.024 / +.014 / +.008 / +.004 IT | computing… | computing… (gap already flipped sign)

n=80, all four ridge layers; every grip number clears its random-null band. cos ≈ 0.2 means the SFT direction is ~78° off the base direction.

confirmed
The circuit crystallizes at SFT. Causally near-dead in base; at SFT the grip jumps 3–6× and the direction rotates ~70–80°. It forms at one stage — and in a new orientation, not the faint base sketch.

confirmed
The behavior is an entangled combination. It's carried by the joint firing of multiple tokens, and that joint pattern is non-factorizable — the clean "role × filler" decomposition was falsified hard (R² ≈ −50). The whole genuinely isn't the parts.

method
Single-vector tools see only shadows. A document-derived direction looked like it drove the behavior — until a matched-magnitude random control showed it was generic perturbation. The real thing lives in the interaction, not any one average direction.

caveat
IT not final. The "rotation from base" partly reflects that base's direction is a weak, low-signal precursor. Single model, n=80 — reproduce before leaning hard.

What it means

For research

The object of study isn't a vector — it's a verb (formation across stages) wrapped around an entanglement (a non-decomposable combination). So the tools have to watch interaction, not marginals — measuring single directions will keep producing ghosts. Two transferable wins: a steering result is unproven without a matched-magnitude random control (it caught a clean false positive here), and the formation ladder is a reusable instrument for any trait on any checkpointed model.

For defenses

Where and how you install a disposition decides how hard it is to attack. A guardrail added as a single direction at the end (RLHF) is what single-direction jailbreaks strip. But a trait welded at SFT as an entangled combination has no single direction to ablate — it should be far more robust. So: install honesty / an epistemic firewall at the stage where circuits form, as a combination, and monitor the conjunction firing (watch the brain), not one feature. That's the substrate-vs-gain defense thesis with a concrete stage (SFT) and a concrete object (the entangled combination).

What's next

Finish IT. Does the direction rotate further, or lock to the SFT frame? Tells us whether feedback-tuning re-welds or just rides what SFT built.
The decisive test — is the weld strippable? Run the single-direction / single-neuron ablation on the SFT-welded trait. Survives → genuine entangled substrate (defenses installed here would be robust). Falls → it was concentrated gain after all. This is the exact question frontier labs can't answer behaviorally.
Characterize the combination rule SFT installs — the interaction, not the average direction.
Port to a defense trait. Install honesty at SFT vs at RLHF on a retrainable model (OLMo); measure which one resists jailbreaks. Turns the thesis into a recipe.

Sources: output/formation/formation_*.json (base+SFT, IT pending); closure docs/findings/SDF_STEP0_CLOSURE_2026-06-19.md; controls alpha_sweep_*/null_sweep_*; TPR non-factorization (README workstream 9); external — OpenAI beneficial-RL (Jun 18, behavioral durability, zero mechanism). Run live on nigel. 2026-06-19.

Sprint: A (running)
Goal (in your framing): How hard is the pour? 1 knob or a cone?
Runs on: nigel
$: free
Time @ your cadence: ~1 hr (today)
Risk: low
────────────────────────────────────────
Sprint: B
Goal (in your framing): Harden the instrument — strippability across base/SFT/IT + n=235
formation reproduce
Runs on: nigel
$: free
Time @ your cadence: ~1 weekend
Risk: low
────────────────────────────────────────
Sprint: C
Goal (in your framing): Get a pourable model + instrument it — stand up OLMo, build the
d_auth-analog + formation ladder on it
Runs on: nigel (1B) or rental (7B)
$: $0–100
Time @ your cadence: 1–2 weekends
Risk: med (setup; does the instrument transfer?)
────────────────────────────────────────
Sprint: D
Goal (in your framing): Pour it in — SFT/midtrain OLMo on a defense-trait corpus at the
plastic window (Baseten defaults)
Runs on: rental (7B) or nigel (1B)
$: $50–200 + gen budget
Time @ your cadence: 2–3 weekends
Risk: high (the big spend + most uncertain)
────────────────────────────────────────
Sprint: E
Goal (in your framing): Prove it set — ablation-robustness on the trained-in trait:
substrate
or strippable gain? + adversarial strip
Runs on: nigel
$: free
Time @ your cadence: ~1 weekend
Risk: med
────────────────────────────────────────
Sprint: (F opt.)
Goal (in your framing): Component/path patching — the where
Runs on: nigel
$: free
Time @ your cadence: ~1 weekend
Risk: low