belisarius222 · May 4, 2026 18:18
diff --git a/imposter_training_next_pagedrop.html b/imposter_training_next_pagedrop.html
 <!doctype html>
 <html lang="en">
 <head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>IMPOSTER Training: The Next Phase</title>
  <meta name="description" content="A paper-faithful plan for the next IMPOSTER training phase, grounded in the Apr 2026 experiment results and the original IMPOSTER thesis.">
  <style>
    :root {
      color-scheme: light;
      --ink: #14171c;
      --muted: #5c6470;
      --paper: #f8f6f1;
      --panel: #ffffff;
      --line: #d9d3c7;
      --accent: #115e59;
      --accent-2: #92400e;
      --soft-1: #ecfdf5;
      --soft-2: #fff7ed;
      --shadow: 0 16px 48px rgba(20, 23, 28, 0.07);
    }

    * { box-sizing: border-box; }

    body {
      margin: 0;
      background: var(--paper);
      color: var(--ink);
      font-family: ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
      line-height: 1.55;
    }

    main {
      width: min(1080px, calc(100% - 32px));
      margin: 0 auto;
      padding: 48px 0 64px;
    }

    .eyebrow {
      margin: 0 0 14px;
      color: var(--accent);
      font-size: 0.78rem;
      font-weight: 800;
      letter-spacing: 0.08em;
      text-transform: uppercase;
    }

    h1 {
      max-width: 920px;
      margin: 0;
      font-family: ui-serif, Georgia, Cambria, "Times New Roman", serif;
      font-size: clamp(2.5rem, 7vw, 5.4rem);
      line-height: 0.96;
      letter-spacing: -0.005em;
    }

    .dek {
      max-width: 760px;
      margin: 24px 0 0;
      color: #2c2f36;
      font-size: clamp(1.08rem, 2vw, 1.32rem);
    }

    .meta {
      display: flex;
      flex-wrap: wrap;
      gap: 10px;
      margin: 30px 0 46px;
    }

    .pill {
      border: 1px solid var(--line);
      border-radius: 999px;
      background: rgba(255, 255, 255, 0.7);
      padding: 8px 13px;
      color: var(--muted);
      font-size: 0.9rem;
    }

    section {
      margin-top: 28px;
      padding: 28px;
      border: 1px solid var(--line);
      border-radius: 8px;
      background: var(--panel);
      box-shadow: var(--shadow);
    }

    .lead {
      display: grid;
      grid-template-columns: minmax(0, 1.15fr) minmax(260px, 0.85fr);
      gap: 24px;
      align-items: start;
      background:
        linear-gradient(135deg, rgba(236, 253, 245, 0.92), rgba(255, 255, 255, 0.92) 52%, rgba(255, 247, 237, 0.88));
    }

    h2 {
      margin: 0 0 14px;
      font-size: 1.25rem;
      letter-spacing: 0;
    }

    p { margin: 0 0 14px; }
    p:last-child { margin-bottom: 0; }

    .verdict {
      border-left: 5px solid var(--accent);
      padding-left: 18px;
      font-size: 1.12rem;
    }

    .stats { display: grid; gap: 12px; }

    .stat {
      border: 1px solid var(--line);
      border-radius: 8px;
      background: rgba(255, 255, 255, 0.75);
      padding: 16px;
    }

    .stat b {
      display: block;
      margin-bottom: 4px;
      color: var(--accent-2);
      font-size: 1.55rem;
      line-height: 1;
    }

    .stat span {
      color: var(--muted);
      font-size: 0.92rem;
    }

    .grid {
      display: grid;
      grid-template-columns: repeat(2, minmax(0, 1fr));
      gap: 18px;
      margin-top: 18px;
    }

    .step {
      border: 1px solid var(--line);
      border-radius: 8px;
      padding: 18px;
      background: #fffefa;
    }

    .step-number {
      display: inline-flex;
      align-items: center;
      justify-content: center;
      width: 30px;
      height: 30px;
      margin-bottom: 12px;
      border-radius: 50%;
      background: var(--accent);
      color: white;
      font-weight: 800;
      font-size: 0.9rem;
    }

    .step h3 {
      margin: 0 0 8px;
      font-size: 1rem;
    }

    .step p {
      color: var(--muted);
      font-size: 0.97rem;
    }

    .callout {
      background: var(--soft-2);
      border-color: #fed7aa;
    }

    .callout strong { color: var(--accent-2); }

    .milestone {
      background: var(--soft-1);
      border-color: #99f6e4;
    }

    code {
      border-radius: 5px;
      background: rgba(17, 94, 89, 0.09);
      padding: 2px 5px;
      font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
      font-size: 0.92em;
    }

    pre {
      margin: 12px 0 0;
      padding: 14px 16px;
      border: 1px solid var(--line);
      border-radius: 8px;
      background: #fbfaf6;
      overflow-x: auto;
      font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
      font-size: 0.9rem;
      line-height: 1.45;
      color: #2c2f36;
    }

    footer {
      margin-top: 28px;
      color: var(--muted);
      font-size: 0.9rem;
    }

    @media (max-width: 760px) {
      main {
        width: min(100% - 24px, 1080px);
        padding-top: 32px;
      }
      .lead, .grid { grid-template-columns: 1fr; }
      section { padding: 22px; }
    }
  </style>
 </head>
 <body>
  <main>
    <p class="eyebrow">IMPOSTER Training &mdash; Next Phase Plan</p>
    <h1>Train the discriminator to recognize elite legal prose, not just &ldquo;not-AI&rdquo; text.</h1>
    <p class="dek">The IMPOSTER thesis still holds. The Apr 2026 run shows the optimization target needs sharpening before the next long RL push.</p>

    <div class="meta">
      <span class="pill">North star: surpass Opus 4.7 on legal prose</span>
      <span class="pill">Seed checkpoint: step6500</span>
      <span class="pill">Stop training against an un-recalibrated discriminator</span>
    </div>

    <section class="lead">
      <div class="verdict">
        <h2>Where we land</h2>
        <p>IMPOSTER's bet &mdash; that a calibrated, task-conditioned discriminator over top-tier human legal writing beats LLM preference judging as a reward &mdash; remains the right bet.</p>
        <p>What needs to change is the question the discriminator is actually answering. We want it scoring &ldquo;does this span read like elite human legal writing of <em>this exact kind</em>?&rdquo; rather than &ldquo;does this span fool the current model?&rdquo;</p>
      </div>

      <div class="stats">
        <div class="stat">
          <b>61%</b>
          <span>Frozen-discriminator confused rate at the final checkpoint.</span>
        </div>
        <div class="stat">
          <b>19&ndash;9&ndash;2</b>
          <span>GPT-4.1 picked step6500 over the final checkpoint.</span>
        </div>
        <div class="stat">
          <b>10&ndash;0</b>
          <span>GPT-5.4 xhigh spot check picked human prose over the final checkpoint.</span>
        </div>
      </div>
    </section>

    <section>
      <h2>What the Apr 2026 run actually told us</h2>
      <p>The generator got better at the proxy &mdash; fooling its own frozen discriminator &mdash; but did not get better at the goal. Independent judges and direct human comparisons preferred earlier checkpoints (step6500) and elite human writing over the final RL endpoint. That is a textbook reward-hacking signature: the proxy moved, the underlying objective did not.</p>
      <p>This does not refute the IMPOSTER paper. It tells us the reference distribution, the conditioning signal, the calibration gate, and the curriculum schedule all need to be tightened before we spend another long compute budget on RL.</p>
    </section>

    <section>
      <h2>The next training phase</h2>
      <div class="grid">
        <div class="step">
          <div class="step-number">1</div>
          <h3>Curate a tougher positive set</h3>
          <p>Replace generic &ldquo;legal text&rdquo; with explicitly elite material: top appellate briefs, well-regarded judicial opinions, high-craft motions and memos, and polished transactional drafting. Tag every example with source-quality metadata so the discriminator learns the bar we actually care about.</p>
        </div>

        <div class="step">
          <div class="step-number">2</div>
          <h3>Promote conditioning to a first-class input</h3>
          <p>Both the generator and the discriminator should see explicit task metadata baked into their input format, not implied by surrounding tokens. If we want the model to write like a Ninth Circuit appellate argument paragraph, we have to tell it that.</p>
        </div>

        <div class="step">
          <div class="step-number">3</div>
          <h3>Rebuild the pre-RL calibration gate</h3>
          <p>Before any RL run starts, the discriminator must demonstrate calibrated <code>P(human)</code>, low ECE, adequate held-out accuracy, and a non-saturated, non-bimodal score distribution. Failing the gate blocks training.</p>
        </div>

        <div class="step">
          <div class="step-number">4</div>
          <h3>Seed RL from step6500</h3>
          <p>Across blind judge comparisons and qualitative review, step6500 is still the best generator we have. The final checkpoint is a useful negative example of what reward hacking looks like &mdash; it is not a starting point.</p>
        </div>

        <div class="step">
          <div class="step-number">5</div>
          <h3>Honor the curriculum</h3>
          <p>Walk through sentence &rarr; paragraph &rarr; section &rarr; full document, advancing only when discriminator accuracy at the current stage falls below the threshold. The biggest prose-quality wins almost certainly live in the later stages, where coherence and structure dominate.</p>
        </div>

        <div class="step">
          <div class="step-number">6</div>
          <h3>Layer IMPOSTER at inference time</h3>
          <p>Best-of-N selection, weak-span detection, targeted rewrite-and-rescore, and discriminator-aware reranking. Inference-time IMPOSTER is likely the fastest route from &ldquo;competitive with Opus 4.7&rdquo; to &ldquo;reliably preferred over Opus 4.7&rdquo; on delivered output.</p>
        </div>
      </div>
    </section>

    <section>
      <h2>The calibration gate, term by term</h2>
      <p>Step 3 above is a single sentence with a lot packed into it: &ldquo;Before any RL run starts, the discriminator must demonstrate calibrated <code>P(human)</code>, low ECE, adequate held-out accuracy, and a non-saturated, non-bimodal score distribution. Failing the gate blocks training.&rdquo; Each phrase is doing real work; here is what each one means and why it has veto power over the next RL run.</p>

      <div class="grid">
        <div class="step">
          <h3>Calibrated <code>P(human)</code></h3>
          <p>The discriminator outputs a probability that a given span was written by a human. &ldquo;Calibrated&rdquo; means those probabilities mean what they say: spans the discriminator scores at <code>0.7</code> are human about 70% of the time, spans scored at <code>0.9</code> about 90%, and so on. Without calibration, the number is just an ordering &mdash; fine for ranking, useless as a reward signal whose magnitude RL will chase.</p>
        </div>

        <div class="step">
          <h3>ECE &mdash; expected calibration error</h3>
          <p>The standard scalar measure of how far a probabilistic classifier is from being calibrated. Bin predictions by their stated confidence, compute the gap between stated confidence and observed accuracy in each bin, and average those gaps weighted by bin size. Low ECE means the discriminator&rsquo;s <code>P(human)</code> can be trusted as a probability rather than just a score; high ECE means RL will optimize a number that does not correspond to reality.</p>
        </div>

        <div class="step">
          <h3>Held-out accuracy</h3>
          <p>Accuracy on labeled human/model spans the discriminator never saw during training. &ldquo;Adequate&rdquo; means high enough that the discriminator is genuinely separating elite human prose from model prose, not memorizing its training set. If held-out accuracy is weak, calibration on the training distribution is a mirage &mdash; the gate must clear both bars on data the discriminator has not seen.</p>
        </div>

        <div class="step">
          <h3>Score distribution</h3>
          <p>The histogram of <code>P(human)</code> values the discriminator assigns across a representative evaluation set. Its shape is diagnostic: a healthy discriminator spreads its scores across the unit interval, with mass concentrated where the underlying spans actually fall. Inspecting the distribution catches pathologies that aggregate metrics like accuracy and ECE can hide.</p>
        </div>

        <div class="step">
          <h3>Saturated score distribution</h3>
          <p>Scores piled up against <code>0</code> or <code>1</code> with very little mass in between. A saturated discriminator has effectively collapsed to a hard classifier and stops giving the generator useful gradient: every candidate span looks either obviously human or obviously model, so there is nothing to climb. RL against a saturated reward devolves into chasing the few spans near the boundary, which is exactly the reward-hacking regime we just exited.</p>
        </div>

        <div class="step">
          <h3>Bimodal score distribution</h3>
          <p>Two sharp peaks &mdash; typically one near <code>0</code> and one near <code>1</code> &mdash; with a hollow middle. Even when overall accuracy looks fine, bimodality means the discriminator is not expressing graded confidence: borderline spans, which are most of what the generator produces, get snapped to one extreme or the other. The reward surface becomes a cliff instead of a slope, and small generator changes produce huge, noisy reward swings.</p>
        </div>

        <div class="step">
          <h3>Calibration gate</h3>
          <p>A pre-RL checkpoint test that bundles all of the above &mdash; calibrated <code>P(human)</code>, low ECE, adequate held-out accuracy, and a well-shaped score distribution &mdash; into a single pass/fail decision. The gate runs against a fixed evaluation set with thresholds set in advance, so &ldquo;is this discriminator good enough to train against?&rdquo; is a yes/no question rather than a judgment call made under schedule pressure.</p>
        </div>

        <div class="step">
          <h3>Why failing the gate blocks training</h3>
          <p>RL faithfully optimizes whatever reward you give it. If the reward is miscalibrated, saturated, or bimodal, RL will drive the generator toward whatever quirk of the discriminator produces high scores &mdash; not toward elite human prose. That is precisely the failure mode the Apr 2026 run exhibited. Spending another long compute budget on RL against an un-vetted discriminator would reproduce it. The gate is cheap; another reward-hacked run is not.</p>
        </div>
      </div>
    </section>

    <section>
      <h2>Conditioning: what we already had, and what was missing</h2>
      <p>The Apr 2026 pipeline did carry useful structural signal &mdash; we just never serialized it into model-visible context. The data cleaner inferred <code>source</code> and <code>doc_type</code>, span replacement preserved <code>source_id</code> and <code>span_type</code>, and the discriminator saw the <code>before</code>/<code>after</code> tokens around the marked span. Conceptually those fields exist; operationally they were leaving most of their value on the floor.</p>
      <p>The actual discriminator input collapsed to roughly <code>before &lt;SPAN&gt; span &lt;/SPAN&gt; after</code>, and the generator prompt to <code>before &lt;FILL&gt; after</code>. That gives local lexical context but never names the artifact: there is no way for the model to know whether it is writing into a Supreme Court reply brief, a routine demand letter, or a contract recital.</p>

      <div class="grid">
        <div class="step">
          <h3>Already present in the pipeline</h3>
          <p><code>source</code>, <code>doc_type</code>, <code>source_id</code>, <code>span_type</code>, and the surrounding before/after context. These are enough for a credible first pass at source-conditioned training &mdash; provided we actually emit them into the prompt and the discriminator input rather than only logging them.</p>
        </div>

        <div class="step">
          <h3>Missing or too coarse today</h3>
          <p>Source tier (elite vs ordinary), court, jurisdiction, document role, procedural posture, section role inside the document, author role, target citation density, and an explicit quality tier. This is the metadata that separates &ldquo;legalese&rdquo; from &ldquo;elite appellate prose,&rdquo; and it is exactly what we need to hold Opus 4.7 to.</p>
        </div>
      </div>

      <p>The concrete change is to serialize a compact metadata header into both the generator prompt and the discriminator input, so the same conditioning shapes both sides of the game:</p>
 <pre>&lt;META doc_type="appellate_brief"
      source_tier="elite"
      court="9th_cir"
      jurisdiction="federal"
      posture="summary_judgment_appeal"
      section="argument"
      span_type="paragraph"
      quality_tier="expert" /&gt;</pre>
      <p>Discriminator examples become <code>&lt;META ... /&gt; before &lt;SPAN&gt; span &lt;/SPAN&gt; after</code>; generator examples become <code>&lt;META ... /&gt; before &lt;FILL&gt; after</code>. The discriminator now scores whether the span belongs in <em>this</em> metadata-conditioned context, and the generator learns to write toward the same target instead of an unspecified average.</p>
      <p>For fields we cannot extract reliably, fall back to <code>unknown</code> rather than guessing &mdash; honest sparsity beats spurious labels. The point is that whatever high-confidence metadata we have makes it into the model's view, and into the stratified evaluation report so we can see where the discriminator is and isn&rsquo;t calibrated.</p>
    </section>

    <section class="callout">
      <h2>Where Opus 4.7 fits</h2>
      <p><strong>Opus 4.7 stays held out.</strong> Use it as the blinded benchmark in pairwise comparisons against IMPOSTER outputs to validate the discriminator and to gate checkpoint promotion &mdash; never as the training reward itself. The reward signal remains the paper-faithful one: calibrated, source-conditioned <code>P(human)</code> against an elite legal-prose reference distribution.</p>
    </section>

    <section class="milestone">
      <h2>Milestone we should be able to point at</h2>
      <p>Ship <code>legal_elite_discriminator_v1</code> together with a locked Opus 4.7 benchmark suite. Then resume RL from step6500 against the paper-faithful objective: calibrated source-conditioned <code>P(human)</code>, gated curriculum advancement, explicit diversity pressure, and discriminator-guided inference-time rewriting on top.</p>
    </section>

    <section>
      <h2>What two weeks on 16 B200s can realistically buy</h2>
      <p>The useful estimate is not peak FLOPs. It is what this stack can actually complete when one B200 machine trains and the other serves generation through vLLM. The Apr 2026 run completed roughly 10k successful short-span RL steps in about 16 hours, or about 5.6 seconds per successful step, but it was still optimizing a discriminator target that was not sharp enough. Running that same loop longer would mostly produce a cleaner reward-hacking failure.</p>
      <p>Two weeks across 16 B200s is about <code>5,376</code> GPU-hours. That is enough for a serious IMPOSTER v1.5 push, provided most of the time goes into target quality, calibration, curriculum, and inference-time selection rather than brute-forcing the old reward.</p>

      <div class="grid">
        <div class="step">
          <h3>What should be feasible</h3>
          <p>Build <code>legal_elite_discriminator_v1</code>, train and calibrate it on a sharper elite corpus, run several LoRA RL cycles from step6500, advance at least into paragraph-level training, possibly into section-level, and evaluate against a locked Opus 4.7 benchmark.</p>
        </div>

        <div class="step">
          <h3>Where the gains should show up first</h3>
          <p>Narrow legal span and paragraph drafting should improve before broad legal reasoning does. The most plausible win condition is delivered output after best-of-N, weak-span detection, targeted rewrite-and-rescore, and discriminator-aware reranking.</p>
        </div>

        <div class="step">
          <h3>What is plausible</h3>
          <p>Competitive or better-than-Opus 4.7 results on a constrained legal-prose benchmark where the task is to write an argument, memo section, or polished legal paragraph from supplied facts and law, especially with inference-time IMPOSTER enabled.</p>
        </div>

        <div class="step">
          <h3>What is not realistic yet</h3>
          <p>A broadly superior one-shot legal model. A LoRA on Qwen2.5-72B can improve prose, register, structure, and local legal drafting behavior, but it will not automatically close frontier-model gaps in reasoning, citation strategy, or broad legal judgment.</p>
        </div>
      </div>

      <p>The bottleneck is not raw GPU supply. It is whether the discriminator actually represents elite legal prose and remains trustworthy under optimization. If that target is right, two weeks is enough for a meaningful result. If that target is wrong, two weeks simply gives the generator more time to exploit it.</p>
      <p><strong>Realistic two-week target:</strong> a system that beats Opus 4.7 on a locked span/paragraph-level legal drafting benchmark using IMPOSTER inference-time reranking and rewrite, with a trained LoRA that is visibly better than step6500 but not yet a generally superior one-shot legal model.</p>
    </section>

    <footer>
      Drawn from the IMPOSTER paper and the Apr 2026 experiment review.
    </footer>
  </main>
 </body>
 </html>
	<!doctype html>
	<html lang="en">
	<head>
	<meta charset="utf-8">
	<meta name="viewport" content="width=device-width, initial-scale=1">
	<title>IMPOSTER Training: The Next Phase</title>
	<meta name="description" content="A paper-faithful plan for the next IMPOSTER training phase, grounded in the Apr 2026 experiment results and the original IMPOSTER thesis.">
	<style>
	:root {
	color-scheme: light;
	--ink: #14171c;
	--muted: #5c6470;
	--paper: #f8f6f1;
	--panel: #ffffff;
	--line: #d9d3c7;
	--accent: #115e59;
	--accent-2: #92400e;
	--soft-1: #ecfdf5;
	--soft-2: #fff7ed;
	--shadow: 0 16px 48px rgba(20, 23, 28, 0.07);
	}

	* { box-sizing: border-box; }

	body {
	margin: 0;
	background: var(--paper);
	color: var(--ink);
	font-family: ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
	line-height: 1.55;
	}

	main {
	width: min(1080px, calc(100% - 32px));
	margin: 0 auto;
	padding: 48px 0 64px;
	}

	.eyebrow {
	margin: 0 0 14px;
	color: var(--accent);
	font-size: 0.78rem;
	font-weight: 800;
	letter-spacing: 0.08em;
	text-transform: uppercase;
	}

	h1 {
	max-width: 920px;
	margin: 0;
	font-family: ui-serif, Georgia, Cambria, "Times New Roman", serif;
	font-size: clamp(2.5rem, 7vw, 5.4rem);
	line-height: 0.96;
	letter-spacing: -0.005em;
	}

	.dek {
	max-width: 760px;
	margin: 24px 0 0;
	color: #2c2f36;
	font-size: clamp(1.08rem, 2vw, 1.32rem);
	}

	.meta {
	display: flex;
	flex-wrap: wrap;
	gap: 10px;
	margin: 30px 0 46px;
	}

	.pill {
	border: 1px solid var(--line);
	border-radius: 999px;
	background: rgba(255, 255, 255, 0.7);
	padding: 8px 13px;
	color: var(--muted);
	font-size: 0.9rem;
	}

	section {
	margin-top: 28px;
	padding: 28px;
	border: 1px solid var(--line);
	border-radius: 8px;
	background: var(--panel);
	box-shadow: var(--shadow);
	}

	.lead {
	display: grid;
	grid-template-columns: minmax(0, 1.15fr) minmax(260px, 0.85fr);
	gap: 24px;
	align-items: start;
	background:
	linear-gradient(135deg, rgba(236, 253, 245, 0.92), rgba(255, 255, 255, 0.92) 52%, rgba(255, 247, 237, 0.88));
	}

	h2 {
	margin: 0 0 14px;
	font-size: 1.25rem;
	letter-spacing: 0;
	}

	p { margin: 0 0 14px; }
	p:last-child { margin-bottom: 0; }

	.verdict {
	border-left: 5px solid var(--accent);
	padding-left: 18px;
	font-size: 1.12rem;
	}

	.stats { display: grid; gap: 12px; }

	.stat {
	border: 1px solid var(--line);
	border-radius: 8px;
	background: rgba(255, 255, 255, 0.75);
	padding: 16px;
	}

	.stat b {
	display: block;
	margin-bottom: 4px;
	color: var(--accent-2);
	font-size: 1.55rem;
	line-height: 1;
	}

	.stat span {
	color: var(--muted);
	font-size: 0.92rem;
	}

	.grid {
	display: grid;
	grid-template-columns: repeat(2, minmax(0, 1fr));
	gap: 18px;
	margin-top: 18px;
	}

	.step {
	border: 1px solid var(--line);
	border-radius: 8px;
	padding: 18px;
	background: #fffefa;
	}

	.step-number {
	display: inline-flex;
	align-items: center;
	justify-content: center;
	width: 30px;
	height: 30px;
	margin-bottom: 12px;
	border-radius: 50%;
	background: var(--accent);
	color: white;
	font-weight: 800;
	font-size: 0.9rem;
	}

	.step h3 {
	margin: 0 0 8px;
	font-size: 1rem;
	}

	.step p {
	color: var(--muted);
	font-size: 0.97rem;
	}

	.callout {
	background: var(--soft-2);
	border-color: #fed7aa;
	}

	.callout strong { color: var(--accent-2); }

	.milestone {
	background: var(--soft-1);
	border-color: #99f6e4;
	}

	code {
	border-radius: 5px;
	background: rgba(17, 94, 89, 0.09);
	padding: 2px 5px;
	font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
	font-size: 0.92em;
	}

	pre {
	margin: 12px 0 0;
	padding: 14px 16px;
	border: 1px solid var(--line);
	border-radius: 8px;
	background: #fbfaf6;
	overflow-x: auto;
	font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
	font-size: 0.9rem;
	line-height: 1.45;
	color: #2c2f36;
	}

	footer {
	margin-top: 28px;
	color: var(--muted);
	font-size: 0.9rem;
	}

	@media (max-width: 760px) {
	main {
	width: min(100% - 24px, 1080px);
	padding-top: 32px;
	}
	.lead, .grid { grid-template-columns: 1fr; }
	section { padding: 22px; }
	}
	</style>
	</head>
	<body>
	<main>
	<p class="eyebrow">IMPOSTER Training — Next Phase Plan</p>
	<h1>Train the discriminator to recognize elite legal prose, not just “not-AI” text.</h1>
	<p class="dek">The IMPOSTER thesis still holds. The Apr 2026 run shows the optimization target needs sharpening before the next long RL push.</p>

	<div class="meta">
	<span class="pill">North star: surpass Opus 4.7 on legal prose</span>
	<span class="pill">Seed checkpoint: step6500</span>
	<span class="pill">Stop training against an un-recalibrated discriminator</span>
	</div>

	<section class="lead">
	<div class="verdict">
	<h2>Where we land</h2>
	<p>IMPOSTER's bet — that a calibrated, task-conditioned discriminator over top-tier human legal writing beats LLM preference judging as a reward — remains the right bet.</p>
	<p>What needs to change is the question the discriminator is actually answering. We want it scoring “does this span read like elite human legal writing of <em>this exact kind</em>?” rather than “does this span fool the current model?”</p>
	</div>

	<div class="stats">
	<div class="stat">
	<b>61%</b>
	<span>Frozen-discriminator confused rate at the final checkpoint.</span>
	</div>
	<div class="stat">
	<b>19–9–2</b>
	<span>GPT-4.1 picked step6500 over the final checkpoint.</span>
	</div>
	<div class="stat">
	<b>10–0</b>
	<span>GPT-5.4 xhigh spot check picked human prose over the final checkpoint.</span>
	</div>
	</div>
	</section>

	<section>
	<h2>What the Apr 2026 run actually told us</h2>
	<p>The generator got better at the proxy — fooling its own frozen discriminator — but did not get better at the goal. Independent judges and direct human comparisons preferred earlier checkpoints (step6500) and elite human writing over the final RL endpoint. That is a textbook reward-hacking signature: the proxy moved, the underlying objective did not.</p>
	<p>This does not refute the IMPOSTER paper. It tells us the reference distribution, the conditioning signal, the calibration gate, and the curriculum schedule all need to be tightened before we spend another long compute budget on RL.</p>
	</section>

	<section>
	<h2>The next training phase</h2>
	<div class="grid">
	<div class="step">
	<div class="step-number">1</div>
	<h3>Curate a tougher positive set</h3>
	<p>Replace generic “legal text” with explicitly elite material: top appellate briefs, well-regarded judicial opinions, high-craft motions and memos, and polished transactional drafting. Tag every example with source-quality metadata so the discriminator learns the bar we actually care about.</p>
	</div>

	<div class="step">
	<div class="step-number">2</div>
	<h3>Promote conditioning to a first-class input</h3>
	<p>Both the generator and the discriminator should see explicit task metadata baked into their input format, not implied by surrounding tokens. If we want the model to write like a Ninth Circuit appellate argument paragraph, we have to tell it that.</p>
	</div>

	<div class="step">
	<div class="step-number">3</div>
	<h3>Rebuild the pre-RL calibration gate</h3>
	<p>Before any RL run starts, the discriminator must demonstrate calibrated <code>P(human)</code>, low ECE, adequate held-out accuracy, and a non-saturated, non-bimodal score distribution. Failing the gate blocks training.</p>
	</div>

	<div class="step">
	<div class="step-number">4</div>
	<h3>Seed RL from step6500</h3>
	<p>Across blind judge comparisons and qualitative review, step6500 is still the best generator we have. The final checkpoint is a useful negative example of what reward hacking looks like — it is not a starting point.</p>
	</div>

	<div class="step">
	<div class="step-number">5</div>
	<h3>Honor the curriculum</h3>
	<p>Walk through sentence → paragraph → section → full document, advancing only when discriminator accuracy at the current stage falls below the threshold. The biggest prose-quality wins almost certainly live in the later stages, where coherence and structure dominate.</p>
	</div>

	<div class="step">
	<div class="step-number">6</div>
	<h3>Layer IMPOSTER at inference time</h3>
	<p>Best-of-N selection, weak-span detection, targeted rewrite-and-rescore, and discriminator-aware reranking. Inference-time IMPOSTER is likely the fastest route from “competitive with Opus 4.7” to “reliably preferred over Opus 4.7” on delivered output.</p>
	</div>
	</div>
	</section>

	<section>
	<h2>The calibration gate, term by term</h2>
	<p>Step 3 above is a single sentence with a lot packed into it: “Before any RL run starts, the discriminator must demonstrate calibrated <code>P(human)</code>, low ECE, adequate held-out accuracy, and a non-saturated, non-bimodal score distribution. Failing the gate blocks training.” Each phrase is doing real work; here is what each one means and why it has veto power over the next RL run.</p>

	<div class="grid">
	<div class="step">
	<h3>Calibrated <code>P(human)</code></h3>
	<p>The discriminator outputs a probability that a given span was written by a human. “Calibrated” means those probabilities mean what they say: spans the discriminator scores at <code>0.7</code> are human about 70% of the time, spans scored at <code>0.9</code> about 90%, and so on. Without calibration, the number is just an ordering — fine for ranking, useless as a reward signal whose magnitude RL will chase.</p>
	</div>

	<div class="step">
	<h3>ECE — expected calibration error</h3>
	<p>The standard scalar measure of how far a probabilistic classifier is from being calibrated. Bin predictions by their stated confidence, compute the gap between stated confidence and observed accuracy in each bin, and average those gaps weighted by bin size. Low ECE means the discriminator’s <code>P(human)</code> can be trusted as a probability rather than just a score; high ECE means RL will optimize a number that does not correspond to reality.</p>
	</div>

	<div class="step">
	<h3>Held-out accuracy</h3>
	<p>Accuracy on labeled human/model spans the discriminator never saw during training. “Adequate” means high enough that the discriminator is genuinely separating elite human prose from model prose, not memorizing its training set. If held-out accuracy is weak, calibration on the training distribution is a mirage — the gate must clear both bars on data the discriminator has not seen.</p>
	</div>

	<div class="step">
	<h3>Score distribution</h3>
	<p>The histogram of <code>P(human)</code> values the discriminator assigns across a representative evaluation set. Its shape is diagnostic: a healthy discriminator spreads its scores across the unit interval, with mass concentrated where the underlying spans actually fall. Inspecting the distribution catches pathologies that aggregate metrics like accuracy and ECE can hide.</p>
	</div>

	<div class="step">
	<h3>Saturated score distribution</h3>
	<p>Scores piled up against <code>0</code> or <code>1</code> with very little mass in between. A saturated discriminator has effectively collapsed to a hard classifier and stops giving the generator useful gradient: every candidate span looks either obviously human or obviously model, so there is nothing to climb. RL against a saturated reward devolves into chasing the few spans near the boundary, which is exactly the reward-hacking regime we just exited.</p>
	</div>

	<div class="step">
	<h3>Bimodal score distribution</h3>
	<p>Two sharp peaks — typically one near <code>0</code> and one near <code>1</code> — with a hollow middle. Even when overall accuracy looks fine, bimodality means the discriminator is not expressing graded confidence: borderline spans, which are most of what the generator produces, get snapped to one extreme or the other. The reward surface becomes a cliff instead of a slope, and small generator changes produce huge, noisy reward swings.</p>
	</div>

	<div class="step">
	<h3>Calibration gate</h3>
	<p>A pre-RL checkpoint test that bundles all of the above — calibrated <code>P(human)</code>, low ECE, adequate held-out accuracy, and a well-shaped score distribution — into a single pass/fail decision. The gate runs against a fixed evaluation set with thresholds set in advance, so “is this discriminator good enough to train against?” is a yes/no question rather than a judgment call made under schedule pressure.</p>
	</div>

	<div class="step">
	<h3>Why failing the gate blocks training</h3>
	<p>RL faithfully optimizes whatever reward you give it. If the reward is miscalibrated, saturated, or bimodal, RL will drive the generator toward whatever quirk of the discriminator produces high scores — not toward elite human prose. That is precisely the failure mode the Apr 2026 run exhibited. Spending another long compute budget on RL against an un-vetted discriminator would reproduce it. The gate is cheap; another reward-hacked run is not.</p>
	</div>
	</div>
	</section>

	<section>
	<h2>Conditioning: what we already had, and what was missing</h2>
	<p>The Apr 2026 pipeline did carry useful structural signal — we just never serialized it into model-visible context. The data cleaner inferred <code>source</code> and <code>doc_type</code>, span replacement preserved <code>source_id</code> and <code>span_type</code>, and the discriminator saw the <code>before</code>/<code>after</code> tokens around the marked span. Conceptually those fields exist; operationally they were leaving most of their value on the floor.</p>
	<p>The actual discriminator input collapsed to roughly <code>before <SPAN> span </SPAN> after</code>, and the generator prompt to <code>before <FILL> after</code>. That gives local lexical context but never names the artifact: there is no way for the model to know whether it is writing into a Supreme Court reply brief, a routine demand letter, or a contract recital.</p>

	<div class="grid">
	<div class="step">
	<h3>Already present in the pipeline</h3>
	<p><code>source</code>, <code>doc_type</code>, <code>source_id</code>, <code>span_type</code>, and the surrounding before/after context. These are enough for a credible first pass at source-conditioned training — provided we actually emit them into the prompt and the discriminator input rather than only logging them.</p>
	</div>

	<div class="step">
	<h3>Missing or too coarse today</h3>
	<p>Source tier (elite vs ordinary), court, jurisdiction, document role, procedural posture, section role inside the document, author role, target citation density, and an explicit quality tier. This is the metadata that separates “legalese” from “elite appellate prose,” and it is exactly what we need to hold Opus 4.7 to.</p>
	</div>
	</div>

	<p>The concrete change is to serialize a compact metadata header into both the generator prompt and the discriminator input, so the same conditioning shapes both sides of the game:</p>
	<pre><META doc_type="appellate_brief"
	source_tier="elite"
	court="9th_cir"
	jurisdiction="federal"
	posture="summary_judgment_appeal"
	section="argument"
	span_type="paragraph"
	quality_tier="expert" /></pre>
	<p>Discriminator examples become <code><META ... /> before <SPAN> span </SPAN> after</code>; generator examples become <code><META ... /> before <FILL> after</code>. The discriminator now scores whether the span belongs in <em>this</em> metadata-conditioned context, and the generator learns to write toward the same target instead of an unspecified average.</p>
	<p>For fields we cannot extract reliably, fall back to <code>unknown</code> rather than guessing — honest sparsity beats spurious labels. The point is that whatever high-confidence metadata we have makes it into the model's view, and into the stratified evaluation report so we can see where the discriminator is and isn’t calibrated.</p>
	</section>

	<section class="callout">
	<h2>Where Opus 4.7 fits</h2>
	<p><strong>Opus 4.7 stays held out.</strong> Use it as the blinded benchmark in pairwise comparisons against IMPOSTER outputs to validate the discriminator and to gate checkpoint promotion — never as the training reward itself. The reward signal remains the paper-faithful one: calibrated, source-conditioned <code>P(human)</code> against an elite legal-prose reference distribution.</p>
	</section>

	<section class="milestone">
	<h2>Milestone we should be able to point at</h2>
	<p>Ship <code>legal_elite_discriminator_v1</code> together with a locked Opus 4.7 benchmark suite. Then resume RL from step6500 against the paper-faithful objective: calibrated source-conditioned <code>P(human)</code>, gated curriculum advancement, explicit diversity pressure, and discriminator-guided inference-time rewriting on top.</p>
	</section>

	<section>
	<h2>What two weeks on 16 B200s can realistically buy</h2>
	<p>The useful estimate is not peak FLOPs. It is what this stack can actually complete when one B200 machine trains and the other serves generation through vLLM. The Apr 2026 run completed roughly 10k successful short-span RL steps in about 16 hours, or about 5.6 seconds per successful step, but it was still optimizing a discriminator target that was not sharp enough. Running that same loop longer would mostly produce a cleaner reward-hacking failure.</p>
	<p>Two weeks across 16 B200s is about <code>5,376</code> GPU-hours. That is enough for a serious IMPOSTER v1.5 push, provided most of the time goes into target quality, calibration, curriculum, and inference-time selection rather than brute-forcing the old reward.</p>

	<div class="grid">
	<div class="step">
	<h3>What should be feasible</h3>
	<p>Build <code>legal_elite_discriminator_v1</code>, train and calibrate it on a sharper elite corpus, run several LoRA RL cycles from step6500, advance at least into paragraph-level training, possibly into section-level, and evaluate against a locked Opus 4.7 benchmark.</p>
	</div>

	<div class="step">
	<h3>Where the gains should show up first</h3>
	<p>Narrow legal span and paragraph drafting should improve before broad legal reasoning does. The most plausible win condition is delivered output after best-of-N, weak-span detection, targeted rewrite-and-rescore, and discriminator-aware reranking.</p>
	</div>

	<div class="step">
	<h3>What is plausible</h3>
	<p>Competitive or better-than-Opus 4.7 results on a constrained legal-prose benchmark where the task is to write an argument, memo section, or polished legal paragraph from supplied facts and law, especially with inference-time IMPOSTER enabled.</p>
	</div>

	<div class="step">
	<h3>What is not realistic yet</h3>
	<p>A broadly superior one-shot legal model. A LoRA on Qwen2.5-72B can improve prose, register, structure, and local legal drafting behavior, but it will not automatically close frontier-model gaps in reasoning, citation strategy, or broad legal judgment.</p>
	</div>
	</div>

	<p>The bottleneck is not raw GPU supply. It is whether the discriminator actually represents elite legal prose and remains trustworthy under optimization. If that target is right, two weeks is enough for a meaningful result. If that target is wrong, two weeks simply gives the generator more time to exploit it.</p>
	<p><strong>Realistic two-week target:</strong> a system that beats Opus 4.7 on a locked span/paragraph-level legal drafting benchmark using IMPOSTER inference-time reranking and rewrite, with a trained LoRA that is visibly better than step6500 but not yet a generally superior one-shot legal model.</p>
	</section>

	<footer>
	Drawn from the IMPOSTER paper and the Apr 2026 experiment review.
	</footer>
	</main>
	</body>
	</html>
No results found