Template Learning System — Revised Plan

tender-marinating-stearns.md

Template Learning System — Revised Plan

Based on coworker's proposal, adjusted for actual codebase state. Core insight preserved: separate quality principles (what makes a template good) from style examples (what makes a template distinctive).

---

Phase 1 — Fix Signal Quality (Foundation)

1a. Reason capture on star rating

The reason field exists in LanceDB (vectordb.ts:33) and is accepted by the save API (templateExamples.ts:241), but InlineStarRating.tsx only captures the star click. Add an optional text input that appears after clicking a star.

Files:

• frontend/src/components/template-lab/InlineStarRating.tsx — After star click, expand to show a small text input + "Save" button. Current flow: click star → immediately fires onRate(n). New flow: click star → show input → press Enter or click Save → fires onRate(n, reason).
• frontend/src/hooks/template-lab/useTemplateRating.ts — Update rateMessage signature to accept optional reason: string, pass it through to saveTemplateExample() at line 75.

Not required. Just surfaced. Even 3-5 word answers help.

1b. Enrich extractStructuralHints

Current output (exampleInjection.ts:45-76) is a stat line: "4 photos, 12s, fadeIn+slideUp, Portfolio, 5★". The full template JSON is in ex.template. Pull more signal without adding an LLM call.

File: backend/src/lib/exampleInjection.ts

Add to the per-example summary:

• Color palette range (extract dominant fill_color values from shapes/backgrounds)
• Font size hierarchy (headline vs body sizes from extractTemplateCategorySummary)
• Element density (total element count / duration)
• Animation timing patterns (stagger intervals, entrance delays)
• The reason field if populated (e.g., "clean typography, strong contrast")

This gives the architect real design intelligence at zero additional cost. The extraction helpers already exist in templateStructure.ts — reuse extractTemplateCategorySummary.

1c. Diversity enforcement at retrieval

File: backend/src/lib/exampleInjection.ts — findSimilarExamples() (lines 125-178)

After selecting top 2 good candidates by score, check cosine distance between them. If distance < 0.15 (too similar), keep #1, skip #2, pick next highest-scoring candidate that's sufficiently different. Two nearly-identical examples waste context.

const good1 = goodCandidates[0];
let good2 = goodCandidates[1];
// If good2 is too similar to good1, find next diverse candidate
if (good2 && cosineSimilarity(good1.vector, good2.vector) > 0.85) {
  good2 = goodCandidates.find(c => c !== good1 && cosineSimilarity(good1.vector, c.vector) <= 0.85);
}

1d. Category hard-filtering at retrieval

File: backend/src/lib/exampleInjection.ts — findSimilarExamples()

Currently vector search returns all examples regardless of category. Add the architect's archetype as a parameter. Before scoring, filter candidates to same archetype category mapping:

archetype mapping:
  carousel, slideshow → "Portfolio"
  hero-showcase → "Lead Gen" or "Proof & Trust"
  before-after → "Portfolio"
  grid → "Services"

Pass archetype from generateTemplate.ts after the architect step. If filtering yields < 3 candidates, fall back to unfiltered.

1e. Enhance curation UI

ExamplesPanel.tsx already has: list view, category tabs, delete with confirmation, copy to drafts, hover preview. Missing: inline editing of ratings/reasons, showing bad examples (currently filters to rating >= 3).

File: frontend/src/components/template-lab/ExamplesPanel.tsx

• Add a toggle to show all ratings (not just >= 3)
• Add inline "edit reason" — click reason text, turns into input, save on Enter
• Add "re-rate" — click stars to update rating (calls new PATCH endpoint)

File: backend/src/routes/templateExamples.ts

• Add PATCH /template-lab/examples/:id — update rating and/or reason on existing example (read-then-write pattern like incrementExampleUsage)

---

Phase 2 — Principles Layer (The Real Investment)

This is the architectural change. After accumulating ~30-40 rated examples, extract stable design principles that apply to all future generations.

2a. Principles table in SQLite

File: backend/src/db/schema.ts

export const designPrinciples = sqliteTable("design_principles", {
  id: text("id").primaryKey(),
  principle: text("principle").notNull(),     // "Headlines contrast at min 4.5:1"
  source: text("source").notNull(),           // "extracted" | "manual"
  exampleCount: integer("example_count"),      // how many examples informed this
  confidence: text("confidence"),              // "high" | "medium" | "low"
  createdAt: text("created_at").default(sql`CURRENT_TIMESTAMP`),
  updatedAt: text("updated_at").default(sql`CURRENT_TIMESTAMP`),
});

Not in LanceDB — these aren't for vector search, they're for direct injection.

2b. Extraction script/endpoint

File: backend/src/routes/templateExamples.ts (new endpoint) or backend/scripts/extract-principles.ts

1. Load all examples with rating >= 4
2. For each, include: template JSON, rating, reason, structural summary
3. Send to Claude with: "Extract the design principles that appear consistently across these high-rated templates. What do they have in common? Output concrete, actionable rules."
4. Parse response into individual principles
5. Store in SQLite design_principles table

Run manually from DevTools or as a script. Not automated — humans review before persisting.

2c. Inject principles into every generation

File: backend/src/lib/builder.ts — buildBuilderUserPrompt()

New section injected before examples:

## QUALITY CONSTRAINTS (always apply)
- Headlines contrast against photo backgrounds at minimum 4.5:1 ratio
- When using 4+ photos, stagger entry animations by 0.3-0.5s intervals
- Brand color appears exactly once as an anchor element
- ...loaded from design_principles table...

These are injected into EVERY generation regardless of examples. They encode accumulated taste without locking in a specific aesthetic.

2d. Restructure prompt injection

Current: examples mixed with everything else. New layered structure in the builder user prompt:

1. Quality constraints → from principles table (always injected)
2. Anti-patterns → from bad examples in vector DB (always injected)
3. Style reference → from good examples (injected only if similarity > threshold)
4. Creative brief → user input (primary driver, never overridden)

Style reference becomes optional. If the user's request is novel (no similar examples above threshold), that's fine — quality constraints still apply.

File: backend/src/lib/prompts/buildSystemPrompt.ts — renderSimilarExamples() (lines 121-151)

Split into renderQualityConstraints(), renderAntiPatterns(), renderStyleReferences(). Each injected at its proper position in the prompt.

---

Phase 3 — Convergence Protection (Build Alongside Phase 2)

3a. Tune recency decay

File: backend/src/lib/exampleInjection.ts — scoreExample() (line 107)

Change from 10% weight / 180-day horizon to 20% weight / 60-day horizon:

// Before:
const recency = Math.max(0, 1 - daysSinceCreated / 180);
const score = similarity * 0.35 + ratingScore * 0.35 + effectiveness * 0.2 + recency * 0.1;

// After:
const recency = Math.max(0, 1 - daysSinceCreated / 60);
const score = similarity * 0.30 + ratingScore * 0.30 + effectiveness * 0.20 + recency * 0.20;

3b. "Explore" mode toggle

Add a toggle in Template Lab: "Use my library / Explore freely."

• Library mode (default): inject principles + anti-patterns + style examples (current behavior + principles)
• Explore mode: inject ONLY quality constraints (principles). No style examples. Clean creative path.

Files:

• frontend/src/components/template-lab/ChatPanel.tsx — toggle in + menu (like photoSource toggle)
• frontend/src/components/TemplateLab.tsx — new exploreMode state, pass through API
• backend/src/routes/templateLab/shared.ts — add exploreMode?: boolean
• backend/src/lib/generateTemplate.ts — when exploreMode, skip findSimilarExamples(), inject only principles

---

Phase 4 — External User Readiness (When Contractors Arrive)

Not building now. Document for later:

• Per-user taste profiles: Weight examples similar to contractor's saved templates
• Implicit signals: Track generate→bookmark (strong positive), generate→abandon (negative), generate→post (very strong positive)
• Engagement as confirmation: Late.dev metrics confirm quality model, don't drive it

---

What NOT to Build

• A/B testing framework (insufficient volume)
• Model fine-tuning (RAG + principles outperforms at this scale)
• Automated promotion/demotion (keep humans in curation loop)
• Engagement-driven generation quality (feedback loop too slow)

---

Implementation Order

#	Item	Files	Depends On
1	Reason capture	InlineStarRating.tsx, useTemplateRating.ts	—
2	Enrich structural hints	exampleInjection.ts	—
3	Diversity enforcement	exampleInjection.ts	—
4	Category hard-filtering	exampleInjection.ts, generateTemplate.ts	—
5	Curation UI enhancements	ExamplesPanel.tsx, templateExamples.ts	—
6	Principles SQLite table	schema.ts	—
7	Principles extraction	new endpoint/script	6, needs ~30+ examples
8	Principles injection	builder.ts, buildSystemPrompt.ts	7
9	Restructure prompt layers	buildSystemPrompt.ts	8
10	Recency decay tuning	exampleInjection.ts	—
11	Explore mode toggle	ChatPanel, TemplateLab, chat.ts, generateTemplate.ts	8

Items 1-5 are independent and can be done in any order. Items 6-9 are sequential. Items 10-11 can be done anytime after 8.

---

Verification

1. Reason capture: Rate a template, type a reason, verify it appears in ExamplesPanel and in the LanceDB record
2. Enriched hints: Check backend logs for richer structural hint output during generation
3. Diversity: Rate 3+ similar templates, generate — verify two different examples selected (check logs)
4. Category filtering: Generate a carousel prompt — verify only Portfolio examples injected
5. Curation UI: Edit a reason, re-rate an example, toggle to show bad examples
6. Principles: Run extraction with 30+ examples, review output, inject into generation, verify in prompt audit
7. Explore mode: Toggle on, generate — verify no style examples in prompt, only principles
8. npm run typecheck clean in both frontend and backend
9. Existing tests pass (exampleInjection tests, vectordb tests)