PLS job 0ce49729 AI prediction repo change handoff

acceptance-tests.md

Acceptance Tests

Test 1: Repo Availability Gate

Given this workspace has no .git, when the job completes, then it must not claim a PR exists.

Pass:

• artifact states repo is missing.
• next worker action is repo_change/github_pr after backend repo attachment.
• no fake GitHub PR URL is reported.

Test 2: Migration Completeness

Given backend repo is attached, when migration is implemented, then all five scorecard objects must exist.

Pass:

• weekly_prediction_scorecards
• scorecard_metric_snapshots
• scorecard_repair_backlog
• scorecard_approval_events
• scorecard_dispatch_events

Test 3: API Contract

Given migration is live, when APIs are implemented, then scorecard card, approval, line summary, dispatch-next, and backlog update endpoints must pass.

Pass:

• card endpoint returns gate, metrics, backlog, evidence refs, next action.
• approve endpoint rejects non-owner.
• line-summary endpoint returns short action-oriented copy.
• dispatch-next maps gate deterministically.

Test 4: UI States

Given sample payloads, when dashboard card renders, then all three states must work.

Pass:

• ship_weekly_scorecard shows publish/cadence.
• repair_first shows repair backlog and blocks publish.
• blocked shows missing data/owner decision gap.

Test 5: People Sync

Given final gate, when LINE sync is prepared, then each person receives only their required signal.

Pass:

• Louis receives ship/repair/block request.
• zihrou receives rubric/reviewer backlog only.
• iron receives source adapter backlog only.
• raw evidence remains in backend/artifact.

Test 6: Completion Evidence

Given this handoff pack is complete, then durable artifact, uploaded files, readiness model, E2E verification, people sync, learning memory, owner/due/acceptance, and decision record must all exist.

Pass:

• public artifact URL returns HTTP 200.
• 8 files uploaded to PLS.
• learning memory JSON validates.
• artifact JSON includes required PLS kinds.

artifact-url-or-pr.md

Artifact URL or PR

Durable primary artifact:

https://gist.github.com/esz135888/6a6731e5b83bed946fdc332c29b29401

Gist id:

6a6731e5b83bed946fdc332c29b29401

PR status:

No GitHub PR was created because /Users/louis/Desktop/ai360-vscode-mp-1 is not a Git repository. This is intentionally recorded to avoid fake GitHub success. Next step is attaching the PLS backend repo and dispatching a repo_change or github_pr worker.

Verification:

• Public Gist URL responds with HTTP 200 after redirects.
• File list includes handoff HTML, production brief, data model, acceptance tests, decision record, learning memory, market sources, and this artifact record.

data-model.md

Repo Change Data Model and API Contract

Migration Targets

weekly_prediction_scorecards

Field	Type	Required	Notes
id	uuid	yes	primary id
project_id	uuid	yes	PLS project
week_start	date	yes	scorecard week
calibration_run_id	uuid	yes	D7 run
correction_batch_id	uuid	yes	D14 routes
adoption_gate	text	yes	ship_weekly_scorecard, repair_first, blocked
owner_user_id	uuid	yes	Louis
status	text	yes	draft, pending_approval, approved, published, blocked
decision_record_ref	text	yes	artifact/PR record

scorecard_metric_snapshots

Field	Type	Required	Notes
id	uuid	yes	primary id
scorecard_id	uuid	yes	parent
hit_rate	numeric	yes	quality
miss_rate	numeric	yes	quality
unknown_rate	numeric	yes	threshold <0.25
routed_non_hit_rate	numeric	yes	threshold 1.0
unresolved_gap_rate	numeric	yes	threshold <=0.2
reviewer_agreement_rate	numeric	yes	target >=0.8
snapshot_hash	text	yes	audit hash

scorecard_repair_backlog

Field	Type	Required	Notes
id	uuid	yes	backlog id
scorecard_id	uuid	yes	parent
route_type	text	yes	source/rubric/prompt/owner follow-up
owner_user_id	uuid	yes	iron/zihrou/Louis/worker
due_at	timestamptz	yes	due
acceptance_rule	text	yes	done definition
status	text	yes	open/in_progress/verified/closed

scorecard_approval_events

Field	Type	Required	Notes
id	uuid	yes	event id
scorecard_id	uuid	yes	parent
actor_user_id	uuid	yes	owner/delegate
final_gate	text	yes	ship/repair/block
rationale	text	yes	decision reason
created_at	timestamptz	yes	audit

scorecard_dispatch_events

Field	Type	Required	Notes
id	uuid	yes	event id
scorecard_id	uuid	yes	parent
dispatch_kind	text	yes	project_runner/scorecard_improvement/repo_change
dispatch_prompt	text	yes	next worker instruction
status	text	yes	queued/claimed/completed/failed

APIs

API	Method	Purpose
/ai-prediction/scorecards/:id/card	GET	dashboard card payload
/ai-prediction/scorecards/:id/approve	POST	Louis final gate approval
/ai-prediction/scorecards/:id/line-summary	GET	short LINE copy
/ai-prediction/scorecards/:id/dispatch-next	POST	create next Hermes job
/ai-prediction/scorecards/:id/backlog/:route_id	PATCH	update repair backlog

Sync and Permissions

• Worker can recommend gate and dispatch prompt, but cannot approve final gate.
• Louis or delegated owner approves final gate.
• zihrou updates rubric/reviewer backlog.
• iron updates source adapter backlog.
• LINE sync must use summary endpoint and never include raw evidence JSON.
• Every write includes actor id, worker id where applicable, timestamp, metric snapshot hash, run ids, route ids, and decision record ref.

decision-record.md

Decision Record: Repo Change Handoff Instead of Fake PR

Date: 2026-05-24
Status: Recommended
Owner: Louis
Target worker: repo_change / github_pr

Problem

The dashboard implementation spec is ready, but the current workspace is not a Git repository. Creating or claiming a PR from here would be dishonest and would violate the rule against pretending GitHub success.

Options Considered

Option A: Pretend a PR exists

Rejected. This would create false delivery evidence.

Option B: Produce another dashboard/spec pack

Rejected. The chain already has dashboard implementation specs; repeating them would not move production forward.

Option C: Produce repo-change-ready handoff

Recommended. It explicitly records the repo blocker, defines migration/API/UI/permission tests, and gives the next worker a concrete repo_change task.

Recommendation

Adopt Option C. Attach the PLS backend repo and dispatch a repo_change/github_pr worker using this handoff as the implementation contract.

Landing Path

1. Attach or clone the PLS backend repo.
2. Create a branch for AI prediction weekly scorecard card.
3. Implement migrations, APIs, UI card, LINE summary, and dispatch-next.
4. Run backend/frontend tests.
5. Open PR and reference this handoff artifact.

Feedback Needed If Not Adopted

• Backend repo path or GitHub repo is unavailable.
• Scorecard data should live in another service.
• LINE summary is owned by another worker.
• Approval gate should not live in PLS.
• Project should pause before implementation.

learning-memory.json

{
  "job_id": "0ce49729-3b04-4771-84d8-a3460552316c",
  "project_topic": "AI prediction verification module for signals and action-item evidence",
  "current_artifact": "Repo Change Handoff Pack",
  "previous_artifacts": [
    "PLS Weekly Scorecard Dashboard Implementation Pack",
    "D30 Weekly Scorecard Adoption Gate",
    "D14 Correction Router",
    "D7 Calibration Run Control Tower"
  ],
  "owner": "Louis",
  "due": "2026-06-21",
  "repo_status": {
    "workspace_has_git_repo": false,
    "verified_by": "git status returned not a git repository",
    "do_not_claim_pr": true
  },
  "market_learning": [
    "AI observability value comes from connecting eval evidence to production action workflows.",
    "The next bottleneck is implementation in the PLS backend, not more artifact design.",
    "Repo handoff must preserve approval and audit boundaries before dashboard release."
  ],
  "next_worker_rule": {
    "if_backend_repo_available": "Run repo_change/github_pr implementation from this handoff.",
    "if_backend_repo_missing": "Ask PLS/Louis to attach backend repo before continuing.",
    "if_schema_exists": "Implement API/UI/LINE/dispatch-next only.",
    "if_schema_missing": "Start with migrations listed in data-model.md.",
    "do_not_repeat_static_pack": "Do not create another dashboard artifact unless implementation feedback changes the requirements."
  },
  "acceptance_gate": {
    "no_fake_pr": true,
    "migration_complete": true,
    "api_contract_complete": true,
    "owner_approval_enforced": true,
    "line_short_only": true,
    "dispatch_next_deterministic": true
  }
}

production-brief.md

AI 預測驗證 Repo Change Handoff Pack

場景

上一輪已產出 PLS Weekly Scorecard Dashboard Implementation Pack。本輪檢查工作區後確認目前路徑不是 Git repo，無法誠實建立 PR。因此本輪產出 repo-change-ready handoff：把 migration、API、UI、permissions、LINE、worker dispatch 拆到下一個 repo_change worker 可直接開 PR 的程度。

Owner: Louis
Target worker: repo_change / github_pr
Due: 2026-06-21
Primary artifact: repo-change-handoff-console.html

30 天發展路徑

時點	成果	驗收
D1	將 handoff 轉成 repo issue/branch。	backend repo 與 owner 明確。
D7	DB migration + scorecard card API 完成。	card payload 可回傳三態。
D14	UI card + LINE summary + dispatch-next 完成。	owner approval、短摘要、下一輪派工可跑。
D30	weekly scorecard operating cadence。	Louis 每週 approve gate，PLS 自動派 repair/publish。

目的到目的 E2E

原始目的：核對上次 review 的 AI prediction 是否命中。
本輪目的：讓上一輪 dashboard spec 能真正進 repo PR，而不是停在 artifact。

E2E:

1. D7/D14/D30 artifacts 形成 scorecard evidence chain。
2. Repo worker 建 migration/API/UI/dispatch。
3. Louis 在 PLS card approve gate。
4. zihrou/iron 依 backlog 修 rubric/source。
5. PLS worker 根據 dispatch-next 自動派 publish、repair、rerun 或 blocked decision。

價值 / 錢路徑

• 營收：把可信 AI prediction pattern 從 artifact 推到實際 PLS 後台，才能擴大到業務/客訴/特殊申請。
• 省成本：一次 PR 串起 scorecard、LINE、dispatch，減少人工接力。
• 降風險：owner approval 與 audit fields 防止未驗證 AI 結論進入管理週報。
• 提高轉換：Louis 只需看 PLS card 做 ship/repair/block 決策。
• 釋放人力：worker dispatch rules 讓後續任務自動化。

提升人的能力

• Louis：從 artifact review 升級為 PLS 後台 gate approval。
• zihrou：接 rubric backlog 而不是讀整份報告。
• iron：接 source adapter backlog 而不是被動補資料。
• repo worker：有明確 file area、API、schema、tests，不必重新拆需求。

Solution Stack

層	內容
脈絡框架	D7/D14/D30 evidence -> repo PR -> PLS card -> people sync -> dispatch-next。
作業流程	attach backend repo -> migrate schema -> implement APIs -> render card -> test gates。
資料 / DB	weekly_prediction_scorecards、scorecard_metric_snapshots、scorecard_repair_backlog、scorecard_approval_events、scorecard_dispatch_events。
可操作工具	HTML handoff console、schema/API spec、acceptance tests、decision record、learning memory。
驗收指標	三態 render、owner approval enforced、LINE short only、dispatch deterministic、audit complete。
採用升級	下一輪不再產 static artifact；應 attach repo and open PR。

市場脈絡

成熟 AI observability/evaluation 流程會將 traces、evals、annotations、monitoring、action workflow 接在同一系統內。此 handoff 將該做法落到 PLS：scorecard 不只是 dashboard，而是 approval + audit + people sync + worker dispatch。

repo-change-handoff-console.html

<!doctype html>
<html lang="zh-Hant">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>AI 預測驗證 Repo Change Handoff Console</title>
  <style>
    :root{--ink:#17212b;--muted:#617080;--line:#d9dfe7;--bg:#f6f8fb;--panel:#fff;--blue:#2457d6;--green:#14805a;--amber:#a35f00;--red:#b42318;--soft:#eef2f7}
    *{box-sizing:border-box}body{margin:0;background:var(--bg);color:var(--ink);font-family:Inter,ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;line-height:1.5}
    header{background:#fff;border-bottom:1px solid var(--line);padding:22px clamp(18px,4vw,48px)}main{padding:22px clamp(18px,4vw,48px) 48px;display:grid;gap:16px}
    h1{margin:0;font-size:clamp(24px,3vw,36px);letter-spacing:0}h2{margin:0 0 12px;font-size:18px}h3{margin:0 0 8px;font-size:15px}p{margin:0 0 10px}
    .sub{color:var(--muted);max-width:1120px;margin-top:8px}.tag{display:inline-flex;align-items:center;height:24px;padding:0 8px;border:1px solid var(--line);border-radius:999px;background:#fbfcfe;color:var(--muted);font-size:12px;margin-right:6px}
    .grid{display:grid;gap:16px}.cols4{grid-template-columns:repeat(4,minmax(0,1fr))}.cols3{grid-template-columns:repeat(3,minmax(0,1fr))}.cols2{grid-template-columns:repeat(2,minmax(0,1fr))}
    .panel{background:var(--panel);border:1px solid var(--line);border-radius:8px;padding:16px}.metric{min-height:112px;display:flex;flex-direction:column;justify-content:space-between}.label{font-size:13px;color:var(--muted)}.value{font-size:30px;font-weight:760}.ok{color:var(--green)}.warn{color:var(--amber)}.stop{color:var(--red)}
    table{width:100%;border-collapse:collapse;font-size:13px}th,td{border-bottom:1px solid var(--line);padding:10px 8px;text-align:left;vertical-align:top}th{background:#fbfcfe;color:var(--muted)}code{background:var(--soft);border-radius:4px;padding:2px 5px;font-size:12px}
    ul{padding-left:18px;margin:0}li{margin:6px 0}.state{border-left:4px solid var(--blue);padding-left:12px}.state.warn{border-color:var(--amber)}.state.stop{border-color:var(--red)}
    @media(max-width:980px){.cols4,.cols3,.cols2{grid-template-columns:1fr}}
  </style>
</head>
<body>
  <header>
    <h1>Repo Change Handoff Console</h1>
    <p class="sub">此工作區沒有 Git repo，因此本輪把 PLS weekly scorecard dashboard 轉成 repo-change-ready handoff：下一個 worker 拿到後可直接在後台 repo 建 migration、API、UI card、LINE summary endpoint 與 dispatch-next。</p>
    <p><span class="tag">Owner: Louis</span><span class="tag">Target worker: repo_change</span><span class="tag">Due: 2026-06-21</span><span class="tag">Status: PR-ready spec</span></p>
  </header>
  <main>
    <section class="grid cols4">
      <div class="panel metric"><span class="label">Repo Status</span><span class="value warn">missing</span><span class="label">current workspace has no .git</span></div>
      <div class="panel metric"><span class="label">Build Scope</span><span class="value ok">4</span><span class="label">migration / API / UI / worker dispatch</span></div>
      <div class="panel metric"><span class="label">Acceptance</span><span class="value ok">6</span><span class="label">backend-ready tests</span></div>
      <div class="panel metric"><span class="label">Next Action</span><span class="value stop">PR</span><span class="label">attach backend repo and implement</span></div>
    </section>
    <section class="panel">
      <h2>30 天路徑</h2>
      <div class="grid cols4">
        <div class="state"><h3>D1</h3><p>把本 handoff 轉為 repo issue/branch，確認 PLS backend module owner。</p></div>
        <div class="state"><h3>D7</h3><p>完成 DB migration、scorecard card payload API、line-summary endpoint。</p></div>
        <div class="state"><h3>D14</h3><p>完成 dashboard card UI、owner approval、dispatch-next 串接與權限測試。</p></div>
        <div class="state"><h3>D30</h3><p>進入週節奏：scorecard 自動重算，Louis approve gate，PLS 派下輪 repair/publish。</p></div>
      </div>
    </section>
    <section class="grid cols2">
      <div class="panel">
        <h2>Purpose-to-Purpose E2E</h2>
        <ul>
          <li>原始目的：核對 AI review prediction 是否命中。</li>
          <li>產出物：repo-change-ready migration/API/UI/dispatch spec。</li>
          <li>人採用：工程 worker 開 PR；Louis approve gate；zihrou/iron 修 backlog。</li>
          <li>指標改善：prediction accuracy 可週更、unknown/source gap 降低、人工 review 時間下降。</li>
        </ul>
      </div>
      <div class="panel">
        <h2>Value / Money Path</h2>
        <ul>
          <li>營收：讓可信 AI prediction pattern 進入高價流程，而不是停留在 artifact。</li>
          <li>省成本：工程 PR 一次把 scorecard、LINE、worker dispatch 串起來，減少人工接力。</li>
          <li>降風險：owner approval 與 audit 防止未驗證 AI 結論出現在管理週報。</li>
          <li>釋放人力：dispatch-next 讓 PLS 自動派 repair/rerun/publish。</li>
        </ul>
      </div>
    </section>
    <section class="panel">
      <h2>PR Work Breakdown</h2>
      <table>
        <tr><th>File Area</th><th>Change</th><th>Acceptance</th></tr>
        <tr><td>DB migration</td><td>新增 scorecard views/tables and audit fields</td><td>schema can store gate, metrics, evidence, approval</td></tr>
        <tr><td>API</td><td><code>GET /scorecards/:id/card</code>, <code>POST /approve</code>, <code>GET /line-summary</code>, <code>POST /dispatch-next</code></td><td>returns deterministic card and action payloads</td></tr>
        <tr><td>UI</td><td>PLS backend card with gate header, metrics, backlog, evidence drawer</td><td>renders ship/repair/block states</td></tr>
        <tr><td>Worker dispatch</td><td>map final gate to next Hermes job type</td><td>publish, repair source, repair rubric, rerun, manual decision</td></tr>
        <tr><td>Permissions</td><td>Louis approve, zihrou rubric, iron source, worker recommend only</td><td>non-owner approval rejected</td></tr>
      </table>
    </section>
    <section class="grid cols3">
      <div class="panel"><h2>Data / API</h2><p>Primary objects: <code>weekly_prediction_scorecards</code>, <code>scorecard_metric_snapshots</code>, <code>scorecard_repair_backlog</code>, <code>scorecard_approval_events</code>, <code>scorecard_dispatch_events</code>.</p></div>
      <div class="panel"><h2>Permissions / Audit</h2><p>Every approval and dispatch stores actor, timestamp, metric snapshot hash, calibration run id, correction batch id, model/prompt version, and decision record ref.</p></div>
      <div class="panel"><h2>People Sync</h2><p>LINE outer message asks for one signal only: Louis ship/repair/block; zihrou rubric fix; iron source adapter fix. Raw evidence stays in backend.</p></div>
    </section>
    <section class="panel">
      <h2>LINE 草稿</h2>
      <p>AI 預測驗證 dashboard 已整理成 repo-change handoff。請 Louis 指派 PLS backend repo/owner；下一輪 worker 應開 PR 實作 scorecard card、approve gate、LINE summary、dispatch-next。驗收：三態可 render、owner approval enforced、LINE 短摘要、dispatch deterministic。</p>
    </section>
  </main>
</body>
</html>

sources.md

Market Context Sources

Checked on 2026-05-24 Asia/Taipei.

Sources

• OpenTelemetry GenAI observability article, published 2026-05-14: https://opentelemetry.io/blog/2026/genai-observability/
• OpenTelemetry GenAI semantic conventions: https://opentelemetry.io/docs/specs/semconv/gen-ai/
• Arize Phoenix documentation: https://arize.com/docs/phoenix/
• Arize Phoenix evals with explanations: https://arize.com/docs/phoenix/learn/evaluation/evals-with-explanations
• Evidently OSS: https://www.evidentlyai.com/evidently-oss
• Evidently documentation overview: https://docs.evidentlyai.com/docs/library/overview
• LangSmith evaluations: https://www.langchain.com/langsmith/evaluation

Takeaway

Mature AI evaluation systems connect traces, evals, reviewer annotations, monitoring, and production action. This pack moves PLS from artifact design to repo implementation by defining the DB/API/UI/worker dispatch contract required to operationalize that loop.