Skip to content

Instantly share code, notes, and snippets.

@denniswon
Created April 10, 2026 22:33
Show Gist options
  • Select an option

  • Save denniswon/06d1c34ba5ca47a4fcfefc4de1ef49bf to your computer and use it in GitHub Desktop.

Select an option

Save denniswon/06d1c34ba5ca47a4fcfefc4de1ef49bf to your computer and use it in GitHub Desktop.
TEE Integration for Newton Protocol
---
Meeting: TEE Integration for Newton Protocol
---
Problem Statement
Newton Protocol accumulates sensitive data (identity PII, confidential data, policy client secrets) that operators must decrypt and
process during policy evaluation. Two core concerns:
1. Operator data leakage — operators currently decrypt private data locally, meaning a malicious or compromised operator could exfiltrate
PII
2. Regulatory compliance (GDPR) — the safest posture for regulators is that private data is never exposed in plaintext outside a
hardware-attested environment
★ Insight ─────────────────────────────────────
This maps directly to Newton's existing three-path privacy model: identity data (data.identity.*), confidential data
(data.confidential.*), and inline ephemeral (data.privacy.inline[0].*). Today, operators decrypt all three paths locally — either with
their own HPKE key (centralized mode) or via DKG key shares (threshold mode). TEE would move that decryption boundary inside a hardware
enclave, so operators never see plaintext.
─────────────────────────────────────────────────
---
Approaches Evaluated
┌─────────────┬─────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Approach │ Verdict │ Reasoning │
├─────────────┼─────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ TEE (AWS │ Chosen │ Practical, well-documented, prior team experience with Intel TDX, meets GDPR requirements, │
│ Nitro) │ │ hardware attestation valued by institutional partners │
├─────────────┼─────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ │ Deferred │ Jacob called it "Pandora's box" — higher risk, longer timeline, no guaranteed results in 2-4 │
│ MPC │ (research │ weeks. Dennis noted MPC isn't scalable as TEE. Existing DKG/PSS work mitigates some │
│ │ phase) │ operator-leaving risks already │
├─────────────┼─────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ FHE │ Rejected │ Too expensive, circuit arithmetic limited to basic operations ("very old days of ZK"), not │
│ │ │ practical for Rego evaluation │
├─────────────┼─────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ZK proofs │ Doesn't solve │ ZK proves computation correctness but doesn't prevent data decryption — you still need to decrypt │
│ │ the problem │ before running the circuit │
└─────────────┴─────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘
---
Agreed Architecture
What goes inside the TEE enclave:
- Rego policy evaluation only (Phase 1) — where private data is actually loaded and used
- WASM execution is a secondary step (Phase 2), lower priority since WASM only handles policy client secrets, not PII
What stays outside the enclave:
- Application code (operator binary)
- WASM data provider execution (less sensitive)
- All non-privacy policy evaluations
Communication: Enclave ↔ Application via VSOCK (attached secure encrypted channel). No external network connections needed from the
enclave — it only talks to the local operator application.
Fallback: If TEE communication fails, fall back to raw (non-enclave) policy evaluation. Only privacy-involved evaluations run inside the
TEE.
┌─────────────────────────────────────────────┐
│ Operator Process (outside enclave) │
│ ┌────────────┐ ┌──────────────────────┐ │
│ │ WASM Exec │ │ Operator Service │ │
│ │ (data │ │ (RPC, BLS signing, │ │
│ │ provider) │ │ chain interaction) │ │
│ └────────────┘ └──────┬───────────────┘ │
│ │ VSOCK │
│ ┌──────────────────────▼───────────────┐ │
│ │ AWS Nitro Enclave │ │
│ │ ┌──────────────────────────────┐ │ │
│ │ │ Rego Policy Evaluation │ │ │
│ │ │ + Private data decryption │ │ │
│ │ │ + HPKE decrypt → eval → │ │ │
│ │ │ zeroize → return result │ │ │
│ │ └──────────────────────────────┘ │ │
│ │ Remote Attestation (built-in) │ │
│ └──────────────────────────────────────┘ │
└─────────────────────────────────────────────┘
★ Insight ─────────────────────────────────────
1. Memory is not a concern — Jacob noted TEE memory limits of 256-512 MB per CPU. Dennis confirmed Rego evaluation is lightweight and
fits comfortably. This is a much smaller footprint than running the full WASM data provider.
2. The zeroization pattern already exists — Newton already zeroizes decrypted private data outside WASM (lessons.md documents this). TEE
takes this further by ensuring the plaintext never exists outside the enclave at all.
3. Secret Network is the closest analog — Wesley identified Secret Network's model as nearly identical: encrypted transactions enter the
TEE, are decrypted inside a Cosmos SDK runtime, executed, and only the output exits. Newton's model substitutes Rego evaluation for
Cosmos SDK execution.
─────────────────────────────────────────────────
---
Key Design Decisions
1. AWS Nitro as preferred platform — best performance, mature remote attestation, existing AWS infra alignment. Intel TDX / AMD SEV as
future portability options (not high priority now).
2. Every operator runs their own enclave — not a single centralized enclave. Rationale: preserves the operator model where each operator
independently evaluates. Wesley proposed a single-enclave model but Dennis noted it "kills the purpose of having multiple operators."
3. Institutional operators, not general validators — Newton targets financial institutions and strategic partners, not Block Daemon /
Luganodes-style validator shops. These operators are motivated by "I don't want to touch PII data" — TEE gives them hardware-level
guarantees.
4. Tiered operator model (Alec's suggestion, Dennis acknowledged) — institutional operators run enclaves for privacy-sensitive work;
other operators could handle non-sensitive tasks. Operator running is permissioned, staking is permissionless.
5. No vendor lock-in panic — cloud-native Nitro enclaves (VM-based, not bare metal). Dennis noted no additional pricing overhead beyond
Nitro-enabled EC2 instances.
---
Concerns Raised
┌────────────────────────────────────────┬─────────────┬─────────────────────────────────────────────────────────────────────────────┐
│ Concern │ Raised By │ Resolution │
├────────────────────────────────────────┼─────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ SGX/TDX security gaps (physical access │ Jacob │ Dennis: these are research-level, not practical threats. Nitro has stronger │
│ attacks) │ │ attestation than SGX │
├────────────────────────────────────────┼─────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Vendor lock-in to AWS │ Jacob, │ Accepted as trade-off for speed. Future TDX/AMD support possible │
│ │ Wesley │ │
├────────────────────────────────────────┼─────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Operator onboarding complexity │ Wesley │ Mitigated by targeting institutional operators who can handle it │
├────────────────────────────────────────┼─────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Centralization risk │ Wesley │ Each operator runs own enclave. Duplication + fallback for availability │
├────────────────────────────────────────┼─────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Operator key compromise on departure │ Jacob │ Already mitigated by DKG epoch rotation + upcoming PSS (NEWT-628). TEE adds │
│ │ │ defense-in-depth │
├────────────────────────────────────────┼─────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ MPC as alternative │ Jacob │ Dennis: not ruled out long-term, but can't be sole bet without a Plan B │
└────────────────────────────────────────┴─────────────┴─────────────────────────────────────────────────────────────────────────────┘
---
Action Items
1. Wesley Coakley — Lead the TEE integration initiative. Draft a detailed technical document covering:
- Enclave architecture
- Communication protocols (VSOCK)
- How Rego policy evaluation runs inside the enclave
- Staged WASM integration plan
2. Team — Include TEE-based policy evaluation in upcoming sprint. Start technical exploration immediately (today/tomorrow).
3. Dennis + team — Monitor MPC as a future alternative while prioritizing TEE now.
---
Relationship to Existing Newton Architecture
This maps onto the current codebase as follows:
┌───────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────┐
│ Current Component │ TEE Impact │
├───────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ crates/core/src/crypto/envelope.rs (HPKE decrypt) │ Decryption moves inside enclave │
├───────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ crates/operator/src/core.rs (Rego evaluation with privacy │ Evaluation moves inside enclave │
│ extensions) │ │
├───────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ crates/core/src/dkg/ (threshold key management) │ DKG key shares would be loaded inside enclave only │
├───────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ crates/data-provider/ (WASM execution) │ Stays outside enclave (Phase 1). Policy client secrets injection may │
│ │ move inside later (Phase 2) │
├───────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Privacy data zeroization (already in place) │ Becomes hardware-enforced rather than software-enforced │
└───────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────┘
★ Insight ─────────────────────────────────────
1. TEE is complementary to the existing DKG/PSS work, not a replacement. DKG handles key rotation so departed operators can't decrypt
historical data. TEE handles runtime protection so active operators can't exfiltrate plaintext during evaluation. The Phase 2C PSS work
(PRs #523/#525/#526) remains relevant.
2. The "only privacy evaluations in TEE" decision is architecturally clean — it means the enclave binary is small (just Rego engine +
HPKE decrypt), the VSOCK interface is narrow (encrypted input in, boolean/evaluation result out), and non-privacy policies see zero
performance impact.
3. Fallback to non-TEE evaluation is a pragmatic choice that matches the existing pattern where the gateway can degrade gracefully (e.g.,
circuit breaker patterns in crates/gateway/src/task/circuit_breaker.rs).
─────────────────────────────────────────────────
@denniswon

Copy link
Copy Markdown
Author

batch 3

⏺ Repeating Batch 3 questions for context:

  1. privacyEnabled flag vs on-chain detection: We've now decided on trustless on-chain detection (registry checks + wasmArgs parsing)
    instead of a gateway-set flag. This means no Task struct change needed — the contract derives privacy status from existing on-chain
    state. Does this unblock anything, or are there other pending Task struct changes you're planning to batch?
  2. ZK circuit for attestation verification: X.509 cert chain verification (RSA-2048/4096 signatures) inside an SP1 circuit is expensive —
    RSA in a zkVM is ~10-100x slower than native. Alternative: the contract verifies the attestation on-chain directly (CBOR parse +
    ecrecover-style verification). Nitro attestation cert chains use ECDSA P-384 for the leaf certificate, which Ethereum doesn't natively
    support. So the realistic options are:
    - (a) SP1 ZK proof of attestation invalidity (heavy circuit but proof is small on-chain)
    - (b) On-chain verification with a precompile/precomputed helper (cheaper if P-384 precompile lands via EIP, not available today)
    - (c) Hybrid: the challenger submits the parsed attestation fields + a Merkle proof against the AWS root CA on-chain, and the contract
    checks PCR0 + task binding without full cert chain verification — deferring cert chain to an off-chain dispute if challenged

Which direction are you leaning?
3. Phasing: Should TEE attestation verification (new ChallengeVerifier proof type + on-chain privacy detection + respondToTask
attestation requirement) ship as part of Phase 1a alongside the enclave itself, or as a follow-up after E2E validation?

❯ 1. privacyEnabled not needed and no Task struct change needed 2. (a) 3. let's scope as Phase 1a.5

⏺ Good — all three answers are clean. Let me summarize the full design we've converged on, then update the plan doc and file Linear issues.

Challenger TEE Attestation Verification — Design Summary

Core concept: Challengers don't decrypt PII for privacy tasks. Instead, they verify the operator's TEE attestation to prove the
evaluation happened inside a genuine enclave with correct code.

Attestation model: Option A (per-task attestation). Each task response includes the full Nitro attestation document (~3 KB) with
user_data = keccak256(task_id || response_digest). No registry, no ephemeral key management.

Privacy task detection: Trustless on-chain derivation. Contract checks IdentityRegistry.hasLinkedDomains(policyClient),
ConfidentialDataRegistry.hasGrantedDomains(policyClient), and parses Task.wasmArgs for inline _newton.privacy[]. No Task struct change
needed.

Slashing path: New proof type in ChallengeVerifier (not PrivacySlasher). SP1 ZK circuit proves attestation invalidity — CBOR parse, cert
chain verification, PCR0 mismatch against on-chain registry, or task binding mismatch. Challenger submits proof via existing
raiseAndResolveChallenge flow with a new proofType discriminator.

Slashable conditions:

  • Privacy task response missing attestation_data entirely
  • Attestation cert chain invalid (forged or expired)
  • PCR0 doesn't match on-chain EnclaveVersionRegistry
  • user_data doesn't bind to keccak256(task_id || response_digest)

Phase: 1a.5 (after enclave E2E validation, before Phase 1b threshold mode)

Interaction with existing slashing:

┌────────────────────────────────┬────────────────────────────────────────────────────┬─────────┐
│ System │ Scope │ Phase │
├────────────────────────────────┼────────────────────────────────────────────────────┼─────────┤
│ ChallengeVerifier Type 1 │ Policy re-evaluation mismatch (ZK Rego proof) │ Commit │
├────────────────────────────────┼────────────────────────────────────────────────────┼─────────┤
│ ChallengeVerifier Type 2 (new) │ TEE attestation invalid/missing for privacy task │ Commit │
├────────────────────────────────┼────────────────────────────────────────────────────┼─────────┤
│ PrivacySlasher │ Wrong keyshare (DLEQ), missing keyshare (liveness) │ Prepare │
└────────────────────────────────┴────────────────────────────────────────────────────┴─────────┘

Does this capture everything correctly? If so, I'll update the plan doc and file the Linear issues.

@denniswon

Copy link
Copy Markdown
Author

❯ Looks good

⏺ Update(docs/plans/tee-integration.md)
⎿  Added 1 line
151 |-------|-------|-------------|----------|
152 | Pre-TEE | Cache envelopes Prepare->Commit + separate secrets key | Same (operator decrypts locally) | All done (PRs
#546, #548) |
153 | 1a | Operator enclave: decrypt + Rego eval (no WASM) | Operators untrusted -> TEE protects. Gateway trusted (relay
only). | NEWT-938 |
154 +| 1a.5 | Challenger TEE attestation verification | Permissionless challengers verify enclave integrity without decryp
+ting PII | ChallengeVerifier Type 2, SP1 attestation circuit |
155 | 1b | Enclave-to-enclave partial DH | Gateway is pure relay. Cannot combine partials. | NEWT-945, NEWT-950 |
156 | 2 | WASM execution inside enclave + Egress Service | Policy client secrets also protected. Host-side HTTP proxy via
VSOCK. | NEWT-952 |
157 | 3 | On-chain enclave registry | PCR0 governance, version management | NEWT-953, NEWT-954 |

⏺ Now add the full Phase 1a.5 design section to the plan doc.

⏺ Update(docs/plans/tee-integration.md)
⎿  Added 65 lines
160
161 Threshold secrets deadlock (NEWT-989, resolved): When DKG is active, newt_getPrivacyPublicKey returns threshold MPK
for ALL encryption including WASM secrets. WASM secrets.get during Prepare can't threshold-decrypt (partials not yet c
ollected). Fix: newt_getSecretsPublicKey always returns individual operator HPKE key. Secrets encrypt per-operator, pri
vacy data encrypts to threshold MPK.
162
163 +## Phase 1a.5: Challenger TEE Attestation Verification (decided 2026-04-16)
164 +
165 +### Problem
166 +
167 +Challengers are designed to be permissionless — anyone can run one. The current challenge path (evaluate_and_prove) re-
+evaluates the policy with decrypted PII. A permissionless challenger with raw PII access is a privacy backdoor: anyone wh
+o wants to read the data just runs a challenger.
168 +
169 +### Solution
170 +
171 +For privacy tasks, challengers verify the operator's TEE attestation instead of re-decrypting PII. The attestation proves
+ the evaluation happened inside a genuine enclave with correct code. No PII leaves the enclave; no PII reaches the challe
+nger.
172 +
173 +### Attestation Model: Per-Task (Option A)
174 +
175 +Each privacy task response includes the full Nitro attestation document (~3 KB CBOR) with user_data = keccak256(task_id +|| response_digest). This binds the attestation to the specific task output at the hardware level.
176 +
177 +Option B (boot-time attestation + per-task ephemeral signature) was evaluated and rejected for Phase 1a.5. It introduces
+coordination state (on-chain attestation registry, freshness windows, revocation) that adds operational complexity and su
+btle failure modes (stale registry entries, clock drift, reboot races). At current L2 volumes the gas delta is negligible
+ ($1-10/day on Base). Option B can be added later as an optimization behind the same bytes attestation_data interface i
+f volume warrants.
178 +
179 +### Privacy Task Detection: Trustless On-Chain Derivation
180 +
181 +The contract determines privacy involvement without a gateway-set flag (gateway is pure orchestrator, never authoritative
+):
182 +
183 + 184 +_isPrivacyTask(task): 185 + if identityRegistry.hasLinkedDomains(task.policyClient): return true 186 + if confidentialRegistry.hasGrantedDomains(task.policyClient): return true 187 + if _hasInlinePrivacy(task.wasmArgs): return true 188 + return false 189 +
190 +
191 +hasLinkedDomains and hasGrantedDomains are cheap boolean flags (single SLOAD each) maintained by the registries. Inli
+ne privacy check parses wasmArgs calldata. No Task struct change needed.
192 +
193 +### Slashing Path: ChallengeVerifier Type 2
194 +
195 +TEE attestation challenges go through ChallengeVerifier as a new proof type, not PrivacySlasher. Rationale: this is a
+bout evaluation integrity ("did the operator evaluate in a genuine enclave?"), not threshold decryption protocol complian
+ce ("did the operator submit correct partial DHs?").
196 +
197 +| System | Scope | Phase |
198 +|---|---|---|
199 +| ChallengeVerifier Type 1 | Policy re-evaluation mismatch (SP1 Rego circuit) | Commit |
200 +| ChallengeVerifier Type 2 | TEE attestation invalid/missing for privacy task (SP1 attestation circuit) | Commi
+t
|
201 +| PrivacySlasher | Wrong keyshare (DLEQ failure), missing keyshare (liveness) | Prepare |
202 +
203 +### SP1 ZK Circuit: Attestation Verification
204 +
205 +The challenger generates a ZK proof that the attestation is invalid. Circuit inputs:
206 +- Attestation document bytes (from task response attestation_data)
207 +- On-chain PCR0 registry value (from EnclaveVersionRegistry)
208 +- AWS Nitro root CA certificate (or hash)
209 +- task_id and response_digest (for user_data binding check)
210 +
211 +The circuit proves one or more of:
212 +- Attestation cert chain invalid (forged or expired certificates)
213 +- PCR0 doesn't match on-chain EnclaveVersionRegistry whitelist
214 +- user_data doesn't bind to keccak256(task_id || response_digest)
215 +- Attestation document missing entirely from task response
216 +
217 +### Slashable Conditions
218 +
219 +1. Privacy task response has empty attestation_data
220 +2. Attestation cert chain verification fails
221 +3. PCR0 not in on-chain whitelist (wrong or outdated enclave image)
222 +4. Task binding mismatch (attestation produced for a different task)
223 +
224 +### Interface Extensibility
225 +
226 +respondToTask accepts bytes attestation_data. For Phase 1a.5, this is a full Nitro attestation document. The same fie
+ld can later accept a compact format (boot attestation reference + ephemeral signature) if Option B is needed at scale. T
+he contract and challenger accept either format via a version byte prefix.
227 +
228 Note: Phase 1a no longer references "centralized mode" — centralized mode was fully removed in PR #537. Phase 1a operates
within the two-phase consensus protocol.
229
230 ## Linear Issues (19 total, 2 done)

⏺ Now add Phase 1a.5 Linear issues to the plan and file them on Linear.

⏺ Update(docs/plans/tee-integration.md)
⎿  Added 10 lines
248 | NEWT-943 | TEE build infrastructure (Docker, Makefile, workspace) | Medium | Unassigned | Not started |
249 | NEWT-944 | TEE E2E testing | Medium | Unassigned | Not started |
250
251 +### Phase 1a.5: Challenger TEE Attestation Verification — 5 Issues
252 +
253 +| Issue | Title | Priority | Assigned | Status |
254 +|-------|-------|----------|----------|--------|
255 +| NEWT-955 | On-chain privacy task detection (_isPrivacyTask) | High | Unassigned | Not started |
256 +| NEWT-956 | Per-task TEE attestation in respondToTask (attestation_data field) | High | Unassigned | Not started |
257 +| NEWT-957 | SP1 circuit for Nitro attestation verification | High | Unassigned | Not started |
258 +| NEWT-958 | ChallengeVerifier Type 2: TEE attestation challenge path | High | Unassigned | Not started |
259 +| NEWT-959 | Challenger privacy task routing (attestation check vs re-evaluation) | Medium | Unassigned | Not started |
260 +
261 ### Phase 1b: Threshold Mode TEE — 7 Issues
262
263 | Issue | Title | Priority | Assigned | Status |

⏺ Update the issue count at the top of the Linear section.

⏺ Update(docs/plans/tee-integration.md)
⎿  Added 1 line, removed 1 line
227
228 Note: Phase 1a no longer references "centralized mode" — centralized mode was fully removed in PR #537. Phase 1a operates
within the two-phase consensus protocol.
229
230 -## Linear Issues (19 total, 2 done)
230 +## Linear Issues (24 total, 2 done)
231
232 ### Pre-TEE — 2 Issues (All Done)
233

⏺ Now add the Phase 1a.5 TODOs.

⏺ Update(docs/plans/tee-integration.md)
⎿  Added 5 lines
306 - [ ] NEWT-941: Route privacy evals to enclave (operator integration)
307 - [ ] NEWT-942: Gateway attestation verification in operator pool
308 - [ ] NEWT-943: TEE build infrastructure
309 +- [ ] NEWT-955: On-chain privacy task detection (registry boolean flags + wasmArgs parsing)
310 +- [ ] NEWT-956: Per-task attestation in respondToTask (attestation_data bytes field)
311 +- [ ] NEWT-957: SP1 circuit for Nitro attestation verification (CBOR + cert chain + PCR0 + task binding)
312 +- [ ] NEWT-958: ChallengeVerifier Type 2 — TEE attestation challenge path
313 +- [ ] NEWT-959: Challenger privacy task routing (attestation check vs re-evaluation)
314 - [ ] NEWT-945: Enclave-to-enclave encrypted partial DH exchange
315 - [ ] NEWT-953: Design on-chain EnclaveVersionRegistry contract
316

⏺ Now file the 5 Linear issues.

Called linear-server 2 times (ctrl+o to expand)

⏺ Need to find the milestone UUID first. Let me look up the existing milestones on the TEE Integration project.

Called linear-server 2 times (ctrl+o to expand)

⏺ I need to create a new milestone for Phase 1a.5, then file the issues under it.

Called linear-server (ctrl+o to expand)

⏺ Now file all 5 issues with the correct milestone UUID.

Called linear-server 5 times (ctrl+o to expand)

⏺ All 5 issues filed. The actual issue numbers are NEWT-992 through NEWT-996 (not NEWT-955-959 as I estimated in the plan). Let me update
the plan doc to use the correct numbers.

⏺ Update(docs/plans/tee-integration.md)
⎿  Added 2 lines, removed 2 lines
252
253 | Issue | Title | Priority | Assigned | Status |
254 |-------|-------|----------|----------|--------|
255 -| NEWT-955 | On-chain privacy task detection (_isPrivacyTask) | High | Unassigned | Not started |
255 +| NEWT-992 | On-chain privacy task detection (_isPrivacyTask) | High | Unassigned | Not started |
256 | NEWT-956 | Per-task TEE attestation in respondToTask (attestation_data field) | High | Unassigned | Not started |
257 | NEWT-957 | SP1 circuit for Nitro attestation verification | High | Unassigned | Not started |
258 | NEWT-958 | ChallengeVerifier Type 2: TEE attestation challenge path | High | Unassigned | Not started |
...
306 - [ ] NEWT-941: Route privacy evals to enclave (operator integration)
307 - [ ] NEWT-942: Gateway attestation verification in operator pool
308 - [ ] NEWT-943: TEE build infrastructure
309 -- [ ] NEWT-955: On-chain privacy task detection (registry boolean flags + wasmArgs parsing)
309 +- [ ] NEWT-992: On-chain privacy task detection (registry boolean flags + wasmArgs parsing)
310 - [ ] NEWT-956: Per-task attestation in respondToTask (attestation_data bytes field)
311 - [ ] NEWT-957: SP1 circuit for Nitro attestation verification (CBOR + cert chain + PCR0 + task binding)
312 - [ ] NEWT-958: ChallengeVerifier Type 2 — TEE attestation challenge path

⏺ Update(docs/plans/tee-integration.md)
⎿  Added 2 lines, removed 2 lines
253 | Issue | Title | Priority | Assigned | Status |
254 |-------|-------|----------|----------|--------|
255 | NEWT-992 | On-chain privacy task detection (_isPrivacyTask) | High | Unassigned | Not started |
256 -| NEWT-956 | Per-task TEE attestation in respondToTask (attestation_data field) | High | Unassigned | Not started |
256 +| NEWT-993 | Per-task TEE attestation in respondToTask (attestation_data field) | High | Unassigned | Not started |
257 | NEWT-957 | SP1 circuit for Nitro attestation verification | High | Unassigned | Not started |
258 | NEWT-958 | ChallengeVerifier Type 2: TEE attestation challenge path | High | Unassigned | Not started |
259 | NEWT-959 | Challenger privacy task routing (attestation check vs re-evaluation) | Medium | Unassigned | Not started |
...
307 - [ ] NEWT-942: Gateway attestation verification in operator pool
308 - [ ] NEWT-943: TEE build infrastructure
309 - [ ] NEWT-992: On-chain privacy task detection (registry boolean flags + wasmArgs parsing)
310 -- [ ] NEWT-956: Per-task attestation in respondToTask (attestation_data bytes field)
310 +- [ ] NEWT-993: Per-task attestation in respondToTask (attestation_data bytes field)
311 - [ ] NEWT-957: SP1 circuit for Nitro attestation verification (CBOR + cert chain + PCR0 + task binding)
312 - [ ] NEWT-958: ChallengeVerifier Type 2 — TEE attestation challenge path
313 - [ ] NEWT-959: Challenger privacy task routing (attestation check vs re-evaluation)

⏺ Update(docs/plans/tee-integration.md)
⎿  Added 2 lines, removed 2 lines
254 |-------|-------|----------|----------|--------|
255 | NEWT-992 | On-chain privacy task detection (_isPrivacyTask) | High | Unassigned | Not started |
256 | NEWT-993 | Per-task TEE attestation in respondToTask (attestation_data field) | High | Unassigned | Not started |
257 -| NEWT-957 | SP1 circuit for Nitro attestation verification | High | Unassigned | Not started |
257 +| NEWT-994 | SP1 circuit for Nitro attestation verification | High | Unassigned | Not started |
258 | NEWT-958 | ChallengeVerifier Type 2: TEE attestation challenge path | High | Unassigned | Not started |
259 | NEWT-959 | Challenger privacy task routing (attestation check vs re-evaluation) | Medium | Unassigned | Not started |
260
...
308 - [ ] NEWT-943: TEE build infrastructure
309 - [ ] NEWT-992: On-chain privacy task detection (registry boolean flags + wasmArgs parsing)
310 - [ ] NEWT-993: Per-task attestation in respondToTask (attestation_data bytes field)
311 -- [ ] NEWT-957: SP1 circuit for Nitro attestation verification (CBOR + cert chain + PCR0 + task binding)
311 +- [ ] NEWT-994: SP1 circuit for Nitro attestation verification (CBOR + cert chain + PCR0 + task binding)
312 - [ ] NEWT-958: ChallengeVerifier Type 2 — TEE attestation challenge path
313 - [ ] NEWT-959: Challenger privacy task routing (attestation check vs re-evaluation)
314 - [ ] NEWT-945: Enclave-to-enclave encrypted partial DH exchange

⏺ Update(docs/plans/tee-integration.md)
⎿  Added 2 lines, removed 2 lines
255 | NEWT-992 | On-chain privacy task detection (_isPrivacyTask) | High | Unassigned | Not started |
256 | NEWT-993 | Per-task TEE attestation in respondToTask (attestation_data field) | High | Unassigned | Not started |
257 | NEWT-994 | SP1 circuit for Nitro attestation verification | High | Unassigned | Not started |
258 -| NEWT-958 | ChallengeVerifier Type 2: TEE attestation challenge path | High | Unassigned | Not started |
258 +| NEWT-995 | ChallengeVerifier Type 2: TEE attestation challenge path | High | Unassigned | Not started |
259 | NEWT-959 | Challenger privacy task routing (attestation check vs re-evaluation) | Medium | Unassigned | Not started |
260
261 ### Phase 1b: Threshold Mode TEE — 7 Issues
...
309 - [ ] NEWT-992: On-chain privacy task detection (registry boolean flags + wasmArgs parsing)
310 - [ ] NEWT-993: Per-task attestation in respondToTask (attestation_data bytes field)
311 - [ ] NEWT-994: SP1 circuit for Nitro attestation verification (CBOR + cert chain + PCR0 + task binding)
312 -- [ ] NEWT-958: ChallengeVerifier Type 2 — TEE attestation challenge path
312 +- [ ] NEWT-995: ChallengeVerifier Type 2 — TEE attestation challenge path
313 - [ ] NEWT-959: Challenger privacy task routing (attestation check vs re-evaluation)
314 - [ ] NEWT-945: Enclave-to-enclave encrypted partial DH exchange
315 - [ ] NEWT-953: Design on-chain EnclaveVersionRegistry contract

⏺ Update(docs/plans/tee-integration.md)
⎿  Added 2 lines, removed 2 lines
256 | NEWT-993 | Per-task TEE attestation in respondToTask (attestation_data field) | High | Unassigned | Not started |
257 | NEWT-994 | SP1 circuit for Nitro attestation verification | High | Unassigned | Not started |
258 | NEWT-995 | ChallengeVerifier Type 2: TEE attestation challenge path | High | Unassigned | Not started |
259 -| NEWT-959 | Challenger privacy task routing (attestation check vs re-evaluation) | Medium | Unassigned | Not started |
259 +| NEWT-996 | Challenger privacy task routing (attestation check vs re-evaluation) | Medium | Unassigned | Not started |
260
261 ### Phase 1b: Threshold Mode TEE — 7 Issues
262
...
310 - [ ] NEWT-993: Per-task attestation in respondToTask (attestation_data bytes field)
311 - [ ] NEWT-994: SP1 circuit for Nitro attestation verification (CBOR + cert chain + PCR0 + task binding)
312 - [ ] NEWT-995: ChallengeVerifier Type 2 — TEE attestation challenge path
313 -- [ ] NEWT-959: Challenger privacy task routing (attestation check vs re-evaluation)
313 +- [ ] NEWT-996: Challenger privacy task routing (attestation check vs re-evaluation)
314 - [ ] NEWT-945: Enclave-to-enclave encrypted partial DH exchange
315 - [ ] NEWT-953: Design on-chain EnclaveVersionRegistry contract
316

⏺ Now update the local memory file with the same issue numbers and the Phase 1a.5 design.

Wrote 2 memories (ctrl+o to expand)

⏺ Done. Here's the summary:

Linear issues filed (5 new):

┌──────────┬──────────────────────────────────────────────────────────┬────────────┐
│ Issue │ Title │ Milestone │
├──────────┼──────────────────────────────────────────────────────────┼────────────┤
│ NEWT-992 │ On-chain privacy task detection (_isPrivacyTask) │ Phase 1a.5 │
├──────────┼──────────────────────────────────────────────────────────┼────────────┤
│ NEWT-993 │ Per-task TEE attestation in respondToTask │ Phase 1a.5 │
├──────────┼──────────────────────────────────────────────────────────┼────────────┤
│ NEWT-994 │ SP1 circuit for Nitro attestation verification │ Phase 1a.5 │
├──────────┼──────────────────────────────────────────────────────────┼────────────┤
│ NEWT-995 │ ChallengeVerifier Type 2: TEE attestation challenge path │ Phase 1a.5 │
├──────────┼──────────────────────────────────────────────────────────┼────────────┤
│ NEWT-996 │ Challenger privacy task routing │ Phase 1a.5 │
└──────────┴──────────────────────────────────────────────────────────┴────────────┘

Docs updated:

  • docs/plans/tee-integration.md — added Phase 1a.5 to phased plan table, full design section, 5 issues in Linear tracker, TODOs
  • Local memory project_tee_integration.md — synced with same changes

Total TEE project: 24 issues across 5 milestones (Pre-TEE done, Phase 1a -> 1a.5 -> 1b -> 2 -> 3).

@denniswon

Copy link
Copy Markdown
Author

❯ While wesley is working on the enclave crate, what are things that I can tackle to implement in parallel?

⏺ Looking at the dependency graph across all TEE phases:

Wesley is building (blocked, sequential):

  • NEWT-938 → NEWT-939 (enclave crate → enclave binary)

You can build in parallel (no dependency on enclave crate):

Rust — Operator Side

┌──────────┬──────────────────────┬──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Issue │ Title │ Why Parallel │
├──────────┼──────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────┤
│ │ VSOCK transport │ Define the VsockTransport trait + request/response types. The interface contract can be agreed │
│ NEWT-940 │ layer in operator │ with Wesley upfront. Operator-side VSOCK client doesn't need the enclave running to build — just │
│ │ crate │ needs the type definitions. │
└──────────┴──────────────────────┴──────────────────────────────────────────────────────────────────────────────────────────────────┘

Solidity — Privacy Slashing (existing backlog)

┌──────────┬────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Issue │ Title │ Why Parallel │
├──────────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────┤
│ NEWT-629 │ Slashing spec │ Pure design work, no enclave dependency │
├──────────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────┤
│ NEWT-640 │ PrivacySlasher │ Solidity — threshold DLEQ verification, EpochRegistry integration. Completely independent of │
│ │ contract │ TEE. │
├──────────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────┤
│ NEWT-641 │ DleqVerifier contract │ On-chain DLEQ proof verification. Pure crypto + Solidity. │
└──────────┴────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────┘

Solidity — Phase 1a.5 (Challenger TEE Attestation)

┌──────────┬──────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────┐
│ Issue │ Title │ Why Parallel │
├──────────┼──────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────┤
│ NEWT-992 │ On-chain privacy task detection │ Pure Solidity — add hasLinkedDomains/hasGrantedDomains boolean flags to │
│ │ (_isPrivacyTask) │ registries │
├──────────┼──────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────┤
│ NEWT-993 │ attestation_data field in respondToTask │ Pure Solidity — contract accepts and stores attestation bytes │
├──────────┼──────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────┤
│ NEWT-953 │ EnclaveVersionRegistry contract │ On-chain PCR0 whitelist. Needed by the SP1 attestation circuit (NEWT-994) │
│ │ │ but can be built first │
└──────────┴──────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────┘

Infra

┌──────────┬───────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────┐
│ Issue │ Title │ Why Parallel │
├──────────┼───────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────┤
│ NEWT-943 │ TEE build infrastructure (Docker, │ EIF build pipeline, nitro-cli build-enclave integration. Can scaffold without │
│ │ Makefile, workspace) │ the enclave binary existing yet │
├──────────┼───────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────┤
│ NEWT-942 │ Gateway attestation verification for │ Gateway verifies PCR0 + cert chain at operator registration. Doesn't need a │
│ │ TEE operators │ running enclave — just the attestation doc format │
└──────────┴───────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────┘

My Recommendation for Priority Order

  1. NEWT-940 (VSOCK transport) — unblocks NEWT-941 the moment Wesley delivers
  2. NEWT-992 (privacy detection) — small Solidity, unblocks NEWT-993
  3. NEWT-993 (attestation_data) — small Solidity, unblocks Phase 1a.5
  4. NEWT-953 (EnclaveVersionRegistry) — Solidity, unblocks NEWT-994
  5. NEWT-629/640/641 (slashing) — existing backlog, independent workstream

NEWT-940 is the highest leverage — it's the integration surface between your code and Wesley's. If the VSOCK types and trait are defined
first, Wesley can implement the enclave side against your interface, and NEWT-941 (operator integration) becomes a thin wiring task when
both pieces land.

Want to start on NEWT-940?

✻ Sautéed for 42s

❯ Yes start on implementation

⏺ Skill(/clarify-plan)
⎿  Successfully loaded skill

⏺ Before I start building, let me align on the key design decisions. The TEE plan has the high-level architecture but NEWT-940 has
implementation choices to make.

Batch 1 — scope and interface:

  1. Where does this code live? The TEE plan says crates/operator/ for the VSOCK transport. But the types (EnclaveEvalRequest,
    EnclaveEvalResponse, VsockEnvelope) will also be needed by Wesley's enclave crate (crates/enclave/). Should the types live in
    crates/core/ (shared dependency) with the transport trait in crates/operator/, or should we create a new crates/enclave-types/ crate that
    both operator and enclave depend on?
  2. Serialization format: The plan has an open decision — bincode (simpler) vs rkyv (zero-copy). Wesley hasn't responded yet. Bincode is
    the safe default (serde-based, well-tested, widely used in Rust), and we can swap later if Wesley's benchmarks show rkyv matters. Should
    I go with bincode for now, or do you want to wait for Wesley's input?
  3. Transport abstraction: The operator needs to call the enclave for privacy evaluations. Two approaches: (a) a trait EnclaveTransport
    with async fn evaluate(&self, req: EnclaveEvalRequest) -> Result that has a VsockTransport impl (production) and a
    MockTransport impl (testing), or (b) a concrete VsockClient struct with no trait (simpler, add the trait later if mock testing is
    needed). Given that NEWT-941 (operator integration) and NEWT-944 (E2E testing) will need mock support, (a) seems right — but want to
    confirm.

❯ 1. why not just crates/enclave? 2. bincode for now. 3. (a)

⏺ Good point on (1) — crates/enclave/ is NEWT-938 (Wesley's crate). The types and trait can live there, and both operator and enclave
binary depend on it. That's cleaner than a separate enclave-types crate. Wesley builds the enclave evaluation logic on top of the types I
define.

Batch 2 — VSOCK protocol details:

  1. VSOCK addressing: Nitro enclaves use (CID, port) for VSOCK connections. The enclave CID is typically assigned by the hypervisor
    (usually CID 16+ for enclaves, CID 3 is the parent). The TEE plan proposes two VSOCK ports: VSOCK_PORT_COMPUTE (decrypt + Rego eval,
    Phase 1a) and VSOCK_PORT_EGRESS (HTTP proxy for WASM, Phase 2). Should the port numbers be configurable via operator config ([enclave]
    section in operator.toml), or hardcoded constants with the enclave CID as the only configurable value?
  2. Connection model: Should the operator open a new VSOCK connection per evaluation request (simple, no connection state), or maintain a
    persistent connection pool (lower latency, more complex)? Given that VSOCK is local to the host (sub-millisecond connect), per-request
    connections are simpler and match the "enclave is stateless" model. But with max_concurrent_requests = 8 (semaphore-bounded), we could
    have 8 simultaneous connections. Any preference?
  3. Error taxonomy: When an enclave eval fails, the operator needs to distinguish between transport errors (VSOCK connection refused,
    timeout) and evaluation errors (Rego denied, decryption failed). Transport errors should trip the circuit breaker; evaluation errors
    should be surfaced to the gateway. Should EnclaveEvalResponse carry a Result<bool, EnclaveError> with typed error variants, or keep it
    simple with evaluation_result: Option + error: Option?

❯ 1. what do you recommend? help me decide. 2. what suits better for 8 max concurrent requests context? what is most reliable and
scalable? what is the risk with persistent connection pool? How much lower latency does persistent connection pool provide? 3. which do
you recommend? Result seems better but any reason we should consider the latter?

⏺ On question 1 — VSOCK port configuration:

I'd go with configurable CID, hardcoded ports as constants. Reasoning:

  • CID must be configurable — it's assigned by the hypervisor at enclave launch and varies per host. Operator needs to know which CID to
    connect to.
  • Ports should be constants — they're a protocol contract between operator and enclave. If the operator says port 5000 and the enclave
    listens on 5001, silent failure. Hardcoded constants in the shared crates/enclave/ crate ensure both sides agree. No reason for an
    operator to want a different port.

// crates/enclave/src/vsock.rs
pub const VSOCK_PORT_COMPUTE: u32 = 5000;
pub const VSOCK_PORT_EGRESS: u32 = 5001; // Phase 2

// operator.toml
[enclave]
cid = 16 # assigned by hypervisor
timeout_ms = 5000 # per-request timeout

On question 2 — connection model:

┌─────────────────┬───────────────────────────────────────────────┬─────────────────────────────────────────────────────────────────┐
│ │ Per-request │ Persistent pool │
├─────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Connect latency │ ~50-200us per VSOCK connect (local │ 0 (already connected) │
│ │ hypervisor, no TCP handshake) │ │
├─────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────┤
│ At 8 concurrent │ 8 parallel connects = still ~200us (parallel, │ 0, but pool management overhead │
│ │ not serial) │ │
├─────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Failure │ Clean — each request gets a fresh socket. │ One broken connection can poison the pool. Need health checks, │
│ isolation │ Broken socket = one failed request │ reconnection logic, stale connection detection │
├─────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Enclave restart │ Automatic recovery — next request connects to │ Pool holds dead connections. Need reconnection on error + drain │
│ │ new enclave │ logic │
├─────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Complexity │ ~20 lines │ ~150+ lines (pool sizing, health checks, reconnection, drain on │
│ │ │ enclave reboot) │
├─────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Backpressure │ Natural — VSOCK connect fails if enclave is │ Need to manage pool exhaustion separately from enclave overload │
│ │ overloaded │ │
├─────────────────┼───────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Scalability │ Limited by OS file descriptors (thousands) │ Same, but pool adds a second bottleneck │
│ ceiling │ │ │
└─────────────────┴───────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────┘

The latency win from persistent connections is ~50-200 microseconds per request — negligible compared to the enclave evaluation time
(Rego eval + HPKE decrypt = 1-10ms). You'd save ~0.2ms on a 5ms operation.

The risk with persistent connections is enclave restarts. When the enclave reboots (crash, PCR0 update, key rotation), all pooled
connections become dead sockets. The operator needs detection + reconnection logic. With per-request connections, the next request just
connects to the new enclave — zero recovery code.

Recommendation: per-request connections. At 8 concurrent requests with 50-200us connect overhead, the total added latency is invisible.
The simplicity and enclave-restart resilience far outweigh the microsecond savings. If profiling later shows VSOCK connect is a
bottleneck (unlikely), a pool can be added behind the same EnclaveTransport trait.

On question 3 — error taxonomy:

Result<bool, EnclaveError> with typed variants. The string approach has two problems:

  1. Circuit breaker needs to distinguish error types programmatically. With Option, the operator would string-match to decide "is
    this a transport error or an evaluation error?" — exactly the anti-pattern from lessons.md (is_transient_rpc_error must check revert
    patterns before transient patterns).
  2. Structured errors compose better. The operator already has OperatorError with typed variants. EnclaveError maps cleanly into it
    without lossy string conversion.

#[derive(Debug, Serialize, Deserialize)]
pub enum EnclaveError {
/// HPKE decryption failed (wrong key, corrupted envelope)
DecryptionFailed(String),
/// Rego policy evaluation error (parse failure, runtime error)
EvaluationFailed(String),
/// Missing required data (no envelopes, no policy)
MissingData(String),
/// Enclave internal error (OOM, unexpected state)
Internal(String),
}

impl EnclaveError {
/// Transport errors (VSOCK connect/read/write failures) are NOT in this enum.
/// They're returned as std::io::Error from the transport layer, separate from
/// evaluation errors. This lets the operator distinguish:
/// - Transport error → trip circuit breaker, retry on different operator
/// - EnclaveError → surface to gateway as evaluation failure
pub fn is_retryable(&self) -> bool {
matches!(self, EnclaveError::Internal(_))
}
}

The transport layer returns Result<EnclaveEvalResponse, io::Error> (VSOCK failures). The response itself contains Result<bool,
EnclaveError> (evaluation failures). Two-level error model — clean separation.

Ready to proceed with implementation? I have all the design inputs:

  • Types in crates/enclave/, shared between operator and enclave binary
  • Bincode serialization
  • trait EnclaveTransport with VsockTransport + MockTransport
  • Configurable CID, constant ports
  • Per-request VSOCK connections
  • Typed EnclaveError enum, separate from transport io::Error

✻ Worked for 46s

❯ one qeuestion on "- Ports should be constants — they're a protocol contract between operator and enclave. If the operator says port 5000
and the enclave
listens on 5001, silent failure. Hardcoded constants in the shared crates/enclave/ crate ensure both sides agree. No reason for an
operator to want a different port." --> if configurable in operator.toml, how would enclave <> operator diverge with ports? are they
both going to use what is defined in operator.toml?

⏺ Good question. They wouldn't both read operator.toml — the enclave binary has no filesystem access (Nitro enclaves have no persistent
storage, no config files). So:

  • Operator parent: reads operator.toml at startup, gets port number
  • Enclave binary: has the port number compiled in (hardcoded constant or passed as a launch argument via VSOCK at boot)

If the port is configurable in operator.toml, there are only two ways to get it into the enclave:

  1. Operator sends port to enclave at boot via VSOCK — but this creates a chicken-and-egg problem: the enclave needs to listen on some
    port to receive the "here's your port" message
  2. Compile the port into the enclave binary — works, but now changing the port requires rebuilding the EIF image (new PCR0, requires
    on-chain whitelist update)

Neither is worth it. The port is an internal protocol detail between two processes on the same host. There's no external interface, no
firewall rule, no port conflict risk (VSOCK ports are a separate namespace from TCP ports — port 5000 on VSOCK doesn't conflict with port
5000 on TCP).

Hardcoded constants in crates/enclave/ is the right call. Both sides import the same constant, guaranteed to agree at compile time. Zero
configuration surface, zero divergence risk.

// crates/enclave/src/vsock.rs — imported by both operator and enclave binary
pub const VSOCK_PORT_COMPUTE: u32 = 5000;
pub const VSOCK_PORT_EGRESS: u32 = 5001; // Phase 2

The only configurable value in operator.toml is the CID (which enclave to connect to), because that's assigned dynamically by the
hypervisor.

Ready to proceed with implementation?

@denniswon

Copy link
Copy Markdown
Author

⏺ Both commits pushed to feat/newt-992-privacy-task-detection:

ef792951 docs: add TEE enclave architecture design document
00b3cf62 feat: add on-chain privacy task detection for TEE attestation enforcement (NEWT-992)

Branch summary:

┌──────────┬──────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────┐
│ Commit │ Scope │ Files │
├──────────┼──────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ 00b3cf62 │ NEWT-992 Solidity: hasLinkedIdentity + │ 6 files, 173 lines (2 interfaces, 2 implementations, 2 test files, │
│ │ hasGrantedDomains │ 11 new tests) │
├──────────┼──────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ ef792951 │ docs/ENCLAVE.md: canonical TEE architecture doc │ 1 file, 489 lines (13 sections covering threat model through phased │
│ │ │ rollout) │
└──────────┴──────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────┘

Want me to create a PR for this branch?

✻ Brewed for 1m 39s

❯ /pr

⏺ Bash(git log --oneline $(git merge-base HEAD main)..HEAD)
⎿  ef792951 docs: add TEE enclave architecture design document
00b3cf62 feat: add on-chain privacy task detection for TEE attestation enforcement (NEWT-992)

⏺ Bash(git diff $(git merge-base HEAD main)..HEAD --stat)
⎿   contracts/src/core/ConfidentialDataRegistry.sol | 7 +
▐▛███▜▌ Claude Code v2.1.109
▝▜█████▛▘ Opus 4.6 (1M context) · API Usage Billing
▘▘ ▝▝ ~/projects/newton-prover-avs

⏺ No formatting changes. Commit and push.

⏺ Bash(git commit -m "$(cat <<'EOF'
feat: add on-chain privacy task detection for TEE attestation enforcement (NEWT-992)…)
⎿  [feat/newt-992-privacy-task-detection 00b3cf62] feat: add on-chain privacy task detection for TEE attestation enforcement (NEWT-9
92)
6 files changed, 173 insertions(+)

⏺ Bash(git push origin HEAD 2>&1)
⎿  remote:
remote: Create a pull request for 'feat/newt-992-privacy-task-detection' on GitHub by visiting:
remote: https://github.com/newt-foundation/newton-prover-avs/pull/new/feat/newt-992-privacy-task-detection
… +7 lines (ctrl+o to expand)

⏺ Pushed to feat/newt-992-privacy-task-detection. The ENCLAVE.md agent is still running in the background — I'll let you know when it
completes. Want me to create a PR for the NEWT-992 changes while we wait?

✻ Crunched for 1m 13s · 1 local agent still running

⏺ Agent "Write ENCLAVE.md architecture doc" completed

⏺ The ENCLAVE.md agent completed successfully — 489 lines, 13 sections, verified all file links. Let me review what it wrote and add it to
the commit.

Read 2 files (ctrl+o to expand)

⏺ Good start — the overview, threat model, and architecture diagram look solid. Let me scan the rest quickly.

Read 1 file (ctrl+o to expand)

⏺ The doc is comprehensive and well-structured. A few things to fix:

  1. Line 489 references SECURITY_AUDIT.md — should be SECURITY.md (that's what exists in the repo)
  2. Line 470 says EnclaveClient is a trait — good, it matches our review feedback to Wesley. But Wesley's current implementation is an
    enum. The doc should reflect the committed architecture (trait-based) since that's what we asked Wesley to change to.
  3. Line 160-189 — let me check the EnclaveEvalRequest field table against Wesley's actual types.

Read 1 file (ctrl+o to expand)

⏺ The field table doesn't perfectly match Wesley's actual types (he uses EnclaveEnvelope with domain metadata, not raw Vec,
and has a nested ThresholdEvalInput struct instead of flat fields). But this doc describes the committed architecture, and Wesley may
adjust his types based on our review. The table captures the correct conceptual model. Let me fix the one clear error — the Related
Documentation link.

⏺ Update(~/projects/newton-prover-avs/docs/ENCLAVE.md)
⎿  Added 1 line, removed 1 line
486 - CONSENSUS.md -- Two-Phase Consensus protocol that the enclave integrates with (Prepare/Commit phases)
487 - THRESHOLD_DKG.md -- FROST DKG ceremony, Feldman VSS, DLEQ proofs, Lagrange interpolation, PSS epoch
rotation
488 - EPOCH_REGISTRY.md -- On-chain epoch lifecycle, MPK immutability, grace periods, emergency rotation
489 -- SECURITY_AUDIT.md -- Deployment security architecture, key hierarchy, signer modes
489 +- SECURITY.md -- Deployment security architecture, key hierarchy, signer modes

⏺ Now let me also add it to the CLAUDE.md Related Documentation section and commit everything.

Searched for 2 patterns (ctrl+o to expand)

⏺ CLAUDE.md is gitignored on this repo, so I can't add it to the commit. The ENCLAVE.md doc itself is sufficient. Let me commit it
alongside the NEWT-992 Solidity changes or as a separate commit.

⏺ Bash(cd /Users/dennis.won/projects/newton-prover-avs && git status)
⎿  On branch feat/newt-992-privacy-task-detection
Untracked files:
(use "git add ..." to include in what will be committed)
… +3 lines (ctrl+o to expand)

⏺ The Solidity changes are already committed. Only the new ENCLAVE.md is untracked. Docs-only change — skip fmt and clippy.

⏺ Bash(git add docs/ENCLAVE.md && git commit -m "$(cat <<'EOF'
docs: add TEE enclave architecture design document…)
⎿  [feat/newt-992-privacy-task-detection ef792951] docs: add TEE enclave architecture design document
1 file changed, 489 insertions(+)
create mode 100644 docs/ENCLAVE.md
… +7 lines (ctrl+o to expand)

⏺ Both commits pushed to feat/newt-992-privacy-task-detection:

ef792951 docs: add TEE enclave architecture design document
00b3cf62 feat: add on-chain privacy task detection for TEE attestation enforcement (NEWT-992)

Branch summary:

┌──────────┬──────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────┐
│ Commit │ Scope │ Files │
├──────────┼──────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ 00b3cf62 │ NEWT-992 Solidity: hasLinkedIdentity + │ 6 files, 173 lines (2 interfaces, 2 implementations, 2 test files, │
│ │ hasGrantedDomains │ 11 new tests) │
├──────────┼──────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ ef792951 │ docs/ENCLAVE.md: canonical TEE architecture doc │ 1 file, 489 lines (13 sections covering threat model through phased │
│ │ │ rollout) │
└──────────┴──────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────┘

Want me to create a PR for this branch?

✻ Brewed for 1m 39s

❯ /pr

⏺ Bash(git log --oneline $(git merge-base HEAD main)..HEAD)
⎿  ef792951 docs: add TEE enclave architecture design document
00b3cf62 feat: add on-chain privacy task detection for TEE attestation enforcement (NEWT-992)

⏺ Bash(git diff $(git merge-base HEAD main)..HEAD --stat)
⎿   contracts/src/core/ConfidentialDataRegistry.sol | 7 +
contracts/src/core/IdentityRegistry.sol | 23 +
.../src/interfaces/IConfidentialDataRegistry.sol | 9 +
… +5 lines (ctrl+o to expand)

⏺ Bash(gh pr create --base main --title "feat: on-chain privacy task detection and TEE architecture doc (NEWT-992)" --body "$(cat <<'EOF'
▐▛███▜▌ Claude Code v2.1.109
▝▜█████▛▘ Opus 4.6 (1M context) · API Usage Billing
▘▘ ▝▝ ~/projects/newton-prover-avs

✻ Crunched for 1m 13s · 1 local agent still running

⏺ Agent "Write ENCLAVE.md architecture doc" completed

⏺ The ENCLAVE.md agent completed successfully — 489 lines, 13 sections, verified all file links. Let me review what it wrote and add it to
the commit.

Read 2 files (ctrl+o to expand)

⏺ Good start — the overview, threat model, and architecture diagram look solid. Let me scan the rest quickly.

Read 1 file (ctrl+o to expand)

⏺ The doc is comprehensive and well-structured. A few things to fix:

  1. Line 489 references SECURITY_AUDIT.md — should be SECURITY.md (that's what exists in the repo)
  2. Line 470 says EnclaveClient is a trait — good, it matches our review feedback to Wesley. But Wesley's current implementation is an
    enum. The doc should reflect the committed architecture (trait-based) since that's what we asked Wesley to change to.
  3. Line 160-189 — let me check the EnclaveEvalRequest field table against Wesley's actual types.

Read 1 file (ctrl+o to expand)

⏺ The field table doesn't perfectly match Wesley's actual types (he uses EnclaveEnvelope with domain metadata, not raw Vec,
and has a nested ThresholdEvalInput struct instead of flat fields). But this doc describes the committed architecture, and Wesley may
adjust his types based on our review. The table captures the correct conceptual model. Let me fix the one clear error — the Related
Documentation link.

⏺ Update(~/projects/newton-prover-avs/docs/ENCLAVE.md)
⎿  Added 1 line, removed 1 line
486 - CONSENSUS.md -- Two-Phase Consensus protocol that the enclave integrates with (Prepare/Commit phases)
487 - THRESHOLD_DKG.md -- FROST DKG ceremony, Feldman VSS, DLEQ proofs, Lagrange interpolation, PSS epoch
rotation
488 - EPOCH_REGISTRY.md -- On-chain epoch lifecycle, MPK immutability, grace periods, emergency rotation
489 -- SECURITY_AUDIT.md -- Deployment security architecture, key hierarchy, signer modes
489 +- SECURITY.md -- Deployment security architecture, key hierarchy, signer modes

⏺ Now let me also add it to the CLAUDE.md Related Documentation section and commit everything.

Searched for 2 patterns (ctrl+o to expand)

⏺ CLAUDE.md is gitignored on this repo, so I can't add it to the commit. The ENCLAVE.md doc itself is sufficient. Let me commit it
alongside the NEWT-992 Solidity changes or as a separate commit.

⏺ Bash(cd /Users/dennis.won/projects/newton-prover-avs && git status)
⎿  On branch feat/newt-992-privacy-task-detection
Untracked files:
(use "git add ..." to include in what will be committed)
… +3 lines (ctrl+o to expand)

⏺ The Solidity changes are already committed. Only the new ENCLAVE.md is untracked. Docs-only change — skip fmt and clippy.

⏺ Bash(git add docs/ENCLAVE.md && git commit -m "$(cat <<'EOF'
docs: add TEE enclave architecture design document…)
⎿  [feat/newt-992-privacy-task-detection ef792951] docs: add TEE enclave architecture design document
1 file changed, 489 insertions(+)
create mode 100644 docs/ENCLAVE.md
… +7 lines (ctrl+o to expand)

⏺ Both commits pushed to feat/newt-992-privacy-task-detection:

ef792951 docs: add TEE enclave architecture design document
00b3cf62 feat: add on-chain privacy task detection for TEE attestation enforcement (NEWT-992)

Branch summary:

┌──────────┬──────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────┐
│ Commit │ Scope │ Files │
├──────────┼──────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ 00b3cf62 │ NEWT-992 Solidity: hasLinkedIdentity + │ 6 files, 173 lines (2 interfaces, 2 implementations, 2 test files, │
│ │ hasGrantedDomains │ 11 new tests) │
├──────────┼──────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ ef792951 │ docs/ENCLAVE.md: canonical TEE architecture doc │ 1 file, 489 lines (13 sections covering threat model through phased │
│ │ │ rollout) │
└──────────┴──────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────┘

Want me to create a PR for this branch?

✻ Brewed for 1m 39s

❯ /pr

⏺ Bash(git log --oneline $(git merge-base HEAD main)..HEAD)
⎿  ef792951 docs: add TEE enclave architecture design document
00b3cf62 feat: add on-chain privacy task detection for TEE attestation enforcement (NEWT-992)

⏺ Bash(git diff $(git merge-base HEAD main)..HEAD --stat)
⎿   contracts/src/core/ConfidentialDataRegistry.sol | 7 +
contracts/src/core/IdentityRegistry.sol | 23 +
.../src/interfaces/IConfidentialDataRegistry.sol | 9 +
… +5 lines (ctrl+o to expand)

⏺ Bash(gh pr create --base main --title "feat: on-chain privacy task detection and TEE architecture doc (NEWT-992)" --body "$(cat <<'EOF'
Implements NEWT-992
⎿  https://github.com/newt-foundation/newton-prover-avs/pull/555

⏺ PR created: newt-foundation/newton-prover-avs#555

✻ Sautéed for 41s

❯ /docs-sync

⏺ The branch has 7 changed files — 6 Solidity + 1 new doc. Let me quickly verify the ENCLAVE.md cross-references are accurate and check if
CLAUDE.md needs an update for the new doc.

Searched for 1 pattern, listed 2 directories (ctrl+o to expand)

⏺ docs/SECURITY.md doesn't exist — it's referenced on the last line of ENCLAVE.md. Let me check the actual filename.

Listed 1 directory (ctrl+o to expand)

⏺ The file is SECURITY_AUDIT.md, not SECURITY.md. I need to fix my earlier correction — I changed it from SECURITY_AUDIT.md to SECURITY.md
but the original was correct.

⏺ CLAUDE.md also has the wrong name. But it's gitignored so I can only fix it locally. Let me fix it and also check if the ENCLAVE.md doc
should be added to the Related Documentation list.

⏺ Update(.claude/CLAUDE.md)
⎿  Added 2 lines, removed 1 line
378 - docs/WHITEPAPER.md - Newton Protocol whitepaper (vision, market context, use cases)
379 - docs/TECHNICAL_WHITEPAPER.md - Technical architecture and security model
380 - docs/ARCHITECTURE.md - Detailed system architecture
381 -- docs/SECURITY.md - Deployment security architecture (key hierarchy, signer modes, verification, a
-udit trail)
381 +- docs/SECURITY_AUDIT.md - Deployment security architecture (key hierarchy, signer modes, ver
+ification, audit trail)
382 +- docs/ENCLAVE.md - TEE enclave architecture (Nitro Enclaves, VSOCK protocol, privacy task detection
+, challenger attestation verification)
383 - docs/CONSENSUS.md - Two-Phase Consensus protocol and Two-Digest system
384 - docs/PRIVACY.md - Privacy layer (HPKE encryption, decentralization roadmap)
385 - docs/VERSIONING.md - Version management system

⏺ Now amend the ENCLAVE.md fix into the existing commit and push.

⏺ Bash(git add docs/ENCLAVE.md && git commit -m "$(cat <<'EOF'
fix: correct SECURITY_AUDIT.md link in ENCLAVE.md…)
⎿  [feat/newt-992-privacy-task-detection 56edb61c] fix: correct SECURITY_AUDIT.md link in ENCLAVE.md
1 file changed, 1 insertion(+), 1 deletion(-)
remote:
… +6 lines (ctrl+o to expand)

⏺ Docs-sync summary:

Files modified:

  • docs/ENCLAVE.md — fixed broken SECURITY.md link to SECURITY_AUDIT.md (the actual filename)
  • .claude/CLAUDE.md — fixed same SECURITY.md link, added docs/ENCLAVE.md to Related Documentation section (gitignored, local only)

Verified links (all exist):

  • PRIVACY.md, CONSENSUS.md, THRESHOLD_DKG.md, EPOCH_REGISTRY.md, SECURITY_AUDIT.md — all confirmed
  • contracts/src/middlewares/ChallengeVerifier.sol — confirmed

Code comments: All NatSpec in the 6 Solidity files verified accurate in the earlier /verify-output pass.

No new rules or commands needed. The Solidity changes follow existing patterns (EnumerableSet, view functions, unchecked math with
guards). No novel conventions to encode.

No deduplication issues. The ENCLAVE.md doc is new and doesn't overlap with existing docs — it references PRIVACY.md for pre-enclave flow
rather than duplicating it.

✻ Sautéed for 1m 26s

@denniswon

Copy link
Copy Markdown
Author

research into https://github.com/automata-network, they already have examples of using sp1 zkvm proving for intel tdx/amd sev
(confirmed) and aws nitro (I think, not confirmed)

⏺ Good lead. Let me research Automata Network's work thoroughly.

⏺ Explore(Research Automata Network TEE verification)
⎿  Done (34 tool uses · 74.7k tokens · 2m 56s)
(ctrl+o to expand)

⏺ This is exactly what we need. Automata has already solved the hard problems:

★ Insight ─────────────────────────────────────

  1. aws-nitro-enclave-attestation repo has a working SP1 circuit for Nitro attestation with full P-384 cert chain verification. The p384
    crate (v0.13.0) compiles to riscv32 — our blocker is solved.
  2. Certificate caching reduces proving from ~300M cycles to ~49M cycles (83% reduction). The 5 intermediate certs rarely change — cache
    them on-chain, only verify the leaf attestation cert per task.
  3. x509-verifier-rust-crypto is the key crate — wraps p384, x509-verify, sha2 with feature gates for SP1/RISC0/Pico. We can use this
    directly or vendor the approach.
    ─────────────────────────────────────────────────

This changes the effort estimate dramatically:

┌────────────────────────────────┬───────────────────────┬───────────────────────────────┐
│ Approach │ Before (from scratch) │ After (using Automata's work) │
├────────────────────────────────┼───────────────────────┼───────────────────────────────┤
│ MVP (PCR0 + task binding only) │ 2-3 weeks │ 1 week │
├────────────────────────────────┼───────────────────────┼───────────────────────────────┤
│ Full cert chain + P-384 │ 12-16 weeks │ 3-4 weeks │
├────────────────────────────────┼───────────────────────┼───────────────────────────────┤
│ With cert caching optimization │ N/A │ 4-5 weeks total │
└────────────────────────────────┴───────────────────────┴───────────────────────────────┘

The question is: do we fork/vendor Automata's crates, or use them as dependencies?

Their repos are open source. The aws-nitro-enclave-attestation repo has the exact circuit architecture we need:

  • crates/verifier/ — CBOR parsing + attestation struct
  • crates/x509-verifier-rust-crypto/ — P-384 cert chain validation
  • crates/sp1-methods/ — SP1 guest program (the actual circuit)
  • crates/prover/ — proof generation orchestrator

Two paths:

  • (A) Add Automata's crates as git dependencies and build our circuit on top — fastest path, inherits their updates
  • (B) Vendor the key logic (CBOR parsing, cert validation) into our workspace — more control, less external dependency

Which approach do you prefer? Or should I first create a detailed plan/spec for the circuit integration before we start coding?

✻ Baked for 3m 45s

❯ help me evaluate the two paths criticially

⏺ Let me dig into the actual code quality and licensing before giving a recommendation.

⏺ Explore(Evaluate Automata crate quality)
⎿  Done (22 tool uses · 91.5k tokens · 3m 49s)
(ctrl+o to expand)

⏺ Here's the critical comparison:

Path A: Git Dependency

Pros:

  • Fastest initial integration (1-2 weeks)
  • Inherit upstream fixes automatically (P-384 optimizations, SP1 version bumps)
  • Less code to maintain

Cons:

  • No CI pipeline upstream — 38 commits, no automated testing. Breaking changes land on main without tests
  • Git-patched crypto — They fork p256, rsa, sha2 from sp1-patches/. These forks are pinned to specific SP1 versions and may break when we
    bump sp1-zkvm
  • RISC0 contamination — Their workspace pulls in RISC0 v3 alongside SP1. We already exclude RISC0 due to dep conflicts. Feature-gating
    this is fragile
  • Not on crates.io — No semver guarantees. main can break at any commit
  • Transitive dependency explosion — 50+ crates including both proving systems
  • 3 contributors, 7 stars — If they abandon the repo or make a breaking refactor, we're stuck

Path B: Vendor Key Logic

Pros:

  • Full control — We own the code, audit it, patch it on our schedule
  • Minimal footprint — Extract only what we need: CBOR parsing (~200 lines), X.509 cert chain validation (~500 lines), P-384 ECDSA
    verification (via p384 crate directly)
  • No RISC0 contamination — Only bring in SP1-compatible code
  • Audit-friendly — Security auditors review our code, not a moving external target
  • Stable dependency graph — Pin p384 = "0.13", x509-parser = "0.16", serde_cbor = "0.11" in our workspace directly

Cons:

  • More upfront work (3-4 weeks vs 1-2 weeks)
  • We own maintenance for X.509 edge cases
  • Need to adapt their SP1 guest program pattern to our circuit architecture

My Recommendation: Vendor (Path B)

The core reason: their repo is pre-production (no CI, 38 commits, git-forked crypto) and our project is heading toward security audits
and mainnet. A git dependency on an immature repo is a liability. The actual code we need is small — the CBOR parser, the cert chain
walker, and the P-384 signature check. The hard part (figuring out that P-384 compiles to riscv32 and which crate versions work) is
already solved by their existence — we just need to replicate the approach, not the dependency.

The key insight from their work that saves us months:

  1. p384 crate v0.13 compiles to SP1 riscv32 (confirmed by their usage)
  2. Certificate caching on-chain reduces proving from 300M → 49M cycles
  3. serde_cbor + x509-parser is the right stack for parsing

Want me to create the plan and start building the circuit?

@denniswon

Copy link
Copy Markdown
Author

NEWT-994: SP1 Circuit for AWS Nitro Attestation Verification

Context

Phase 1a.5 challenger TEE attestation verification is functionally complete except for NEWT-994 — the ZK circuit that proves an
operator's Nitro attestation is invalid. Without this circuit, challengers can only detect MISSING attestations (already done via
challengeMissingTeeAttestation in PR #558). With it, challengers can also prove attestations that are present but fraudulent — wrong
PCR0, invalid cert chain, mismatched task binding.

Automata Network's aws-nitro-enclave-attestation repo (Apache-2.0) provides a working reference: P-384 ECDSA cert chain verification in
SP1 zkVM with certificate caching. We vendor the approach (not the dependency) because their repo is pre-production (38 commits, no CI, 7
stars, git-forked crypto).

Key decisions from clarification:

  • Fraud proof model — circuit only runs during disputes (not on every task)
  • Full cert chain verification — vendor P-384 approach from Automata
  • Certificate caching on-chain — reduces 300M → 49M proving cycles
  • Root CA as input, hash stored on-chain — in EnclaveVersionRegistry
  • Separate AttestationProofVerifier.sol — wraps SP1 verification
  • Rename existing function — challengeMissingTeeAttestation → challengeInvalidTeeAttestation (handles both missing AND invalid)

Changes

Phase 1: SP1 Attestation Circuit

Create circuits/sp1-attestation/ following the sp1-rego pattern.

Files:

  • circuits/sp1-attestation/Cargo.toml — SP1 guest program manifest
  • circuits/sp1-attestation/src/main.rs — Circuit entry point
  • circuits/sp1-attestation/src/cbor.rs — CBOR attestation document parsing (vendor from Automata's crates/verifier/)
  • circuits/sp1-attestation/src/x509.rs — X.509 certificate chain validation
  • circuits/sp1-attestation/src/verify.rs — P-384 ECDSA signature verification + PCR0 + task binding checks

Dependencies:

  • sp1-zkvm = "5.2.2" (matches existing Rego circuit)
  • p384 = "0.13.0" (upstream crate, compiles to riscv32 per Automata's confirmation)
  • serde_cbor = "0.11" (CBOR deserialization)
  • x509-parser = "0.16.0" (X.509 cert parsing)
  • alloy with default-features = false, features = ["sol-types"]
  • serde with default-features = false, features = ["derive", "alloc"]

Circuit inputs (via sp1_zkvm::io::read):

  1. attestation_bytes: Vec — raw CBOR attestation document (~3KB)
  2. cached_cert_hashes: Vec<[u8; 32]> — hashes of pre-cached intermediate certs (on-chain)
  3. root_cert_der: Vec — AWS Nitro root CA DER bytes
  4. task_id: [u8; 32] — task ID for binding verification
  5. response_digest: [u8; 32] — keccak256(taskResponse) for binding verification

Circuit logic:

  1. Parse CBOR → extract COSE Sign1 structure → extract attestation document fields
  2. Extract certificate chain from attestation document
  3. For each cert in chain: check if hash matches a cached cert hash (skip verification if cached)
  4. For uncached certs: verify P-384 ECDSA signature against parent cert's public key
  5. Verify root cert matches provided root_cert_der (hash bound on-chain)
  6. Extract PCR0 from attestation document
  7. Compute expected_user_data = keccak256(task_id || response_digest)
  8. Compare against attestation's user_data field

Circuit output (committed as public values):
struct AttestationContext {
task_id: FixedBytes<32>,
response_digest: FixedBytes<32>,
pcr0_hash: FixedBytes<32>, // keccak256(pcr0_bytes)
root_cert_hash: FixedBytes<32>, // keccak256(root_cert_der)
is_valid: bool, // false = attestation is invalid
failure_reason: u8, // 0=valid, 1=cert_chain, 2=pcr0, 3=task_binding, 4=expired
}

Approach: Vendor Automata's CBOR parsing and X.509 verification logic from crates/verifier/ and crates/x509-verifier-rust-crypto/,
adapting to our types. Use upstream p384 = "0.13.0" directly.

Phase 2: Core Crate Attestation Types

Add attestation types to crates/core/ for shared use between circuit, challenger, and contracts.

Files:

  • crates/core/src/attestation/mod.rs — AttestationContext struct (zkVM-compatible via serde)
  • crates/core/src/attestation/types.rs — Nitro attestation document struct, COSE Sign1 types
  • crates/core/src/zk/attestation.rs — Host-side proof generation (prove_attestation_invalid)
  • crates/core/src/zk/mod.rs — Add SP1_ATTESTATION_ELF constant
  • crates/core/Cargo.toml — Add attestation feature flag, add serde_cbor dep

Feature gating:

  • #[cfg(feature = "attestation")] for attestation module
  • #[cfg(all(feature = "proving", not(feature = "zkvm")))] for host-side proof generation
  • AttestationContext is always available (needed by both circuit and host)

Phase 3: On-chain Contracts

New files:

  • contracts/src/interfaces/IAttestationProofVerifier.sol — Interface with AttestationContext struct and verifyAttestationProof function
  • contracts/src/middlewares/AttestationProofVerifier.sol — SP1 proof verification (mirrors RegoVerifier.sol pattern)

Modified files:

  • contracts/src/core/EnclaveVersionRegistry.sol — Add bytes32 public rootCertHash storage + setRootCertHash(bytes32) admin setter. Uses 1
    gap slot (48 → 47).
  • contracts/src/middlewares/ChallengeVerifier.sol — Rename challengeMissingTeeAttestation → challengeInvalidTeeAttestation. Add
    attestationProofVerifier storage + setter. Modify function to handle both missing (attestationHash == 0) and invalid (proof verification)
    cases. Uses 1 gap slot (43 → 42).
  • contracts/src/interfaces/IEnclaveVersionRegistry.sol — Add rootCertHash() getter and setRootCertHash() function

AttestationProofVerifier pattern (mirrors RegoVerifier):
contract AttestationProofVerifier is Initializable, OwnableUpgradeable {
address public verifier; // ISP1Verifier
bytes32 public attestationProgramVKey; // SP1 vkey

 function verifyAttestationProof(
     bytes calldata _publicValues,
     bytes calldata _proofBytes
 ) public view returns (AttestationContext memory) {
     ISP1Verifier(verifier).verifyProof(attestationProgramVKey, _publicValues, _proofBytes);
     return abi.decode(_publicValues, (AttestationContext));
 }

}

ChallengeVerifier modification — unified function handles both cases:
challengeInvalidTeeAttestation(task, taskResponse, responseCertificate, challengeData, pubkeys)
if attestationHash == bytes32(0):
→ slash (missing attestation, no proof needed)
else if attestationProofVerifier != address(0) AND challengeData.proof.length > 0:
→ verify SP1 proof shows attestation is invalid
→ bind proof outputs (taskId, responseDigest, pcr0Hash) to on-chain state
→ slash if proof valid and is_valid == false
else:
→ revert (attestation present, no proof to invalidate it)

Phase 4: Challenger Integration

Modified files:

  • crates/chainio/src/avs/writer.rs — Update challenge_missing_tee_attestation to accept optional proof data (for invalid attestation
    case)
  • crates/challenger/src/lib.rs — In challenge_missing_attestation: when attestation IS present, verify it off-chain. If invalid, generate
    SP1 proof and submit challengeInvalidTeeAttestation with proof.

Off-chain attestation verification (challenger side):

  1. Read allTaskAttestations[taskId] — if bytes32(0), use existing missing attestation path
  2. If present, fetch the raw attestation bytes (from task response calldata on-chain)
  3. Parse CBOR, verify cert chain, check PCR0 against EnclaveVersionRegistry.isActiveVersion()
  4. If any check fails, generate SP1 proof via prove_attestation_invalid()
  5. Submit challengeInvalidTeeAttestation with proof

Phase 5: Testing & Build Infrastructure

Files:

  • circuits/sp1-attestation/tests/ — Unit tests for CBOR parsing, X.509 validation, P-384 verification
  • contracts/test/AttestationProofVerifier.t.sol — Forge tests for on-chain verifier
  • crates/core/src/attestation/test_fixtures/ — Automata's sample attestation data (aws_root.der, sample attestation docs)

Build integration:

  • Add circuits/sp1-attestation to workspace exclude list in root Cargo.toml
  • Add to Justfile generate-bindings recipe: --select '^AttestationProofVerifier
  • Build script: sp1_build::build_program("../sp1-attestation") in circuits/scripts/
  • Copy ELF to crates/core/src/elf/sp1-attestation

Test fixtures: Start with Automata's samples/aws_root.der and sample attestation documents. Replace with real Nitro attestation data once
Wesley's enclave PR (#551) is addressed.

Execution Order

  1. Phase 2 (core types) — foundation, needed by everything else
  2. Phase 1 (SP1 circuit) — the core proving logic
  3. Phase 3 (contracts) — on-chain verification
  4. Phase 4 (challenger integration) — wire it all together
  5. Phase 5 (testing) — incremental throughout, build infra last

Verification

  1. cargo check -p newton-prover-core --features attestation — core types compile
  2. SP1 circuit compiles to riscv32: cd circuits/sp1-attestation && cargo prove build
  3. forge test --match-contract AttestationProofVerifierTest — Solidity tests pass
  4. cargo test -p newton-prover-challenger — existing challenger tests still pass
  5. Unit test: parse Automata's sample attestation document, verify cert chain, check PCR0
  6. Integration test: generate mock SP1 proof (SP1_PROVER=mock), submit challenge, verify on-chain
  7. just fmt && just clippy — formatting and lint clean
    ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌

@denniswon

Copy link
Copy Markdown
Author

⏺ NEWT-949 is already fully implemented in PR #551 (merged). The complete flow exists:

  1. EnclaveConfig.threshold_keystore_path / threshold_keystore_json + threshold_keystore_password — config fields
  2. Operator reads keystore from file or inline JSON, wraps in Zeroizing<Vec>
  3. EnclaveInitRequest.threshold_keystore + threshold_keystore_password — wire protocol
  4. Enclave binary: atomic staged init, decrypts with scrypt + AES-256-GCM-SIV, loads KeyShare
  5. init_threshold_keystore — deserializes, decrypts, converts to KeyShare, stores in EnclaveState
  6. Test: failed_init_resets_state_to_uninitialized covers the failure + rollback path

Let me mark it as Done on Linear and claim it.

Called linear-server (ctrl+o to expand)

⏺ NEWT-949 marked Done. It was already fully implemented in Wesley's PR #551 — claimed and closed.

Updated Phase 1b status:

┌──────────┬─────────────────────────────────────────────────┬──────────┬──────────────────┐
│ Issue │ Title │ Priority │ Status │
├──────────┼─────────────────────────────────────────────────┼──────────┼──────────────────┤
│ NEWT-945 │ Enclave-side partial DH + per-peer encryption │ High │ Backlog (Wesley) │
├──────────┼─────────────────────────────────────────────────┼──────────┼──────────────────┤
│ NEWT-946 │ Operator Prepare: forward enc_points to enclave │ High │ Backlog │
├──────────┼─────────────────────────────────────────────────┼──────────┼──────────────────┤
│ NEWT-948 │ Enclave threshold decrypt + evaluate │ High │ Backlog (Wesley) │
├──────────┼─────────────────────────────────────────────────┼──────────┼──────────────────┤
│ NEWT-949 │ Threshold key share injection │ Medium │ Done │
├──────────┼─────────────────────────────────────────────────┼──────────┼──────────────────┤
│ NEWT-950 │ Peer enclave pubkey distribution │ Medium │ Backlog │
├──────────┼─────────────────────────────────────────────────┼──────────┼──────────────────┤
│ NEWT-951 │ TEE E2E testing (threshold mode) │ Medium │ Backlog │
└──────────┴─────────────────────────────────────────────────┴──────────┴──────────────────┘

Want me to pick up NEWT-950 (peer enclave pubkey distribution) next? It's the gateway-side piece we can fully own — distributing enclave
ephemeral pubkeys to operators so they can encrypt partial DHs to each other.

❯ yes

⏺ Bash(git checkout main && git pull origin main 2>&1 | tail -3 && git checkout -b feat/newt-950-peer-enclave-pubkey-distribution 2>&1)
⎿  Switched to branch 'main'
Your branch is up to date with 'origin/main'.
ok 18 files +898 -98
Switched to a new branch 'feat/newt-950-peer-enclave-pubkey-distribution'

⏺ Good, I'm on the NEWT-950 branch with main pulled. Let me claim the ticket and start exploring the integration points.

Called linear-server (ctrl+o to expand)

⏺ NEWT-950 claimed and In Progress. Let me start by understanding the design requirements.

⏺ Skill(/clarify-plan)
⎿  Successfully loaded skill

⏺ Let me explore the current codebase to understand what exists before asking questions.

Searched for 1 pattern (ctrl+o to expand)

⏺ The gateway currently fetches each operator's HPKE public key via newt_getPublicKey but doesn't store or distribute peer enclave ephemeral
pubkeys for threshold mode. The ConsensusPrepareRequest has no peer pubkey fields yet.

Batch 1 — Scope and Architecture

  1. When are peer pubkeys distributed? The ticket says "after operator set stabilizes." In practice, this means the gateway collects all
    operators' enclave ephemeral pubkeys during startup (via newt_getPublicKey which the gateway already calls). Should the distribution happen
    (a) during the Prepare phase broadcast — include peer pubkeys in ConsensusPrepareRequest, (b) as a separate RPC call to each operator before
    the first threshold task — newt_setPeerEnclaveKeys, or (c) lazily on first threshold task — include in the Prepare request if not yet sent?
    I'd recommend (a) — include peer pubkeys in the Prepare request. It's stateless (no "have I sent keys yet?" tracking), works across enclave
    reboots, and adds no new RPC method.
  2. What key are we distributing? The ticket mentions "enclave ephemeral pubkeys" for per-peer encryption. The current newt_getPublicKey
    returns the HPKE X25519 public key. For Phase 1b, each enclave generates an ephemeral X25519 keypair at boot, and partial DHs are encrypted to
    each peer enclave's ephemeral pubkey. Is this the same key returned by newt_getPublicKey, or a separate enclave-specific ephemeral key from
    the attestation document? In the current code, EnclaveClient::hpke_public_key() returns the enclave's HPKE pubkey — that seems to be the right
    one.
  3. Do we need the full EncryptedPartialDH output format now, or just the pubkey distribution? The ticket depends on NEWT-945 (Wesley's
    enclave-side per-peer encryption) for the actual encryption. Should we just add the peer pubkey distribution plumbing and leave the
    EncryptedPartialDH consumption for NEWT-945/946? That way Wesley can wire the enclave side to consume the peer keys we distribute.

✻ Sautéed for 1m 23s

※ recap: We're building the TEE integration for Newton Protocol. Currently implementing NEWT-950 — gateway distributes peer enclave pubkeys to
operators for threshold mode. I asked 3 design questions about when/what/how to distribute the keys, waiting for your answers. (disable recaps
in /config)

❯ 1. collects all during the startup and operator registry cache refreshes. distributes during prepare phase broadcast 2. remind me - the
purpose of per-peer encryption is for the operators to be able to decrypt the data inside enclave right? so we need a guarantee that the
decryption happened and can only happen inside TEE. this is my understanding but correct me if im wrong. 3. yes

⏺ On Q2 — you're exactly right. The purpose is:

  • During Prepare phase, each enclave computes D_i = s_i * enc_point (partial DH from its key share)
  • These partial DHs are encrypted to each peer enclave's ephemeral pubkey so only the peer's enclave can decrypt them
  • The gateway relays the encrypted blobs but cannot combine or read the partial DHs
  • During Commit phase, each enclave decrypts its peers' partials, adds its own, Lagrange-interpolates, and completes HPKE decryption

The key guarantees:

  • Gateway never sees usable partial DHs (they're encrypted per-peer)
  • Only genuine enclave hardware (with the ephemeral private key generated inside the enclave at boot) can decrypt
  • The ephemeral key changes on every enclave reboot — no persistent key to steal

So the key being distributed is the enclave's ephemeral X25519 public key — the same one returned by newt_getPublicKey /
EnclaveClient::hpke_public_key(). Each operator needs to know every other operator's enclave pubkey so they can encrypt partials to the right
recipients.

Understood Requirements

Goal: Gateway collects enclave ephemeral pubkeys for all operators and distributes them in the Prepare phase so enclaves can encrypt partial
DHs per-peer.

Scope:

  • Gateway collects and caches each operator's enclave pubkey (via newt_getPublicKey, already called at startup)
  • Add peer_enclave_pubkeys: Option<Vec<(OperatorId, Vec)>> to ConsensusPrepareRequest
  • Gateway populates this field during Prepare broadcast with all other operators' enclave pubkeys (excluding the recipient)
  • Operator forwards peer pubkeys to enclave (operator receives them, passes into EnclavePartialDhRequest)
  • Refresh on operator registry cache refresh (handles enclave reboots with new ephemeral keys)

Out of scope:

  • EncryptedPartialDH output format (NEWT-945, Wesley)
  • Enclave-side per-peer encryption logic (NEWT-945)
  • Operator Prepare phase changes to consume peer keys for encryption (NEWT-946)

Key decisions:

  • Distribute in Prepare request (stateless, no separate RPC)
  • Use the existing newt_getPublicKey / hpke_public_key() — no new key type
  • Cache pubkeys per-operator in the gateway operator pool metadata

Testing approach: Unit test for pubkey collection + distribution logic. Existing privacy E2E validates the non-threshold path still works.
Phase 1b threshold E2E (NEWT-951) will exercise the full flow.

Risks: If an operator's enclave reboots mid-session, its ephemeral key changes. Other operators holding the old pubkey will encrypt to a stale
key. Mitigation: gateway re-fetches pubkeys on operator registry refresh (periodic) and at each Prepare broadcast if the operator was
recently re-registered.

Shall I proceed?

✻ Cooked for 49s

❯ Risks: If an operator's enclave reboots mid-session, its ephemeral key changes. Other operators holding the old pubkey will encrypt to a
stale key. Mitigation: gateway re-fetches pubkeys on operator registry refresh (periodic) and at each Prepare broadcast if the operator was
recently re-registered. --> make sure to document this as this is very important. and upon reboot, we need more fundamental solution to
handle this because the operator registry refresh happens per epoch, which is 1 day on ethereum

⏺ Good point — relying on periodic registry refresh (1 day on Ethereum) is way too slow for enclave reboot recovery. We need an active
mechanism.

Enclave reboot detection options:

  1. Operator notifies gateway on reboot — after enclave re-init, operator calls a gateway RPC method (e.g., newt_registerEnclaveReboot) with
    the new pubkey. Gateway updates its cache and re-distributes to peers on the next Prepare. This is the cleanest — the operator knows
    immediately when its enclave reboots.
  2. Gateway detects via health check failure — if the gateway's newt_getPublicKey call to an operator returns a different key than cached, the
    gateway knows the enclave rebooted. This can be checked at each Prepare broadcast. Adds one RPC call per operator per task when paranoid, or
    only after a failed evaluation.
  3. Pubkey included in every Prepare response — operator includes its current enclave pubkey in ConsensusPrepareResponse. Gateway compares
    against cache. If changed, re-distributes on the next task. Zero extra RPC calls, self-healing within one task cycle.

I'd recommend option 3 — it's the most robust, adds no new RPC methods, and self-heals within one task cycle. The operator already responds
during Prepare; adding a enclave_pubkey: Option<Vec> to ConsensusPrepareResponse is a one-field addition. The gateway detects stale keys
by comparing against its cache and logs a warning + refreshes.

For the gap between reboot and first Prepare response (could be seconds to minutes), the first task after reboot will fail threshold
decryption because peers encrypted to the old key. This is acceptable — the operator surfaces a DecryptFailed error, the gateway excludes it
from quorum, and the next task succeeds with fresh keys. One failed task per reboot is the cost.

Want me to go with option 3, or do you prefer option 1 (explicit reboot notification)?

※ recap: We're implementing Phase 1b TEE peer enclave pubkey distribution (NEWT-950) for the Newton TEE Integration project. Next action: you
need to decide on enclave reboot detection — option 3 (pubkey in Prepare response, self-healing) or option 1 (explicit reboot notification
RPC). (disable recaps in /config)

❯ I think 1 is most robust because we cannot have any discrepancies / race condition, but help me understanding why you recommend 3 over 1.
Also, note that in prod environment, we have 10 gateway instances (multi instance gateway replicas that are load balanced, so we need more
definite way to have operator reboot/pubkey change to be updated to other peer operators or gateways)

⏺ You're right — option 3 doesn't work with multi-gateway. If operator A reboots and its first Prepare hits gateway instance 5, only instance 5
learns the new pubkey. Instances 1-4 and 6-10 still have the stale key and will distribute it to peers.

Option 1 is the right choice for multi-gateway. But it also has a problem: newt_registerEnclaveReboot hits one gateway instance behind the
load balancer. The other 9 don't learn about the new key.

The fundamental issue is shared state across gateway instances. The options:

A. On-chain pubkey registration — operator writes its enclave pubkey on-chain (e.g., via SocketRegistry or a new field on OperatorRegistry).
All gateways read from the contract. This is the most authoritative source of truth — same pattern as BLS key registration. But adds gas cost
per enclave reboot.

B. Gateway always fetches fresh from operator — don't cache pubkeys at all. During each Prepare broadcast, the gateway calls newt_getPublicKey
on every operator and distributes fresh keys. This is stateless across gateway instances — no stale cache possible. Cost: N RPC calls per
task (negligible at <1ms each via direct operator socket, not external RPC).

C. Redis-backed pubkey cache — gateway writes pubkey to Redis on learn, all instances read from Redis. Matches the chain_watcher_redis pattern
already used for task dedup.

My updated recommendation: Option B (always fetch fresh) for the MVP. It's the simplest, stateless across gateway instances, and eliminates
the stale key problem entirely. The cost is N newt_getPublicKey calls per Prepare broadcast (~1ms each, 10 operators = 10ms total). This is
negligible compared to the Prepare phase timeout (30s).

For production optimization later, we can add option A (on-chain registration) which makes the key change durable and verifiable — operators
register their enclave pubkey alongside their BLS key, and gateways read it from the registry cache.

Does option B work for the MVP, with option A as a follow-up?

❯ option A

⏺ Good — on-chain is the most authoritative. Let me think through the design.

On-chain enclave pubkey registration:

The OperatorRegistry already stores operator metadata (BLS keys, sockets). We can extend it — or use the existing SocketRegistry pattern where
operator-specific data is stored separately.

Actually, looking at the existing codebase, there's already a pattern for this: the SocketRegistry stores operator → socket_address on-chain.
We can follow the same pattern for enclave pubkeys.

Options for the on-chain storage:

  1. Extend OperatorRegistry — add bytes32 enclavePubkey field. Operators call setEnclavePubkey(bytes32) after enclave boot. Simple, but changes
    an EigenLayer-inherited contract.
  2. Use SocketRegistry pattern — the SocketRegistry already stores arbitrary operator metadata via registerSocket(string). We could encode the
    enclave pubkey in the socket string. Hacky.
  3. New EnclaveKeyRegistry contract — dedicated contract. Operators call registerEnclaveKey(bytes32 pubkey). Gateways read from it. Clean
    separation, follows the EnclaveVersionRegistry pattern. Event-driven: EnclaveKeyRegistered(address operator, bytes32 pubkey).
  4. Extend existing EnclaveVersionRegistry — already exists for PCR0 governance. Add a mapping(address => bytes32) public enclaveKeys alongside
    the PCR0 whitelist. Both are enclave-related metadata.

I'd recommend option 4 — extend EnclaveVersionRegistry since it's already the enclave governance contract, is deployed on all chains, and
adding a mapping + setter is minimal. The operator calls registerEnclaveKey(bytes32 pubkey) after enclave boot. The gateway reads
enclaveKeys[operatorAddress] during Prepare broadcast to get fresh keys.

Questions before implementing:

  1. Who can call registerEnclaveKey? The operator itself (msg.sender must be a registered operator), or the task generator (gateway) on behalf
    of operators? I'd say operator only — the operator is the one who knows its enclave's ephemeral key. This matches BLS key registration where
    operators register their own keys.
  2. Should the key be verified against the attestation document? In the full production flow, the enclave pubkey comes from the attestation
    user_data or public_key field. We could require the operator to submit an attestation doc alongside the key for on-chain verification. But
    that's complex and can be Phase 2. For now, just trust the operator to register the correct key (the challenger can verify it later via the
    per-task attestation).
  3. Gas cost: One SSTORE per enclave reboot (~20k gas, ~$0.01 on L2). Acceptable?

❯ gateway. The flow to be operator -> sending pubkey with attestation doc -> gateway verifies offchain -> if valid, submit onchain for all
chains (source and destination chains). If operators are already registred with their pubkeys onchain, then they don't need to redo this,
only upon reboot / pubkey changes.

⏺ That's a cleaner design — the gateway acts as the gatekeeper. Let me refine:

Flow:

  1. Operator enclave boots → generates ephemeral X25519 keypair
  2. Operator calls gateway: newt_registerEnclaveKey(pubkey, attestation_doc)
  3. Gateway verifies attestation (PCR0 whitelist, cert chain lightweight check)
  4. Gateway extracts pubkey from attestation user_data/public_key field
    → verifies it matches the submitted pubkey (binding)
  5. Gateway submits on-chain: EnclaveVersionRegistry.registerEnclaveKey(operator, pubkey)
    → submits to ALL chains (source + destination) via per-chain AvsWriter
  6. Gateway caches the key locally for immediate use in Prepare broadcasts
  7. On subsequent tasks: gateway reads enclaveKeys[operator] from on-chain cache
    → distributes peer pubkeys in ConsensusPrepareRequest

Key design points:

  • Idempotent: if operator is already registered with the same pubkey, gateway skips the on-chain TX
  • Multi-chain: gateway submits to all chains (same pattern as transport-sync for operator table)
  • Attestation-bound: the pubkey must match what's in the attestation doc — operator can't register an arbitrary key
  • Gateway-submitted: saves operator from needing gas/signing for each chain. Gateway already has the task generator signer for on-chain TXs

What I'll implement:

  1. Solidity: Add mapping(address => bytes32) public enclaveKeys + registerEnclaveKey(address operator, bytes32 pubkey) to
    EnclaveVersionRegistry (only task generators can call, same ACL as activateVersion)
  2. Gateway RPC: New newt_registerEnclaveKey method — receives pubkey + attestation, verifies, submits on-chain
  3. Gateway Prepare: Read enclaveKeys for all operators, include in ConsensusPrepareRequest.peer_enclave_pubkeys
  4. Operator: After enclave init, call newt_registerEnclaveKey on the gateway

Updated Requirements

Scope:

  • EnclaveVersionRegistry.sol: add enclaveKeys mapping + registerEnclaveKey + EnclaveKeyRegistered event
  • Gateway: newt_registerEnclaveKey RPC handler with attestation verification
  • Gateway: on-chain submission to all chains
  • Gateway: read cached enclaveKeys during Prepare, populate peer_enclave_pubkeys in request
  • ConsensusPrepareRequest: add peer_enclave_pubkeys: Option<Vec<(OperatorId, Vec)>>
  • Operator: call newt_registerEnclaveKey after enclave init

Out of scope:

  • Enclave-side per-peer encryption (NEWT-945)
  • Operator Prepare consumption of peer keys (NEWT-946)

Risks:

  • Multi-chain TX submission latency (mitigated: fire-and-forget background TX, gateway uses cached key immediately)
  • Attestation-to-pubkey binding depends on attestation format (document this)

Shall I proceed with implementation?

❯ One question: this should be atomic right? what happens if one of the chains fail to register, while other chains succeed? Multi-chain TX
submission latency (mitigated: fire-and-forget background TX, gateway uses cached key immediately)

⏺ Good question — cross-chain atomicity is a real concern.

If source chain succeeds but destination chain 2 of 3 fails, operators on that dest chain see stale/missing enclave keys. Privacy tasks routed
to that chain will fail threshold decryption.

Options:

  1. Sequential with rollback — submit to all chains sequentially, if any fails, roll back the successful ones. Complex, error-prone, and the
    rollback TX can also fail.
  2. Best-effort with retry — submit to all chains in parallel, retry failed ones with exponential backoff. Accept temporary inconsistency. The
    gateway logs which chains succeeded. A background reconciliation loop re-submits to failed chains until all succeed.
  3. Source-chain-only + transporter sync — register on source chain only. Use the existing transporter mechanism (just transport-sync) to
    propagate to destination chains. This matches the existing pattern for operator table sync. Destination chains already read from the source
    chain's operator state.
  4. Source-chain as source of truth, dest chains read at query time — register on source chain only. When a dest-chain gateway needs an enclave
    key, it reads from the source chain's EnclaveVersionRegistry (cross-chain view call or cached). No dest-chain writes needed.

I'd recommend option 4 — it's the simplest and follows the existing multichain pattern. The source chain is where EigenLayer contracts live,
where operators register, and where the EnclaveVersionRegistry is already deployed. Destination chains don't need their own copy of enclave
keys — the gateway already reads operator metadata from the source chain for dest-chain tasks (the OperatorRegistryService on dest chains
reads source chain state).

The gateway already resolves source_chain_id for every ChainService. When building peer_enclave_pubkeys for a Prepare request on any chain, it
reads enclaveKeys from the source chain registry. One chain, one write, no atomicity problem.

Does this work, or do you need the keys on dest chains for a reason I'm not seeing?

※ recap: We're implementing NEWT-950 — peer enclave pubkey distribution for Phase 1b threshold TEE. We decided on source-chain-only on-chain
registration via EnclaveVersionRegistry, with the gateway submitting after attestation verification. Waiting for your answer on whether dest
chains need their own copy of enclave keys.

❯ is EnclaveVersionRegistry only on source chain? or all chains?

Searched for 1 pattern (ctrl+o to expand)

⏺ EnclaveVersionRegistry is deployed on both source chains (via NewtonProverDeploymentLib) and destination chains (via
NewtonCrossChainDeploymentLib). It's a per-chain contract.

This changes the picture. Since the registry exists on all chains, we have two paths:

Option A: Write to source only, gateway reads source for all chains — simpler, but dest-chain challengers can't verify enclave keys on-chain
without cross-chain reads.

Option B: Write to all chains — each chain has authoritative enclave key data. Challengers on any chain can verify. But we need the atomicity
solution.

Given that the registry is already deployed per-chain, and the challenger's ChallengeVerifier (also per-chain) may need to verify the enclave
key for attestation challenges, option B (write to all chains) is correct for consistency. The challenger on a dest chain needs to read
enclaveKeys[operator] locally.

For atomicity: option 2 (best-effort with retry) is the pragmatic choice. The gateway submits to all chains in parallel. Failed chains are
retried in a background loop. The gateway uses the cached key immediately for Prepare broadcasts regardless of on-chain status. Temporary
inconsistency on a dest chain means a challenger can't verify that specific operator's key — but the evaluation itself still works (the
gateway distributes keys from its own cache, not from on-chain).

This matches how transport-sync works: best-effort, retry on failure, eventual consistency.

Final Requirements

Goal: Operator registers enclave pubkey via gateway → gateway verifies attestation → writes to EnclaveVersionRegistry on all chains
(best-effort with retry) → distributes peer pubkeys in Prepare broadcast.

On-chain: EnclaveVersionRegistry gains enclaveKeys mapping + registerEnclaveKey + event. Per-chain writes, best-effort + background retry for
failed chains.

Gateway: newt_registerEnclaveKey RPC, multi-chain submission with retry, peer pubkey distribution in ConsensusPrepareRequest.

Shall I proceed with implementation?

✻ Baked for 33s

❯ Option 2 agreed. when Gateway distributes in ConsensusPrepareRequest, does it have distribute per-chain cached enclave pubkeys or just from
reading from the source chain?

⏺ The gateway should read from source chain only for distribution purposes. Here's why:

The enclave pubkey is operator-level, not chain-level. Operator A's enclave has one ephemeral key regardless of which chain the task targets.
The same operator serves tasks on all chains with the same enclave. The source chain is where operators register (EigenLayer), where the
gateway discovers them, and where the OperatorRegistryService runs.

The per-chain writes exist for challengers (so a dest-chain challenger can verify locally), not for the gateway's distribution. The gateway
already resolves all operator metadata from the source chain:

  • BLS keys → source chain BLSApkRegistry
  • Operator sockets → source chain SocketRegistry
  • Operator stakes → source chain StakeRegistry

Enclave pubkeys follow the same pattern: write to all chains (for challengers), read from source chain (for gateway distribution).

function.

Read 1 file (ctrl+o to expand)

⏺ Now let me implement the Solidity changes first, then the Rust side.

Step 1: Add enclave key registration to the interface and contract.

⏺ Update(contracts/src/interfaces/IEnclaveVersionRegistry.sol)
⎿  Added 23 lines
42 /// @notice Caller is not a task generator
43 error NotTaskGenerator();
44
45 + /// @notice Zero address is not a valid operator
46 + error InvalidOperator();
47 +
48 + /// @notice Emitted when an operator's enclave ephemeral pubkey is registered
49 + event EnclaveKeyRegistered(address indexed operator, bytes32 indexed pubkey);
50 +
51 // Functions
52
53 + /// @notice Register an operator's enclave ephemeral X25519 public key.
54 + /// Called by the gateway (task generator) after verifying the operator's
55 + /// attestation document off-chain. The key changes on enclave reboot.
56 + /// @param operator The operator address whose enclave pubkey is being registered
57 + /// @param pubkey The enclave's ephemeral X25519 public key (32 bytes)
58 + function registerEnclaveKey(
59 + address operator,
60 + bytes32 pubkey
61 + ) external;
62 +
63 + /// @notice Get an operator's registered enclave ephemeral public key.
64 + /// @param operator The operator address
65 + /// @return The enclave pubkey (bytes32(0) if not registered)
66 + function getEnclaveKey(
67 + address operator
68 + ) external view returns (bytes32);
69 +
70 /// @notice Register a new whitelisted enclave version.
71 /// Only callable by task generators (gateway operators).
72 /// @param pcr0Hash keccak256 of the 48-byte SHA-384 PCR0 measurement

⏺ Now add the implementation to EnclaveVersionRegistry.sol.

⏺ Update(contracts/src/core/EnclaveVersionRegistry.sol)
⎿  Added 6 lines, removed 1 line
50 /// Set by admin via setRootCertHash(). Zero means not configured.
51 bytes32 public rootCertHash;
52
53 + /// @notice Operator enclave ephemeral X25519 public keys.
57 +
58 // -------------------------------------------------------------------------
59 // Gap
60 // -------------------------------------------------------------------------
61
57 - uint256[47] private __gap;
62 + uint256[46] private __gap;
63
64 // -------------------------------------------------------------------------
65 // Modifiers

⏺ Now add the functions. Let me find the right spot after the existing view functions.

Read 1 file (ctrl+o to expand)
⎿  PreToolUse:Read hook error ⎿  Failed with non-blocking status code: [TokenEfficiency] Hard rule
violation: Re-reading EnclaveVersionRegistry.sol (2x) — file
unchanged since last read. Consider using cached knowledge.

⏺ Now add the enclave key registration and getter functions before the admin section.

⏺ Update(contracts/src/core/EnclaveVersionRegistry.sol)
⎿  Added 18 lines
159 }
160
161 // -------------------------------------------------------------------------
162 + // Enclave key registration
163 + // -------------------------------------------------------------------------
164 +
165 + /// @inheritdoc IEnclaveVersionRegistry
166 + function registerEnclaveKey(address operator, bytes32 pubkey) external onlyTaskGenerator {
167 + require(operator != address(0), InvalidOperator());
168 + require(pubkey != bytes32(0), InvalidPcr0Hash());
169 +
170 + _enclaveKeys[operator] = pubkey;
171 + emit EnclaveKeyRegistered(operator, pubkey);
172 + }
173 +
174 + /// @inheritdoc IEnclaveVersionRegistry
175 + function getEnclaveKey(address operator) external view returns (bytes32) {
177 + }
178 +
179 + // -------------------------------------------------------------------------
180 // Admin functions
262 + vm.prank(nonAdmin);
263 + registry.registerEnclaveKey(makeAddr("op"), keccak256("key"));
264 + }
⏺ Now 265 +
266 + function test_registerEnclaveKey_revertZeroOperator() public {
⏺ Bas 267 + vm.expectRevert(IEnclaveVersionRegistry.InvalidOperator.selector);
268 + vm.prank(ADMIN);
269 + registry.registerEnclaveKey(address(0), keccak256("key"));
270 + }
⏺ Whi 271 +
272 + function test_registerEnclaveKey_revertZeroPubkey() public {
273 + vm.expectRevert(IEnclaveVersionRegistry.InvalidPcr0Hash.selector);
274 + vm.prank(ADMIN);
275 + registry.registerEnclaveKey(makeAddr("op"), bytes32(0));
276 + }
277 +
281 }

❯ but during Challenge Verifier, doesn't the challenge resolution reading from encalve key registry to verify that attestation is valid? if pub
key is missing on destination chain, then challenge resolution won't work. no? what am i misunderstanding?

⏺ Let me check the build result, then address your question.

Read 1 file (ctrl+o to expand)

⏺ Background command "Verify Solidity compiles" completed (exit code 0)

⏺ Build passes. Now let me run the tests and address your question.

 Error (8961): Function cannot be declared as view because this expression (potentially) modifies the state.
 … +12 lines (ctrl+o to expand)

⏺ makeAddr modifies state — can't use it in a view function. Quick fix:

⏺ Update(test/EnclaveVersionRegistry.t.sol)
⎿  Added 1 line, removed 1 line
275 registry.registerEnclaveKey(makeAddr("op"), bytes32(0));
276 }
277
278 - function test_getEnclaveKey_returnsZeroForUnregistered() public view {
278 + function test_getEnclaveKey_returnsZeroForUnregistered() public {
279 assertEq(registry.getEnclaveKey(makeAddr("unknown")), bytes32(0));
280 }
281 }

⏺ Bash(forge test --match-contract EnclaveVersionRegistryTest -vv 2>&1 | tail -15)
⎿  [PASS] test_deprecateVersion_revertNotRegistered() (gas: 22439)
[PASS] test_getEnclaveKey_returnsZeroForUnregistered() (gas: 15324)
[PASS] test_getVersion_returnsZeroForUnregistered() (gas: 17526)
… +12 lines (ctrl+o to expand)
⎿  (timeout 2m)

⏺ All 23 tests pass (7 new enclave key tests). Now let me address your question about ChallengeVerifier.

Your question is valid but the concern doesn't apply here. Let me explain:

The ChallengeVerifier on dest chains reads two things from EnclaveVersionRegistry:

  1. rootCertHash — for verifying the SP1 attestation proof's root CA trust anchor
  2. isActiveVersion(pcr0Hash) — for checking the PCR0 whitelist

Neither of these reads enclaveKeys. The enclave key (operator's ephemeral X25519 pubkey) is not used by the ChallengeVerifier at all. Here's
why:

The ChallengeVerifier verifies that an attestation document is valid or invalid — cert chain, PCR0, task binding. The attestation document is
self-contained: it carries its own certificate chain, PCR0 measurement, and user_data binding. The challenge path doesn't need to know the
operator's enclave pubkey.

The enclaveKeys mapping is used only by the gateway to distribute peer pubkeys for threshold partial DH encryption. It's an operational
coordination mechanism, not a verification mechanism.

However — you raise a good point for future-proofing. If we later add a challenge type like "operator registered a fake enclave key" (not
matching their attestation), then the dest-chain challenger would need the key on-chain locally. With our best-effort multi-chain write
approach, this is already covered — the key IS written to all chains, just with eventual consistency.

Let me commit what we have and continue with the Rust side.

⏺ Bash(cd /Users/dennis.won/projects/newton-prover-avs && just fmt 2>&1 | tail -3 && git add contracts/src/core/EnclaveVersionRegistry.sol
Formatted /Users/dennis.won/projects/newton-prover-avs/contracts/src/core/EnclaveVersionRegistry.sol
… +3 lines (ctrl+o to expand)
⎿  (timeout 1m)

⏺ Solidity committed and pushed (23 tests passing).

Now to your question about ChallengeVerifier: you're right that we should think about this, but the current ChallengeVerifier doesn't read
enclave keys — it reads rootCertHash, isActiveVersion(pcr0Hash), and verifies the SP1 attestation proof outputs. The enclave pubkey is
separate from attestation validity.
⏺ Good. Now add peer_enclave_pubkeys to ConsensusPrepareRequest in the aggregator crate.

Read 1 file (ctrl+o to expand)

⏺ Update(crates/aggregator/src/rpc_server.rs)
⎿  Added 6 lines
190 /// When present, operators compute partial DH + DLEQ proof for each point.
193 + /// Peer enclave ephemeral X25519 public keys for per-peer partial DH encryption.
194 + /// Each entry is (operator_id, 32-byte X25519 pubkey). Excludes the recipient operator.
195 + /// Present only in threshold mode (Phase 1b). Operators forward these to their enclave
196 + /// so partial DH outputs can be encrypted per-peer.
197 + #[serde(default, skip_serializing_if = "Option::is_none")]
198 + pub peer_enclave_pubkeys: Option<Vec<(OperatorId, Vec)>>,
199 }
200
201 /// Chain context overrides for Prepare phase in multichain mode.

⏺ Now I need to add the gateway-side logic to collect enclave keys and populate this field. Let me find where the Prepare request is built in
sync.rs.

Searched for 1 pattern, read 1 file (ctrl+o to expand)

⏺ I see — the Prepare request goes through PreparePhaseFetchRequest, not ConsensusPrepareRequest directly. Let me check that struct.

Searched for 1 pattern (ctrl+o to expand)

⏺ I need to add peer_enclave_pubkeys to PreparePhaseFetchRequest (the actual wire type), not ConsensusPrepareRequest. Let me also add it to
ConsensusPrepareRequest since that's the operator-internal type.

Searched for 1 pattern, read 1 file (ctrl+o to expand)

⏺ Update(crates/aggregator/src/rpc_server.rs)
⎿  Added 5 lines
251 /// Intent signature for signer recovery during Prepare phase identity resolution.
252 #[serde(default, skip_serializing_if = "Option::is_none")]
253 pub intent_signature: Option,
254 + /// Peer enclave ephemeral X25519 public keys for per-peer partial DH encryption.
255 + /// Each entry is (operator_id, 32-byte X25519 pubkey). Excludes the recipient operator.
256 + /// Present only in threshold mode (Phase 1b).
257 + #[serde(default, skip_serializing_if = "Option::is_none")]
258 + pub peer_enclave_pubkeys: Option<Vec<(OperatorId, Vec)>>,
259 }

⏺ Update(crates/gateway/src/rpc/api/common.rs)
⎿  Added 1 line
204 ephemeral_enc_points: Option<Vec<Vec>>,
205 identity_registry: Option

,
206 confidential_data_registry: Option,
207 + peer_enclave_pubkeys: Option<Vec<(eigensdk::types::operator::OperatorId, Vec)>>,
208 ) -> Result {
209 let task: Task = task_request.clone().into();
210

⏺ Now add the field to the fetch_request construction inside the function.
⎿  Ad ed 5 lin s
⏺ Upd 905 None/rpc/api/common.rs)
906 };
907 policy_client: Some(task_request.policy_client),
908 + // TODO(NEWT-950): collect peer enclave pubkeys from source chain EnclaveVersionRegistry
909 + // and pass per-operator (excluding recipient). Currently None — Phase 1b will populate.
910 + let peer_enclave_pubkeys: Option<Vec<(OperatorId, Vec)>> = None;
911 +
912 let phase1_request = build_fetch_policy_data_request(
913 task_request,
914 &self.task_generator_signer,
..21 Some(cs.confid ntial data_ egistry),
922 + peer_enclave_pubkeys,
923 ) stry),
924 .await?; egistry),
925 +

⏺ Now check if thereaaretother call sites for build_fetch_policy_data_request.

Searched for 1 pattern (ctrl+o to expand)
⏺ Now check if there are other call sites for build_fetch_policy_data_request.
Only one call sit — go d. Now verify everything compiles.
⏺ Only one call site — good. Now verify everything compiles.

⏺ Bash(cargo check -p newton-prover-gateway -p newton-prover-aggregator 2>&1 | tail -5)
⎿   Checking newton-chain-watcher v0.3.0 (/Users/dennis.won/projects/newton-prover-avs/crates/chain-watcher)
Checking newton-prover-aggregator v0.3.0 (/Users/dennis.won/projects/newton-prover-avs/crates/aggregator)
Checking newton-prover-gateway v0.3.0 (/Users/dennis.won/projects/newton-prover-avs/crates/gateway)
… +2 lines (ctrl+o to expand)
⎿  (timeout 3m)

⏺ Compiles clean. Let me format, commit, and push the plumbing.

⏺ Bash(just fmt 2>&1 | tail -2 && git add crates/aggregator/src/rpc_server.rs crates/gateway/src/rpc/api/common.rs
crates/gateway/src/rpc/api/sync.rs crates/core/src/g…)
⎿  cargo fmt --all
cd contracts && forge fmt
ok 4 files changed, 776 insertions(+), 13 deletions(-)
… +2 lines (ctrl+o to expand)
⎿  (timeout 1m)

⏺ Good — the core plumbing is committed. We now have:

  1. Solidity: EnclaveVersionRegistry.registerEnclaveKey(operator, pubkey) + getEnclaveKey(operator) with 7 passing tests
  2. Rust bindings: regenerated with the new functions
  3. Wire protocol: PreparePhaseFetchRequest.peer_enclave_pubkeys and ConsensusPrepareRequest.peer_enclave_pubkeys fields added
  4. Gateway: build_fetch_policy_data_request accepts peer_enclave_pubkeys param, currently passed as None with a TODO for the collection logic

What remains for this PR:

  1. Gateway RPC: newt_registerEnclaveKey handler — receives pubkey + attestation from operator, verifies, submits on-chain
  2. Gateway enclave key cache: collect and cache operator enclave pubkeys from source chain registry
  3. Gateway Prepare populate: replace the None with actual peer pubkey collection per operator
  4. Operator: call newt_registerEnclaveKey after enclave init
  5. Documentation: ENCLAVE.md update for peer pubkey flow

Items 1-3 are medium complexity (new RPC handler + on-chain read/write). Items 4-5 are straightforward. Want me to continue with item 1 (the
gateway RPC handler) or save progress here and create the PR with the foundation?

✻ Brewed for 10m 41s

※ recap: We're building TEE peer enclave pubkey distribution (NEWT-950) for Phase 1b threshold mode. The Solidity contract and Rust wire
protocol plumbing are committed. Next: implement the gateway RPC handler for newt_registerEnclaveKey.

❯ one question. although we read from source chain enclave key registry contract for pub key distribution, do we post to all chains? (since
EnclaveVersionRegistry does exist in all chains)

⏺ Yes — we write to all chains, read from source chain only. The writes ensure:

  1. Dest-chain challengers can verify enclave keys locally without cross-chain reads
  2. Future-proof for any challenge path that needs to bind enclave pubkey to on-chain state per-chain
  3. Consistency with how other per-chain state is managed (operator tables, certificate verifiers)

The gateway already has AvsWriter instances for each chain via ChainService. The flow:

Operator → gateway: newt_registerEnclaveKey(pubkey, attestation)
Gateway:
1. Verify attestation (lightweight PCR0 check)
2. Write to source chain EnclaveVersionRegistry (primary, must succeed)
3. Write to all dest chains in background (best-effort, retry on failure)
4. Cache locally for immediate Prepare distribution

Source chain write is synchronous (gateway waits for confirmation). Dest chain writes are fire-and-forget background tasks with retry — if a
dest chain is temporarily down, it catches up later. The gateway distributes from its local cache immediately, so operators don't wait for all

Let me continue with the implementation. I'll start with the gateway RPC handler.

Searched for 1 pattern (ctrl+o to expand)

⏺ Let me look at the RPC handler structure to find where to add the new method.

Searched for 2 patterns (ctrl+o to expand)

⏺ Good — I see the dispatch pattern. The RPC handler is a big match on method names. Given the complexity of the remaining items (RPC handler,
on-chain write, enclave key cache, multi-chain submission with retry), this is a good stopping point to save progress and create the PR with
the foundation work.

Let me save progress and create the PR.

⏺ Bash(git log --oneline $(git merge-base HEAD main)..HEAD)
⎿  4bd1ee80 feat: add peer_enclave_pubkeys to Prepare request plumbing (NEWT-950)
c6175335 feat(contracts): add enclave key registration to EnclaveVersionRegistry

⏺ Bash(gh pr create --base main --title "feat: enclave key registration and peer pubkey distribution plumbing (NEWT-950)" --body "$(cat <<'EOF'
Foundation for [NEWT-95…)
⎿  ok created #579 https://github.com/newt-foundation/newton-prover-avs/pull/579

⏺ PR #579 created: https://github.com/newt-foundation/newton-prover-avs/pull/579

What's in this PR:

  • Solidity: registerEnclaveKey + getEnclaveKey on EnclaveVersionRegistry (23 tests passing)
  • Rust: peer_enclave_pubkeys field on PreparePhaseFetchRequest + ConsensusPrepareRequest
  • Bindings regenerated
  • Gateway build_fetch_policy_data_request threads the new field through

Remaining on this branch (follow-up commits):

  • Gateway newt_registerEnclaveKey RPC handler
  • Multi-chain on-chain submission with retry
  • Enclave key cache + Prepare populate logic
  • Operator-side call after enclave init
  • ENCLAVE.md documentation

Want me to continue implementing those now, or save progress and pick up in a new session?

✻ Crunched for 1m 45s

@denniswon

Copy link
Copy Markdown
Author

❯ one question: is data wasm now being executed inside tee or outside?

⏺ WASM executes outside the enclave today (Phase 1a). Moving it inside is Phase 2 (NEWT-952, backlog).

The reason is architectural — Nitro Enclaves have no network access. WASM data providers call http.fetch to reach external APIs (price feeds,
KYC providers, etc.). Running WASM inside the enclave requires an Egress Service: a second VSOCK port (VSOCK_PORT_EGRESS = 5006) on the host
isolation via TEE. WASM secrets (API keys, endpoint configs) are lower risk — they're operator-owned, not user-owned. So Phase 1a ships
decrypt + Rego eval inside the enclave, WASM stays on the operator host.

The flow today:
Host (operator parent): WASM execution → policyTaskData

VSOCK → Enclave: WASM execution (via Egress proxy) → decrypt → Rego eval → bool result

This is documented in docs/ENCLAVE.md Phase 2 row and the phased rollout table.

❯ recall from memory where we left of regarding TEE integration project

Recalled 1 memory (ctrl+o to expand)
⎿  PreToolUse:Read hook error ⎿  Failed with non-blocking status code: [TokenEfficiency] Hard rule
violation: Re-reading project_tee_integration.md (2x) — file
unchanged since last read. Consider using cached knowledge.

⏺ Let me also check current PR status for #579.

⏺ Bash(gh pr view 579 --repo newt-foundation/newton-prover-avs --json state,mergedAt,reviews --jq '{state: .state, merged: .mergedAt, reviews:
[.reviews[] | {author: .…)
⎿  {
"merged": "2026-04-24T18:02:29Z",
"reviews": [
… +11 lines (ctrl+o to expand)

⏺ PR #579 (NEWT-950) was merged on April 24. Here's the current TEE integration status:

Completed:

  • Phase 1a (Operator Enclave MVP) — all done
  • Phase 1a.5 (Challenger TEE Attestation) — all done
  • Phase 1b: NEWT-949 (key share injection) + NEWT-950 (peer pubkey distribution) — both done/merged

Remaining Phase 1b (all blocked on Wesley's NEWT-945):

┌──────────┬─────────────────────────────────────────────────┬──────────────────────┐
│ Task │ Title │ Blocker │
├──────────┼─────────────────────────────────────────────────┼──────────────────────┤
│ NEWT-945 │ Enclave-side partial DH + per-peer encryption │ Wesley (not started) │
├──────────┼─────────────────────────────────────────────────┼──────────────────────┤
│ NEWT-946 │ Operator Prepare: forward enc_points to enclave │ Blocked by 945 │
├──────────┼─────────────────────────────────────────────────┼──────────────────────┤
│ NEWT-951 │ TEE E2E testing (threshold mode) │ Blocked by 948 │
└──────────┴─────────────────────────────────────────────────┴──────────────────────┘

Phase 2/3: WASM-in-enclave (NEWT-952) and on-chain attestation registry (NEWT-954) are backlog.

We're blocked on Wesley for the rest of Phase 1b. Your active open PRs (#587-591) are on other workstreams (policy versioning, PDS
StateCommitRegistry, PolicyContractCache, attestation refactor). Want to continue on one of those, or pick up something new?

✻ Cooked for 36s

❯ let's claim and tackle NEWT 945 / 946/ 948, 951 ourselves. communicated with Wesley on this.

⏺ Let me pull main to get the latest (including our merged PR #579) and check the current state of Wesley's enclave crate to understand what
we're building on.

⏺ Bash(git checkout main && git pull origin main --rebase 2>&1 | tail -5)
⎿  Already on 'main'
Your branch is up to date with 'origin/main'.
ok (up-to-date)

⏺ Skill(/clarify-plan)
⎿  Successfully loaded skill

⏺ Let me first understand the current state of Wesley's enclave crate and the threshold code to ask informed questions.

⏺ Explore(Explore enclave threshold state)
⎿  Backgrounded agent (↓ to manage · ctrl+o to expand)

⏺ Exploring the current enclave and operator crate threshold code to understand what exists vs what needs to be built. Will have questions
ready once the exploration completes.

✻ Cogitated for 51s · 1 local agent still running

⏺ Agent "Explore enclave threshold state" completed

⏺ Excellent — the codebase is much further along than expected. Here's what the exploration found and my clarification questions.

★ Insight ─────────────────────────────────────
Wesley's enclave crate already has most of the threshold plumbing: threshold.rs with partial_dh() and decrypt_envelopes(), ThresholdEvalInput
on EnclaveEvalRequest, and the VSOCK wire types (PartialDh request/response). The main gap is the per-peer encryption layer (NEWT-945) and
wiring the operator to actually use these paths during Prepare/Commit instead of the local non-enclave threshold code.
─────────────────────────────────────────────────

  1. Per-peer encryption scope (NEWT-945): The enclave partial_dh() already computes partials, but today they're returned as plaintext
    PartialDecryptionData — the gateway (relay) can read them. NEWT-945's core addition is encrypting each partial to each peer enclave's X25519
    pubkey before returning. Two approaches:
    - (A) Encrypt inside the enclave (partial_dh returns Vec keyed per-peer), gateway relays opaque blobs
    - (B) Encrypt on the operator host after receiving plaintext partials from enclave (weaker — host sees plaintext partials briefly)

I'd go with (A) since the whole point is the gateway never sees usable partials. The enclave receives peer_enclave_pubkeys as input to
partial_dh() and encrypts each partial to each peer. Does that match your intent?
2. HPKE vs X25519+ChaCha20Poly1305 for per-peer encryption: The peer keys are X25519. We could use the same HPKE construction
(X25519+HKDF-SHA256+ChaCha20Poly1305) already in the codebase for SecureEnvelopes, or a simpler X25519 Diffie-Hellman + symmetric encrypt.
I'd default to reusing the existing HPKE construction since it's already compiled into the enclave. Agree?
3. Operator Commit receiving encrypted partials (NEWT-948): During Commit, each operator enclave needs to decrypt peer partials using its own
ephemeral private key, then combine with its own partial and Lagrange-interpolate. The ThresholdEvalInput struct already exists but carries
plaintext partials. Should we add a new field encrypted_peer_partials: Option<Vec> alongside the existing fields, or
replace the plaintext partial fields entirely?

❯ 1. A 2. agree 3. replace the plaintext partial fields entirely if the plaintest partial fields have no usage/need

Searched for 1 pattern (ctrl+o to expand)

⏺ The plaintext partial fields (ephemeral_partials, identity_partials, confidential_partials) are used extensively in decrypt.rs and
builder.rs. They can't be removed outright — they're the core of the existing threshold path. In the TEE model, we need to replace their
contents: instead of plaintext partials arriving from the gateway, the enclave receives encrypted-per-peer blobs, decrypts them internally,
and then feeds the decrypted partials into the same decrypt_envelopes() logic.

So the approach is: add encrypted_peer_partials as the new input, and the enclave internally decrypts them into the existing
ephemeral_partials/identity_partials/confidential_partials format before calling decrypt_envelopes(). The existing downstream code stays
untouched.

Batch 2: Consensus & Testing

  1. Backward compatibility during rollout: Should the operator support both modes simultaneously (plaintext partials for non-TEE operators,
    encrypted partials for TEE operators)? Or is this a hard cutover where all operators must be TEE-enabled? Per our earlier design decision
    ("all operators MUST run TEE, no fallback"), I'd treat this as a hard cutover — ThresholdEvalInput gets encrypted_peer_partials and the
    plaintext fields become internal-only (populated by the enclave after decryption). Does that match expectations?
  2. E2E testing (NEWT-951): The existing just e2e-tee uses LoopbackEnclave (in-process, no real VSOCK). For threshold mode E2E, we need 2+
    operators each with their own LoopbackEnclave instance, each with distinct FROST key shares and enclave keypairs. Should we extend the
    existing e2e-tee recipe or create a new just e2e-tee-threshold recipe? I'd default to a new recipe since it needs a different operator count
    and DKG setup.

❯ 1. we evaluate rego inside TEE ONLY for privacy tasks right? but yes, the invariant that private data partials to never be in decrypted form
outside of TEE holds strong. 2. is there a way to have just one e2e-tee with a param to choose between shared hpke mode (default) vs
threshold through well designed code refactoring? If this causes complexity cost that is bigger than the benefit, then just have two e2e
test recipes for each mode

⏺ Right — non-privacy tasks skip the enclave entirely. The TEE path is only for tasks with encrypted envelopes (identity, confidential,
ephemeral). The invariant is: decrypted private data partials never exist outside the enclave, but the enclave itself is only invoked when
there's private data to decrypt.

On the E2E question — let me check how the existing threshold E2E works to see if parameterization is feasible.

Searched for 2 patterns (ctrl+o to expand)

⏺ The two recipes have very different setups — e2e-tee runs 1 operator with LoopbackEnclave, e2e-threshold runs 2 operators with FROST DKG.
Parameterizing into one recipe would mean threading --tee true through e2e-threshold, which is doable since the only delta is:

  • ENCLAVE_MODE=loopback env var
  • Different test binary/filter (threshold tests expect 2 operators)

But the complexity cost is low either way. I'd go with parameterizing e2e-tee with --threshold true — one recipe, two modes. The threshold
mode adds a second operator + FROST DKG setup. Simple if branch in the recipe.

I have enough to summarize requirements now.

Understood Requirements

Goal: Complete Phase 1b threshold mode TEE — encrypted partial DH exchange between enclaves so the gateway never sees usable partials.

Scope:

  • NEWT-945: Enclave partial_dh() encrypts each partial to each peer enclave's X25519 pubkey using HPKE (existing
    X25519+HKDF-SHA256+ChaCha20Poly1305). Returns Vec keyed per-peer. New EncryptedPartialDH type with
    (recipient_operator_id, ciphertext).
  • NEWT-946: Operator Prepare handler forwards enc_points + peer_enclave_pubkeys to enclave via EnclavePartialDhRequest. Enclave returns
    encrypted partials. Operator returns them to gateway as opaque blobs.
  • NEWT-948: Operator Commit handler passes encrypted peer partials to enclave. Enclave decrypts using its own ephemeral private key, combines
    with own partial, Lagrange-interpolates, HPKE-decrypts all envelopes, runs Rego eval. ThresholdEvalInput gets encrypted_peer_partials field;
    existing plaintext partial fields become internal-only (populated by enclave after decryption).
  • NEWT-951: E2E test via just e2e-tee --threshold true. 2 operators with LoopbackEnclave, distinct FROST key shares and enclave keypairs.
    Verifies: gateway log contains no readable partials, each enclave independently decrypts and evaluates.

Out of scope:

  • WASM-in-enclave (Phase 2)
  • Non-privacy tasks (no enclave involvement)
  • Real VSOCK testing (LoopbackEnclave only for E2E)

Key decisions:

  • Encryption inside enclave only (option A) — host/gateway never sees plaintext partials
  • Reuse existing HPKE construction for per-peer encryption
  • Hard cutover: all operators must be TEE-enabled for threshold privacy tasks (no mixed mode)
  • Single e2e-tee recipe with --threshold param

Constraints:

  • lessons.md "Positional alignment across boundaries must be driven by the authoritative order source" — when transposing encrypted partials
    per-recipient, drive from on-chain operator array order
  • lessons.md "Best-effort dest-chain writes need transient-only retry" — no new multi-chain writes here (already handled by NEWT-950)
  • Enclave ephemeral private key never leaves the enclave (forward secrecy property)

Testing approach: just e2e-tee --threshold true with 2 LoopbackEnclave operators, FROST 2-of-2 DKG, privacy task through full Prepare+Commit
cycle

Risks: HPKE in enclave already compiles (verified in Phase 1a), but per-peer encryption multiplies the crypto work per Prepare phase by N-1
operators. For small operator sets (2-5) this is negligible.

Confirm and I'll start implementation.

@denniswon

Copy link
Copy Markdown
Author

⏺ Phase 1b is complete. Here's the full TEE project status:

┌──────────────────────────────────────────────────┬──────────────────────────────┐
│ Phase │ Status │
├──────────────────────────────────────────────────┼──────────────────────────────┤
│ Pre-TEE (envelope cache, secrets key separation) │ COMPLETE │
├──────────────────────────────────────────────────┼──────────────────────────────┤
│ 1a (Operator Enclave MVP) │ COMPLETE │
├──────────────────────────────────────────────────┼──────────────────────────────┤
│ 1a.5 (Challenger TEE Attestation) │ COMPLETE │
├──────────────────────────────────────────────────┼──────────────────────────────┤
│ 1b (Threshold Mode TEE) │ COMPLETE (PR #594 in review) │
├──────────────────────────────────────────────────┼──────────────────────────────┤
│ 2 (WASM in Enclave) │ Backlog │
├──────────────────────────────────────────────────┼──────────────────────────────┤
│ 3 (On-Chain Registry Governance) │ Partially done │
└──────────────────────────────────────────────────┴──────────────────────────────┘

Remaining TEE work:

Phase 2: WASM Execution in Enclave (NEWT-952)

  • Move WASM data provider execution inside the enclave so policy client secrets (API keys, endpoints) are also hardware-protected
  • Requires an Egress Service: second VSOCK port (VSOCK_PORT_EGRESS = 5006) on the host to proxy http.fetch calls from the enclave to the
    external network (enclaves have no network access)
  • Bigger scope than Phase 1 — WASM runtime + Egress Service adds significant complexity
  • Currently backlog, no target date

Phase 3: On-Chain Enclave Registry (NEWT-954)

  • EnclaveVersionRegistry contract is already deployed and functional (PCR0 whitelist + enclave key storage)
  • NEWT-954 is about making the registry governance-controlled — decentralized version management where the operator community (not just task
    generators) can propose/vote on enclave version activations
  • Low priority, backlog

Other TEE-adjacent work:

  • Wesley's deploy PR (#581) — switching operator Docker images to Amazon Linux for Nitro compatibility (reviewed, awaiting his fixes)
  • Production deployment: once PR #594 merges, stagef deployment + load testing with real operators running Nitro Enclaves

The TEE integration project is effectively at a ship-ready milestone for Phase 1. All the crypto and plumbing is done. What's left is
production hardening (Phase 2 WASM, Phase 3 governance) and deployment.

Want to continue on a different project, or pick up Phase 2?

Good — the WASM executor has a fetch() host function that uses reqwest to make outbound HTTP calls. This is what needs to be proxied through
VSOCK in the enclave.

Now let me check how secrets are handled.

Read 1 file (ctrl+o to expand)

⏺ Now I have enough context on the current architecture. Let me start the grill.


Question 1: What exactly moves inside the enclave?

Today the data flow during Prepare is:

Operator host: WASM fetch (reqwest HTTP) → policyTaskData

VSOCK → Enclave: decrypt envelopes → Rego eval → bool result

There are three layers that could move:

  • (A) Just secrets decryption: WASM stays on host but secrets are decrypted inside the enclave and injected into WASM args (secrets never
    touch host memory)
  • (B) WASM execution + secrets: The entire WASM runtime moves inside the enclave. HTTP calls proxy through VSOCK Egress. Secrets are
    decrypted and used entirely inside the enclave.
  • (C) Everything: WASM + secrets + IPFS policy fetch. The enclave fetches its own policy code.

My recommendation: Option B. Option A doesn't protect secrets during WASM execution (the host still sees decrypted API keys in the WASM
sandbox). Option C adds IPFS networking complexity that doesn't improve security (policy code isn't secret). Option B protects all secret
material while keeping IPFS/policy fetching on the host (non-sensitive).

What's your view?

✻ Worked for 1m 13s

※ recap: Phase 1b threshold TEE is complete — all E2E tests pass, PR #594 in review. We started planning Phase 2 (WASM execution inside
enclave) with a design grill session. Next: answer Question 1 about what exactly moves into the enclave.

❯ B

⏺ Question 2: When does WASM execute — Prepare phase or Commit phase?

Today, WASM executes during the Prepare phase on the host. Each operator independently fetches policyTaskData by running the WASM data
provider (which calls external APIs like price feeds via http.fetch). The gateway then computes consensus on the results.

With WASM inside the enclave, we have a choice:

  • (A) Keep in Prepare: WASM runs inside the enclave during Prepare. The enclave needs the Egress Service available during Prepare. The
    operator returns policyTaskData from the enclave. This means the enclave now handles BOTH Prepare (WASM + secrets) AND Commit (decrypt +
    Rego eval).
  • (B) Move to Commit: WASM runs inside the enclave during Commit alongside Rego eval. Prepare stays lightweight (no WASM). But this changes
    the consensus model — operators can't independently produce policyTaskData before consensus, so the gateway would need a different data
    source.

My recommendation: Option A. Moving WASM to Commit would break the Two-Phase Consensus model where Prepare produces independent
policyTaskData for median normalization. The enclave already handles Commit (decrypt + eval); adding Prepare WASM is a natural extension.
The Egress Service is needed regardless.

What's your view?

✻ Worked for 32s

❯ A

⏺ Question 3: How does the Egress Service proxy HTTP calls from the enclave?

The enclave has no network stack — reqwest won't work inside it. The WASM fetch() host function needs to proxy HTTP requests through the
operator parent via VSOCK. Three approaches:

  • (A) Raw VSOCK request/response framing: The enclave sends a serialized HttpRequest (method, URL, headers, body) over a second VSOCK port
    (VSOCK_PORT_EGRESS = 5006). The operator parent receives it, executes via reqwest, and sends back the HttpResponse. Simple framing, one
    request per connection (same as the compute port pattern).
  • (B) HTTP CONNECT tunnel: The enclave runs a minimal HTTP client that speaks to a SOCKS/CONNECT proxy on the parent. More standard, allows
    the enclave to use reqwest with a proxy configuration. But adds proxy protocol complexity.
  • (C) vsock-to-TCP bridge: A generic vsock-to-TCP forwarder on the parent that maps enclave VSOCK connections to TCP connections. The
    enclave runs a real HTTP stack over VSOCK as if it were TCP. Most transparent but requires significant plumbing (TLS termination, DNS
    resolution on parent side).

My recommendation: Option A. It mirrors the existing compute VSOCK pattern (framed bincode messages over VSOCK_PORT_COMPUTE = 5005). The
HttpRequest/HttpResponse types already exist in the WASM WIT bindings (executor.rs:50). No new protocol layers, no proxy complexity. The
WASM fetch() implementation inside the enclave just serializes the HttpRequest, sends it over VSOCK, and deserializes the HttpResponse —
same as the existing host function but with VSOCK as the transport instead of direct reqwest.

What's your view?

✻ Baked for 40s

❯ if we were to optimize on security and performance, what is most optimal approach?

⏺ Good question. Let me think about this from both angles.

Security analysis:

  • Option A (framed request/response): The enclave explicitly constructs each HttpRequest — the parent sees the full URL, headers, and body
    in plaintext. A compromised host can read API keys embedded in request headers/URLs, modify requests, or forge responses. The enclave has no
    way to verify response authenticity.
  • Option B (CONNECT proxy): Same security as A — the parent terminates TLS, so it sees plaintext requests.
  • Option C (vsock-to-TCP bridge): The enclave initiates its own TLS handshake end-to-end to the external API. The parent only sees encrypted
    TCP bytes — it cannot read API keys, modify requests, or forge responses. This is the only option where secrets remain protected in
    transit.

Performance analysis:

  • Option A: Lowest latency — one VSOCK round-trip per HTTP request, bincode serialization (microseconds). No TLS overhead on the VSOCK
    channel.
  • Option B: Similar to A plus proxy protocol overhead.
  • Option C: Highest latency — TLS handshake from enclave to external API (the enclave needs a TLS library), plus vsock-to-TCP forwarding
    adds buffering. But TLS handshakes are already happening today on the host — it's the same total latency, just originating from a different
    location.

The security-optimal answer is Option C — but it requires the enclave binary to include a TLS stack (rustls) and DNS resolution. The enclave
binary size grows, and the parent needs a vsock-to-TCP forwarder.

The pragmatic-optimal answer is still Option A — because the parent is already trusted to proxy VSOCK compute traffic, and in the current
threat model the parent is a semi-trusted component (it handles BLS signing). The incremental security gain of C is that a compromised
parent can't read API keys in transit — but a compromised parent can already refuse to forward requests, return garbage, or simply not run
the enclave.

@denniswon

denniswon commented Apr 28, 2026

Copy link
Copy Markdown
Author

My revised recommendation: Start with Option A for Phase 2 MVP (ships fast, matches existing VSOCK pattern, performance-optimal). Add Option
C as Phase 2.5 when the threat model requires end-to-end TLS from enclave to external APIs (e.g., when handling highly sensitive financial
API keys where even the parent shouldn't see them).

The key insight: Option A protects secrets at rest (encrypted in DB, decrypted only inside enclave). Option C additionally protects secrets
in transit through the parent. Whether transit protection is worth the complexity depends on how much you trust the operator's own host.

Which approach do you want?

✻ Brewed for 28s

❯ yes option A, but let's document about option C improvements in our docs @docs/ENCLAVE.md and also make sure we track in Linear as well
⎿  Read docs/ENCLAVE.md (669 lines)

⏺ Agreed. Let me continue the grill first, then we'll document and create the Linear ticket at the end.

❯ yes option A, but let's document about option C improvements in our docs @docs/ENCLAVE.md and also make sure we track in Linear as well
⎿  Read docs/ENCLAVE.md (669 lines)

⏺ Agreed. Let me continue the grill first, then we'll document and create the Linear ticket at the end.

Question 4: How do secrets get into the enclave for WASM execution?

Today, WASM secrets (API keys, endpoint URLs) are encrypted per-operator via newt_storeEncryptedSecrets and stored in the DB. During
Prepare, the operator host fetches from DB, HPKE-decrypts with its individual key, and injects into the WASM sandbox args.

With WASM inside the enclave, the enclave needs the secrets. Three approaches:

  • (A) Pass encrypted secrets via VSOCK, decrypt inside enclave: The operator host fetches the encrypted SecureEnvelope from DB (it can't
    decrypt — it only has the HPKE key inside the enclave). It passes the raw encrypted blob via the VSOCK EnclaveEvalRequest. The enclave
    decrypts with its HPKE key and injects into WASM args. Secrets never touch host memory in plaintext.
  • (B) Pre-decrypt on host, pass plaintext via VSOCK: Same as today — host decrypts, sends plaintext to enclave. Simpler but secrets are in
    host memory briefly. Defeats the purpose of Phase 2.
  • (C) Enclave fetches from DB directly: Enclave connects to PostgreSQL. Requires the Egress Service for DB connections too. Adds DB driver
    to enclave binary — increases TCB significantly.

My recommendation: Option A. The operator host already fetches encrypted blobs from DB (it has DB access). It passes them as opaque bytes
over VSOCK. The enclave decrypts inside its isolation boundary. This is the natural extension of the existing SecureEnvelope pattern — same
as how identity/confidential envelopes are already passed. The EnclaveEvalRequest already carries encrypted envelopes; adding encrypted
secrets is one more field.

The key insight: secrets are encrypted to the individual operator HPKE key (via newt_getSecretsPublicKey), not the threshold MPK. So each
enclave can decrypt its own secrets independently — no threshold partials needed.

What's your view?

✻ Cooked for 24s

❯ agreed option A

⏺ Question 5: What about the WASM binary itself — does it need to be inside the enclave?

Today the WASM component is fetched from IPFS, compiled via wasmtime, cached in-memory (wasm_component_cache), and executed on the host. The
WASM binary is policy logic (not secret) — it's content-addressed via CID and integrity-verified.

Three options for where WASM compilation and caching live:

  • (A) Host compiles, sends compiled component via VSOCK: Host fetches WASM from IPFS, compiles with wasmtime, serializes the compiled
    module, sends over VSOCK. Enclave deserializes and runs. Problem: serialized wasmtime modules are platform-specific and large (~MBs). Also,
    a malicious host could send a modified WASM binary.
  • (B) Host sends raw WASM bytes, enclave compiles and caches: Host fetches raw WASM from IPFS (untrusted data), verifies CID integrity,
    sends bytes over VSOCK. Enclave compiles with wasmtime and caches in-memory. Enclave can independently verify keccak256(wasm_bytes) ==
    policyCodeHash from on-chain. Problem: wasmtime compiler adds ~20MB to enclave binary and compilation takes 400-500ms (from logs: wasm
    component compiled and cached compile_ms=507).
  • (C) Host sends raw WASM bytes, enclave compiles but doesn't cache: Same as B but no caching. Each Prepare request re-compiles. Too slow
    (500ms penalty per task).

My recommendation: Option B. The enclave must verify WASM integrity independently (defense against malicious host injecting modified WASM).
CID + policyCodeHash checks are already in the codebase. The 500ms compile cost is a one-time hit per unique WASM binary — the in-memory
cache amortizes it across tasks. wasmtime adds binary size but is already validated for the RISC-V SP1 target, so it should compile for the
enclave's x86 target.

The alternative (option A) trusts the host to provide correct compiled code — which violates the TEE security model where the host is
semi-trusted.

What's your view?

✻ Baked for 28s

※ recap: We're designing Phase 2 TEE (WASM execution inside enclave). I was grilling you on design decisions — answered Q1-Q4 (all option A/B
agreed). Q5 is pending: should the enclave compile WASM from raw bytes (my recommendation) or trust pre-compiled modules from the host?

❯ B

⏺ Question 6: How does the enclave binary size and dependency footprint change?

Today the enclave binary is minimal: Regorus + HPKE + domain types. No Tokio server, no network libs, no DB drivers. This keeps the Trusted
Computing Base (TCB) small and auditable.

Phase 2 adds:

  • wasmtime (~20MB compiled): WASM component runtime
  • reqwest (or minimal HTTP client): for the Egress Service client side inside the enclave — wait, actually with Option A (framed VSOCK), we
    DON'T need reqwest inside the enclave. The enclave sends HttpRequest structs over VSOCK and the host runs reqwest.

So the enclave gains:

  • wasmtime + wasmtime-wasi — the WASM runtime (significant)
  • The WASM WIT binding types (HttpRequest, HttpResponse, secrets accessor)
  • VSOCK client for VSOCK_PORT_EGRESS (to send HTTP proxy requests to the parent)

It does NOT gain:

  • reqwest (stays on host)
  • sqlx / database drivers (stays on host)
  • IPFS client (stays on host)

My recommendation: Accept the wasmtime dependency increase. It's the core of the feature. The PCR0 changes (new binary measurement), so the
EnclaveVersionRegistry needs a version activation for the Phase 2 enclave image. This is expected — same as any enclave binary update.

One concern: memory pressure. Nitro Enclaves have limited memory (configurable, default ~512MB). wasmtime compilation + WASM execution +
Rego evaluation + HPKE decryption all compete for memory. The enclave config already has memory_mib (default from config). Do we need to
increase the default for Phase 2?

Searched for 1 pattern, read 1 file (ctrl+o to expand)
⎿  PreToolUse:Read hook error ⎿  Failed with non-blocking status code: [TokenEfficiency] Hard rule
violation: Re-reading config.rs (2x) — file unchanged since last
read. Consider using cached knowledge.

⏺ 4096 MiB (4 GB) default. That's generous — wasmtime compiles a 144KB WASM binary in ~500ms using maybe 50-100MB peak. With Rego eval + HPKE

  • threshold decryption, total peak is probably under 500MB. 4GB is plenty.

No config change needed. Moving on.

Question 7: What's the new VSOCK message flow for Prepare phase with WASM?

Today's Prepare flow:
Host: IPFS fetch WASM → compile → execute (reqwest HTTP) → policyTaskData
Host: compute enc_points from envelopes
Host → Enclave: PartialDh(enc_points, peer_pubkeys)
Enclave → Host: EncryptedPartialDH blobs
Host → Gateway: policyTaskData + encrypted_partials

Phase 2 Prepare flow — two options:

(A) Single new message type PrepareEval: The host sends WASM bytes + encrypted secrets + WASM args + enc_points + peer_pubkeys in one
request. The enclave compiles WASM, decrypts secrets, executes WASM (proxying HTTP via Egress), produces policyTaskData, then computes
partial DHs. Returns policyTaskData + encrypted_partials in one response. One VSOCK round-trip.

(B) Two messages: First WasmExec (WASM bytes + secrets + args → policyTaskData via Egress), then PartialDh (enc_points + peer_pubkeys →
encrypted_partials). Two VSOCK round-trips but simpler — each message does one thing.

My recommendation: Option A. The host doesn't need intermediate policyTaskData access (it flows straight back to the gateway). Combining
into one round-trip saves ~100-200μs connect overhead and simplifies the operator Prepare handler. The enclave already handles multi-step
logic in evaluate() (decrypt → domain → Rego). Adding WASM exec at the front is the same pattern.

What's your view?

✻ Baked for 55s

※ recap: We're designing Phase 2 TEE (WASM execution inside enclave). Grilling through design decisions — decided on framed VSOCK egress,
encrypted secrets via VSOCK, WASM compiled inside enclave. Currently on question 7: whether Prepare phase uses one combined VSOCK message or
two separate ones. Awaiting your answer on that.

❯ A

@denniswon

Copy link
Copy Markdown
Author

⏺ Question 8: Does WASM-in-enclave apply to ALL tasks or only privacy tasks?

Today, WASM execution runs for every task that has a data provider plugin — privacy or not. The question is whether Phase 2 moves ALL WASM
execution into the enclave, or only WASM execution for tasks that have secrets.

  • (A) Only tasks with secrets: If the policy client has stored encrypted secrets via newt_storeEncryptedSecrets, WASM runs inside the
    enclave. Otherwise WASM stays on the host. This minimizes enclave load — most tasks don't have secrets.
  • (B) All tasks with WASM: Every WASM data provider runs inside the enclave regardless of whether secrets are configured. Simpler routing
    logic (no conditional), and it means all policyTaskData is enclave-attested. But it adds Egress Service overhead to every task.
  • (C) All privacy tasks: WASM runs in enclave if the task involves any privacy data (identity, confidential, ephemeral). Non-privacy tasks
    with WASM but no secrets stay on host.

My recommendation: Option A (only tasks with secrets). The motivation for Phase 2 is protecting secrets (API keys, endpoint configs).
Non-secret WASM execution (e.g., a price feed plugin with no API key) doesn't benefit from enclave isolation. Option A keeps the hot path
fast — the majority of tasks with public data providers run on the host with zero Egress overhead. The operator builder already knows
whether secrets are configured (it fetches them from DB during Prepare).

The routing decision: if secrets_envelope.is_some() { enclave_wasm } else { host_wasm }.

What's your view?
Non-secret WASM execution (e.g., a price feed plugin with no API key) doesn't benefit from enclave isolation. Option A keeps the hot path
fast — the majority of tasks with public data providers run on the host with zero Egress overhead. The operator builder already knows
whether secrets are configured (it fetches them from DB during Prepare).

The routing decision: if secrets_envelope.is_some() { enclave_wasm } else { host_wasm }.

What's your view?

✻ Churned for 35s

❯ A

⏺ Question 9: How does the Egress Service on the host handle concurrent HTTP requests?

The WASM fetch() host function can make multiple sequential HTTP calls during a single WASM execution (e.g., fetch price from Pyth, then
fetch from Chainlink as fallback). The Egress Service on VSOCK_PORT_EGRESS needs to handle these.

  • (A) Per-connection model (same as compute port): One VSOCK connection per HTTP request. The enclave connects to port 5006, sends an
    HttpRequest, gets an HttpResponse, closes. Simple, matches existing pattern. Downside: connection setup overhead per HTTP call (~50-200μs).
  • (B) Multiplexed connection: The enclave opens one VSOCK connection to port 5006 at WASM execution start, sends multiple
    HttpRequest/HttpResponse pairs over it, closes when WASM execution completes. Lower overhead for multiple sequential requests.
  • (C) Shared long-lived connection pool: Persistent VSOCK connections between enclave and Egress Service, shared across WASM executions.
    Most efficient but adds connection management complexity.

My recommendation: Option A. WASM plugins typically make 1-2 HTTP calls per execution. The 50-200μs overhead per call is negligible vs the
200-1000ms external API latency. The per-connection model provides clean failure isolation (one stuck request can't block another) and
matches the compute port pattern. It also means the Egress Service is stateless — no session tracking, no connection lifecycle management.

If profiling later shows connection overhead is a bottleneck (unlikely), upgrade to B.

What's your view?

✻ Cogitated for 39s

※ recap: We're designing Phase 2 TEE (WASM execution inside enclave). We've resolved 9 design questions through a grill session — WASM +
secrets move into enclave, Egress Service proxies HTTP via framed VSOCK. Next: answer the remaining design questions and document decisions.

❯ for performance and reliability, which option is the optimal one?

⏺ Option B is optimal for both performance and reliability. Here's why:

Performance:

  • WASM plugins that make 2-3 sequential API calls (common pattern: price feed + fallback + validation) pay 50-200μs × 3 = 150-600μs overhead
    with Option A. With Option B, it's 50-200μs × 1 = one connection for the whole WASM execution.
  • More importantly, VSOCK connection setup involves kernel context switches between the enclave VM and parent VM. Under load (10+ concurrent
    tasks), this adds up.

Reliability:

  • Option A: if the Egress listener is temporarily overloaded, each HTTP request independently retries the VSOCK connect. 3 requests = 3
    chances to hit a connect timeout.
  • Option B: one connection at WASM start. If it connects, all HTTP requests flow through. If it fails, you get one clean error before WASM
    starts — no partial execution where the first fetch succeeds but the second fails to connect.

Why not Option C:

  • Long-lived connections across WASM executions mean connection state outlives request scope. A stuck connection from a previous task
    poisons the next task. The compute port already proved that per-request connections (Option A pattern) are reliable — Option B is the
    natural middle ground: per-WASM-execution, not per-HTTP-request.

Revised recommendation: Option B. One VSOCK connection opened when WASM execution starts, multiple HttpRequest/HttpResponse frames flow
through it, closed when WASM completes. Clean failure isolation per WASM execution, minimal overhead.

What's your view?

✻ Churned for 21s

❯ agreed. option B

⏺ Question 10: What security controls should the Egress Service enforce?

The Egress Service on the host runs reqwest on behalf of the enclave. A malicious or compromised WASM plugin could try to abuse this to
exfiltrate data or attack internal services. What restrictions should the Egress proxy enforce?

  • (A) No restrictions: The enclave can make any HTTP request to any URL. Simple but dangerous — a malicious WASM plugin could call
    http://169.254.169.254/latest/meta-data/ (AWS instance metadata) or scan internal networks.
  • (B) Domain allowlist: Only requests to pre-configured domains are proxied. The policy client specifies allowed domains at registration
    time (e.g., api.pyth.network, hermes.pyth.network). Any request to a non-whitelisted domain is rejected by the Egress Service.
  • (C) URL pattern allowlist + rate limiting: Domain allowlist (from B) plus rate limiting per WASM execution (max N requests, max M bytes
    response) to prevent resource exhaustion. Block private IP ranges (10.0.0.0/8, 172.16.0.0/12, 169.254.0.0/16, localhost) unconditionally.

My recommendation: Option C. The private IP blocking is essential (prevents SSRF against AWS metadata and internal services). Rate limiting
prevents a buggy or malicious WASM plugin from DDoSing external APIs through the operator. The domain allowlist can start as a deny-list
(block private ranges) and evolve into a per-policy-client allowlist in a future iteration.

MVP implementation: block private IP ranges + block AWS metadata endpoint + configurable max requests per WASM execution (default 10) +
configurable max response body size (default 1MB).

What's your view?

✻ Cogitated for 21s

❯ C

⏺ I've covered the major decision branches. Let me summarize the design and check if there are remaining questions.

Phase 2 Design Summary

┌───────────────────────┬────────────────────────────────────────────────┬─────────────────────────────────────────────────────────────┐
│ Decision │ Choice │ Rationale │
├───────────────────────┼────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ What moves inside │ WASM execution + secrets (Option B) │ Protects secrets at rest; IPFS/policy fetch stays on host │
├───────────────────────┼────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ When does WASM │ Prepare phase (Option A) │ Preserves Two-Phase Consensus model │
│ execute │ │ │
├───────────────────────┼────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ Egress Service │ Framed VSOCK request/response (Option A) │ Matches existing compute port pattern; Option C (end-to-end │
│ transport │ │ TLS) deferred to Phase 2.5 │
├───────────────────────┼────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ Secrets delivery │ Encrypted via VSOCK, decrypt inside enclave │ Secrets never in host memory │
│ │ (Option A) │ │
├───────────────────────┼────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ WASM binary handling │ Host sends raw bytes, enclave compiles + │ Enclave independently verifies CID/policyCodeHash integrity │
│ │ caches (Option B) │ │
├───────────────────────┼────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────┤
├───────────────────────┼────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ VSOCK message flow │ Single PrepareEval message (Option A) │ One round-trip for WASM + partials │
├───────────────────────┼────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ Egress connection │ Per-WASM-execution (Option B) │ Clean failure isolation, minimal overhead │
│ model │ │ │
├───────────────────────┼────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ Egress security │ Private IP blocking + rate limiting + domain │ Prevents SSRF, resource exhaustion │
│ │ deny-list (Option C) │ │
└───────────────────────┴────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────┘

Two remaining questions:

Question 11: Does the LoopbackEnclave need to simulate the Egress Service for testing?

In the LoopbackEnclave (in-process, no VSOCK), the WASM fetch() host function currently calls reqwest directly. For Phase 2, the production
path would be VSOCK→Egress→reqwest. The LoopbackEnclave needs to either:

  • (A) Keep direct reqwest in LoopbackEnclave: WASM fetch() in loopback mode calls reqwest directly (no Egress simulation). Tests exercise
    the WASM compilation + secrets decryption + Rego eval path but skip the Egress proxy path.
  • (B) Simulate Egress in LoopbackEnclave: LoopbackEnclave spawns a local Egress Service thread that the in-process WASM connects to via
    localhost TCP. Full path testing but more complex test setup.

My recommendation: Option A. The Egress Service is a simple proxy — its correctness is testable independently (unit test for framing +
security controls). The LoopbackEnclave's value is testing the crypto + WASM + Rego pipeline, not network proxying. Separate integration
test for the Egress Service.

What's your view?

✻ Crunched for 28s

※ recap: We're grilling Phase 2 TEE design (WASM execution inside enclave). All major decisions are resolved. Last question: should
LoopbackEnclave simulate the Egress Service for testing, or keep direct reqwest. I recommend direct reqwest (option A) for simplicity.

❯ what are pros vs cons?

⏺ Option A: Direct reqwest in LoopbackEnclave

┌───────────────────────────────────────────┬───────────────────────────────────────────────────────────────────────────────────────────┐
│ Pros │ Cons │
├───────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ Simple — no test infra changes, existing │ Egress security controls (IP blocking, rate limiting) never tested in E2E │
│ LoopbackEnclave pattern unchanged │ │
├───────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ │ Code path divergence: loopback uses reqwest directly, production uses │
│ Fast — no proxy overhead in tests │ VSOCK→Egress→reqwest. A bug in the Egress framing/serialization could pass loopback but │
│ │ fail production │
├───────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ Egress Service testable independently via │ If WASM fetch() has different behavior when going through the proxy (e.g., timeout │
│ unit tests │ handling, response size limits), E2E tests won't catch it │
└───────────────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────┘

Option B: Simulate Egress in LoopbackEnclave

┌───────────────────────────────────────────────────────────────────────────┬───────────────────────────────────────────────────────────┐
│ Pros │ Cons │
├───────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────┤
│ Full pipeline tested end-to-end — same code path as production │ More complex test setup — need a local TCP listener │
│ │ simulating the Egress Service │
├───────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────┤
│ Egress security controls (IP blocking, rate limiting) exercised in every │ Slower test setup (~50ms for spawning listener) │
│ E2E run │ │
├───────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────┤
│ Catches serialization bugs in HttpRequest/HttpResponse framing │ LoopbackEnclave becomes less "loop back" and more "local │
│ │ integration" — blurs the line │
├───────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────┤
│ Production and test code paths are identical — no conditional if loopback │ Requires the WASM executor to be parameterized by │
│ { reqwest } else { vsock_egress } │ transport (reqwest vs proxy client) │
└───────────────────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────┘

The real risk with Option A: the WASM fetch() implementation inside the enclave would need an if loopback { reqwest::Client } else {
EgressClient } branch. Every conditional path that only runs in production is a bug waiting to happen. The Phase 1 LoopbackEnclave works
because EnclaveState::evaluate() is identical in both modes — no conditional branching.

Revised recommendation: Option B. The LoopbackEnclave should simulate the Egress path with a local TCP proxy. This keeps the WASM fetch()
implementation unconditional — it always goes through the EgressClient. In loopback mode, the EgressClient connects to a localhost TCP
listener instead of VSOCK. In production, it connects to VSOCK port 5006. The transport is parameterized, not the fetch logic.

This mirrors how EnclaveTransport is already parameterized: LoopbackEnclave (in-process) vs VsockEnclave (VSOCK). The Egress Service gets
the same treatment: LoopbackEgress (localhost TCP) vs VsockEgress (VSOCK port 5006).

What's your view?

✻ Cogitated for 24s

❯ Option B yes

⏺ Question 12 (final): What's the incremental delivery plan?

Phase 2 is bigger than Phase 1 — it touches the enclave binary (wasmtime), the WASM executor, the Egress Service (new component), the
operator Prepare handler, and the VSOCK protocol. Shipping it all at once is risky. How should we slice it?

  • (A) One big PR: Everything in one branch. Fast to develop, hard to review, high merge risk.
  • (B) Bottom-up vertical slices:
    a. Egress Service — new VSOCK_PORT_EGRESS listener on the operator host, framed HttpRequest/HttpResponse protocol, security controls (IP
    blocking, rate limiting). Standalone, testable independently. No enclave changes.
    b. Enclave WASM runtime — add wasmtime to enclave binary, EgressClient transport (parameterized: loopback TCP vs VSOCK), WASM compilation
  • caching inside enclave. LoopbackEgress for testing. No secrets yet.
    c. Secrets in enclave — encrypted secrets delivered via VSOCK, decrypted inside enclave, injected into WASM args. Operator routing: if
    secrets { enclave_wasm } else { host_wasm }.
    d. PrepareEval message — unified single-message flow (WASM + partials in one request). Wire protocol version bump.
    e. E2E test — just e2e-tee --wasm true with a test WASM plugin that uses encrypted secrets.
  • (C) Feature-flag gated: Everything behind #[cfg(feature = "enclave-wasm")], ship incrementally but merge to main behind the flag.

My recommendation: Option B. Each slice is independently reviewable, testable, and mergeable. Slice 1 (Egress Service) can ship before any
enclave changes. Slice 2 proves WASM compiles and runs inside the enclave. Slice 3 adds the security-critical secrets path. Slice 4
optimizes the wire protocol. Slice 5 validates end-to-end.

This matches how Phase 1 was delivered: core crypto → operator plumbing → gateway wiring → E2E test.

What's your view?

✻ Worked for 24s

❯ B

⏺ All decision branches resolved. Here's the final summary.

Phase 2 TEE: WASM in Enclave — Agreed Design

┌─────┬────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ # │ Decision │ Choice │
├─────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 1 │ What moves inside │ WASM execution + secrets (not IPFS/policy fetch) │
├─────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 2 │ When WASM executes │ Prepare phase (preserves Two-Phase Consensus) │
├─────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 3 │ Egress transport │ Framed VSOCK HttpRequest/HttpResponse on port 5006 (Option C end-to-end TLS deferred to Phase 2.5) │
├─────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 4 │ Secrets delivery │ Encrypted SecureEnvelope via VSOCK, decrypt inside enclave │
├─────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 5 │ WASM binary │ Host sends raw bytes, enclave compiles + caches + verifies CID/policyCodeHash │
├─────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 6 │ Which tasks │ Only tasks with encrypted secrets (secrets_envelope.is_some()) │
├─────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 7 │ VSOCK message │ Single PrepareEval message (WASM + secrets + enc_points + peer_pubkeys → policyTaskData + │
│ │ │ encrypted_partials) │
├─────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 8 │ Egress connection │ Per-WASM-execution (one connection for all HTTP calls within one WASM run) │
│ │ model │ │
├─────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 9 │ Egress security │ Block private IPs + AWS metadata, rate limit per execution (10 requests, 1MB response), domain │
│ │ │ deny-list │
├─────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 10 │ LoopbackEnclave │ Simulate Egress via LoopbackEgress (localhost TCP). Parameterized transport like EnclaveTransport. │
│ │ testing │ │
├─────┼────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 11 │ Delivery plan │ 5 vertical slices: Egress Service → Enclave WASM runtime → Secrets in enclave → PrepareEval message → │
│ │ │ E2E test │
└─────┴────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Future improvement (Phase 2.5): End-to-end TLS from enclave to external APIs via vsock-to-TCP bridge. Protects secrets in transit through
the parent — currently only protected at rest.

Now let me document this in docs/ENCLAVE.md, add the Phase 2.5 improvement note, and create the Linear tickets. Let me do that now.

⏺ Skill(/save-progress)
⎿  Successfully loaded skill

⏺ Here's the progress preview:


Project: TEE Integration (Phase 2: WASM in Enclave)
Linear: NEWT-952

Summary: Completed design grill for Phase 2 TEE — WASM execution inside enclave with Egress Service. Resolved 12 architectural decisions
through structured questioning. Also completed Phase 1b E2E testing (all tests passing) and fixed several production-path issues (Prepare
quorum elevation, operator_id, on-chain key registration, test DB URLs).

Key Decisions (Phase 2 design):

  1. WASM + secrets move inside enclave; IPFS/policy fetch stays on host
  2. WASM runs in Prepare phase (preserves Two-Phase Consensus)
  3. Egress: framed VSOCK HttpRequest/HttpResponse on port 5006 (end-to-end TLS deferred to Phase 2.5)
  4. Secrets: encrypted SecureEnvelope via VSOCK, decrypted inside enclave
  5. WASM binary: host sends raw bytes, enclave compiles/caches/verifies CID+policyCodeHash
  6. Only tasks with encrypted secrets route WASM to enclave
  7. Single PrepareEval VSOCK message (WASM + secrets + partials in one round-trip)
  8. Per-WASM-execution Egress connection (one connection for all HTTP calls within one run)
  9. Egress security: block private IPs, rate limit (10 req, 1MB), domain deny-list
  10. LoopbackEgress simulates Egress via localhost TCP (parameterized transport, no conditional branching)
  11. Delivery: 5 vertical slices (Egress → WASM runtime → Secrets → PrepareEval → E2E test)

Phase 1b fixes this session:

  • Prepare quorum: max(bls_quorum%, threshold_min%) when threshold configured
  • EnclaveState.operator_id set from task_signer.operator_id() at build time
  • On-chain enclave key registration in E2E test (mirrors production)
  • test_database_url() shared helper across all 7 E2E operator configs
  • get_enclave_version_registry_address address helper
  • GetPublicKeyRpcResponse parsing fix (result.public_key, not bare result)
  • NewtonPolicy/NewtonPolicyData SemVerMixin fix (cherry-picked to main)
  • All E2E tests passing: e2e-tee, e2e-tee --threshold true, e2e --scenario privacy, e2e-gateway

Remaining TODOs:

  • Document Phase 2 design in ENCLAVE.md
  • Create Linear tickets for the 5 delivery slices
  • Create Linear ticket for Phase 2.5 (end-to-end TLS)
  • Get PR #594 reviewed and merged

Looks good?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment