Two-Phase Consensus Support for sendTask Async Flow

gistfile1.txt

Ready to code?

 Here is Claude's plan:
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
 Plan: Two-Phase Consensus Support for sendTask Async Flow

 Context

 The async sendTask path (channel.rs) currently uses a single-phase broadcast where each operator independently fetches data,
 evaluates, and BLS-signs. This diverges from the sync createTask path (sync.rs), which supports two consensus modes:

 1. Two-Phase (use_two_phase=true): Prepare → median consensus → Commit (ensures operators sign identical data)
 2. Centralized (use_two_phase=false): Gateway generates data centrally → operators sign

 The single-phase process_task broadcast in the async path is strictly worse than centralized — both have the gateway control the
 flow, but centralized guarantees data consistency. By aligning sendTask with createTask, we get consistent behavior across both RPC
 methods.

 All broadcast infrastructure already exists as methods on GatewayServiceImpl — the async path just needs to call them.

 Changes

 1. crates/gateway/src/rpc/types/mod.rs — Add fields to SendTaskRequest

 Add use_two_phase: Option<bool> and timeout: Option<u64> to SendTaskRequest (mirroring CreateTaskRequest). Both are Option for
 backward compatibility — omitting them preserves current behavior (centralized single-phase, config-level timeout).

 2. crates/gateway/src/rpc/api/channel.rs — Main changes

 2a. Capture policy_config
 - Line 233: change _policy_config → policy_config (needed by both broadcast functions)

 2b. Capture inline_envelopes from validation block
 - Lines 190-202: change fire-and-forget validation into the sync.rs pattern that returns Vec<SecureEnvelope> (needed for privacy
 threading through broadcast functions)

 2c. Fix aggregator_block computation
 - Current: uses source_chain_block unconditionally
 - Fix: use task_created_block in single-chain mode, source_chain_block in multichain (matching sync.rs lines 322-326)

 2d. Replace single-phase broadcast with two-phase/centralized branch

 Replace the block at lines ~395-488 (build_operator_async_rpc_request → broadcast → wait_for_aggregation) with:

 if use_two_phase {
     self.broadcast_two_phase_and_aggregate(cs, &task_request, ...)
 } else {
     self.broadcast_centralized_and_aggregate(cs, &task_request, ...)
 }

 Key implications:
 - Remove initialize_task(): Both broadcast functions handle aggregation internally via aggregate_responses() — they don't use the
 aggregator's task tracking (initialize_task / wait_for_aggregation). Remove the initialize_task call (lines 369-393) and the
 wait_for_aggregation call (line 415).
 - Add cancel_task in error paths: Both broadcast functions can fail. On error, the aggregator slot may or may not have been
 initialized internally. No cleanup needed since we removed initialize_task.
 - Add metrics: Record inc_gateway_task_total with mode label "two_phase" or "centralized".
 - Indexer DB write on failure: Write failed task to indexer DB (fire-and-forget spawn), same pattern as sync.rs.

 2e. Update WebSocket progress events
 - For two-phase: add phase-level progress (10% prepare start, 60% commit complete)
 - For centralized: keep existing percentages for backward compatibility
 - Include {"phase": "prepare"} / {"phase": "commit_complete"} in event payloads

 2f. Update imports
 - Add: SecureEnvelope, remove: build_operator_async_rpc_request (no longer used)

 Files modified (total: 2)

 ┌───────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────┐
 │                 File                  │                                      What changes                                       │
 ├───────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
 │ crates/gateway/src/rpc/types/mod.rs   │ Add use_two_phase and timeout fields to SendTaskRequest                                 │
 ├───────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
 │ crates/gateway/src/rpc/api/channel.rs │ Capture policy_config + inline_envelopes, fix aggregator_block, replace single-phase    │
 │                                       │ with two-phase/centralized branch, update WS events, update imports                     │
 └───────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────┘

 Files to reference (read-only)

 ┌──────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────────────┐
 │                 File                 │                                      What to mirror                                      │
 ├──────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
 │ crates/gateway/src/rpc/api/sync.rs   │ Two-phase/centralized branching (lines 355-550), broadcast_two_phase_and_aggregate (line │
 │                                      │  980), broadcast_centralized_and_aggregate (line 1353)                                   │
 ├──────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
 │ crates/gateway/src/rpc/api/common.rs │ Request builders (reused by broadcast functions internally)                              │
 └──────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────────────┘

 Verification

 1. cargo build --workspace — compile check
 2. cargo clippy --workspace -- -D warnings — lint (may fail on pre-existing SP1 issues)
 3. make gateway_e2e — existing E2E (centralized mode, verifies no regression)
 4. make gateway_e2e two_phase_consensus=true — two-phase E2E via createTask (existing test, confirms broadcast functions still work)
 5. Manual test: call newt_sendTask with "use_two_phase": true and verify WebSocket events show phase progression

 Risks

 - Backward compatibility: New fields are Option with defaults matching current behavior. No breaking changes.
 - process_task handler still needed: Operators still register the process_task RPC handler for the watcher path (on-chain event tasks
  processed by process_tasks background loop). We're only removing its usage from channel.rs gateway broadcast.