Created
March 14, 2026 14:38
-
-
Save denniswon/00cda380d0fc9393b469ea41ef2fbe03 to your computer and use it in GitHub Desktop.
Support flexible identity data schemas beyond KYC
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| https://linear.app/magiclabs/issue/NEWT-710/support-flexible-identity-data-schemas-beyond-kyc | |
| ⏺ I now have the complete data flow mapped. Let me summarize what I've found and present the implementation plan. | |
| ★ Insight ───────────────────────────────────── | |
| The full identity data flow has 6 touchpoints that need updating: | |
| 1. Gateway RPC (identity.rs types) — already domain-aware (identity_domain: FixedBytes<32>) | |
| 2. Gateway data provider (data_provider.rs) — encrypt_identity_data() takes typed IdentityData (needs to become domain-agnostic) | |
| 3. chainio (identity_data.rs) — IdentityData sol struct (8 KYC fields), fetch_identity_data() returns opaque String, | |
| EncryptedIdentityData (single string data) | |
| 4. Operator (core.rs) — decrypt_identity_data() returns IdentityData, resolve_identity_data() calls to_rego_data() | |
| 5. Core (rego/mod.rs) — evaluate() takes Option<IdentityData>, calls engine.with_newton_identity_extensions(data) | |
| 6. Regorus (extensions/identity.rs) — IdentityData struct, register_newton_identity_extensions(), 8 KYC built-ins | |
| ───────────────────────────────────────────────── | |
| --- | |
| I have all the context needed. Here's the implementation plan. I'll break it into ordered commits: | |
| Implementation Plan | |
| Commit 1: libs/regorus — Domain trait + KYC migration + generic get | |
| Files changed: | |
| - libs/regorus/src/extensions/identity.rs — The biggest change | |
| What: | |
| - Define IdentityDomainData trait with domain_name() -> &str, get_field(name) -> Option<Value>, reference_date() -> &str | |
| - Rename current IdentityData → KycIdentityData, implement trait | |
| - Rename all 8 built-ins from newton.identity.* → newton.identity.kyc.* | |
| - Add generic newton.identity.get(domain, field_name) built-in | |
| - Refactor register_newton_identity_extensions() → takes Box<dyn IdentityDomainData> + domain name, registers domain-namespaced | |
| built-ins | |
| - Add register_newton_identity_kyc_extensions() convenience method for KYC specifically | |
| - Update Engine::with_newton_identity_extensions() to accept the new trait object | |
| Commit 2: crates/core — Bridge types + evaluate() update | |
| Files changed: | |
| - crates/core/src/rego/mod.rs — Update re-exports, update evaluate() signature | |
| What: | |
| - Re-export KycIdentityData (was IdentityData) and IdentityDomainData trait | |
| - Update evaluate() to accept Option<Box<dyn IdentityDomainData>> instead of Option<IdentityData> | |
| - Update parse_and_evaluate_task() in crates/core/src/common/task.rs | |
| Commit 3: crates/chainio — Domain registry + EIP-712 update | |
| Files changed: | |
| - crates/chainio/src/identity_data.rs | |
| What: | |
| - Remove old IdentityData sol struct (the 8-field one) | |
| - Keep EncryptedIdentityData { string data } as the universal EIP-712 struct | |
| - Add IdentityDomain enum with Kyc variant (extensible) | |
| - Add parse_domain(domain: B256) -> Result<IdentityDomain> mapping keccak256("kyc") → Kyc | |
| - Add deserialize_identity_data(domain: IdentityDomain, json: &str, timestamp: String) -> Result<Box<dyn IdentityDomainData>> for | |
| domain dispatch | |
| - Add design decision comment about single EncryptedIdentityData vs per-domain EIP-712 structs | |
| - Update fetch_identity_data() to return Option<(B256, String)> (domain + raw data) | |
| Commit 4: crates/operator — Domain-aware decryption + Rego injection | |
| Files changed: | |
| - crates/operator/src/core.rs | |
| What: | |
| - Update PolicyTaskDataGenerator::decrypt_identity_data() to return raw JSON (not typed IdentityData) | |
| - Update resolve_identity_data() to use domain dispatch: fetch → get domain → decrypt → deserialize_identity_data(domain, json, | |
| timestamp) → return Box<dyn IdentityDomainData> | |
| - Update DataProviderGenerator impl | |
| Commit 5: crates/gateway — Update gateway-side identity handling | |
| Files changed: | |
| - crates/gateway/src/processor/data_provider.rs | |
| - crates/gateway/src/rpc/types/identity.rs | |
| - Gateway test files | |
| What: | |
| - Update PolicyTaskDataProvider::encrypt_identity_data() to be domain-agnostic (takes raw JSON + domain) | |
| - Update test mocks | |
| Commit 6: Tests + developer guide | |
| Files changed: | |
| - Unit tests across all changed files | |
| - docs/IDENTITY_DOMAIN_GUIDE.md — How to add a new identity domain (step-by-step for external devs) | |
| What: | |
| - KYC domain round-trip test (EIP-712 sign → on-chain → fetch → decrypt → Rego evaluate) | |
| - Generic newton.identity.get test with KYC data | |
| - Domain parse/dispatch tests | |
| - Developer guide: "Adding a new identity domain" with the 4 steps (sol struct, enum variant, Rego built-ins, SDK type) | |
| --- | |
| Key design decisions documented in code: | |
| 1. Single EncryptedIdentityData for all domains (with migration path comment for per-domain structs) | |
| 2. Domain is always required — no default fallback | |
| 3. Compile-time enum dispatch for domain→schema mapping | |
| 4. Both domain-namespaced built-ins (primary) and generic get (escape hatch) | |
| --- | |
| --- | |
| Batch 1: Scope and phasing | |
| 1. Which phases are in scope for this ticket? | |
| The ticket lists Phases 1-4. Phase 1 (schema registry + Rust refactor) and Phase 2 (generic Rego extensions) seem like the core | |
| work. Phase 3 (SDK) and Phase 4 (policy tooling/docs) feel like follow-up tickets. Are we doing Phases 1+2 together, or just Phase | |
| 1 for now? | |
| 2. Which concrete identity domains do we need to ship beyond kyc? | |
| The ticket lists social, credit, and professional as examples. Do we need to implement the actual data shapes and Rego built-ins | |
| for all four domains now, or just build the extensible infrastructure (trait, registry, generic newton.identity.get) and fully | |
| implement kyc as the first domain — leaving others as "add a struct and register it" later? | |
| 3. The ticket says "no backwards compatibility needed" — can we confirm there's zero deployed identity data on any chain (testnet | |
| included)? | |
| This determines whether we can do a clean break: rename all newton.identity.* extensions to newton.identity.kyc.*, make | |
| identity_domain required everywhere, and remove the bare IdentityData sol struct entirely. If there's any deployed data with | |
| existing EIP-712 signatures, those signatures would break when we change the struct. | |
| ✻ Worked for 3m 48s | |
| ❯ 1. we are doing all phases listed in the task that is `newton-prover-avs` repo relevant. For other repo work like SDK, toolings, | |
| docs, etc. we will do that after we do the newton-prover-avs core part 2. We are not shipping for specific identity domains, but | |
| newton vc should be designed to be "flexible" supporting any domain defined, focusing on the extensive architecture and fully | |
| implementing kyc as the first domain (kyc is not the default, but is just one of the domains that we are supporting as the 1st | |
| domain example). At the end, there should be a clear guidance, instructions and explanations for how other devs (external) can | |
| define their own dowmin and just use newton vc that makes most sense to their individual contexts (kyc vs social vs credit vs | |
| professional etc.) 3. no backwards compatibility needed --> confirmed. Do a hard migration, and focus on the core principles | |
| ⏺ Clear. Let me ask the next batch to nail down the technical design decisions. | |
| --- | |
| Batch 2: Technical design | |
| 1. Schema registry: compile-time enum dispatch or runtime registration? | |
| Two approaches for mapping bytes32 domain → data shape: | |
| - Compile-time: A Rust enum like IdentityDomain::Kyc(KycData) | IdentityDomain::Social(SocialData) | ... with match arms for | |
| EIP-712 hashing, deserialization, and Rego conversion. Adding a new domain means adding a variant + implementing the trait. Safer, | |
| but requires a code change + recompile to add domains. | |
| - Runtime: A HashMap<B256, Box<dyn IdentitySchema>> registry where schemas are registered at startup. More flexible for external | |
| teams but harder to get right with EIP-712 type hashes (each domain's EIP-712 struct must be known at compile time for SolStruct | |
| derive). | |
| Given that EIP-712 type hashes are baked into the sol! macro and can't be generated dynamically, I'm leaning toward compile-time | |
| enum dispatch with a clear trait that external contributors implement by adding a new variant. Does that match your mental model, | |
| or do you want the schema shapes themselves to be user-defined at runtime (e.g., JSON schema-driven)? | |
| 2. Rego extension namespacing: per-domain modules or a single generic accessor? | |
| The ticket suggests both: | |
| - A generic newton.identity.get(field_name) that works across any schema | |
| - Domain-namespaced built-ins like newton.identity.kyc.age_gte(min) | |
| For the generic get(field_name): should this return the raw field value from the deserialized data (so policy authors do their own | |
| comparisons in Rego), or should it also support typed helpers? The generic get approach means identity data would need to be | |
| representable as a flat HashMap<String, Value> regardless of domain — which is a design constraint on all future schemas. | |
| 3. Where should the domain-to-schema mapping live architecturally? | |
| Options: | |
| - crates/chainio/src/identity_data.rs (current home of IdentityData) — keeps EIP-712 + on-chain fetching together | |
| - crates/core/src/identity/ (new module) — since core is the shared dependency and Rego types already live in core | |
| - Split: trait + registry in core, EIP-712 impls in chainio (since EIP-712 needs alloy sol types) | |
| The split matters because libs/regorus (Rego extensions) depends on nothing — it can't import from core or chainio. The | |
| IdentityData struct is currently defined in regorus and re-exported by core. With flexible schemas, each domain's Rego data shape | |
| and built-ins need to live in regorus too. | |
| ❯ 1. Given that EIP-712 type hashes are baked into the sol! macro and can't be generated dynamically, I'm leaning toward | |
| compile-time | |
| enum dispatch with a clear trait that external contributors implement by adding a new variant. --> sounds good, agreed. 2. help | |
| me decide: - A generic newton.identity.get(field_name) that works across any schema | |
| - Domain-namespaced built-ins like newton.identity.kyc.age_gte(min) --> which one would provide most natural, seamless | |
| integration and user experience for users/developers? I am think the latter is better but that is just a guess. Correct my | |
| thoughts/intuitions if you have counter arguments. 3. | |
| ⏺ ★ Insight ───────────────────────────────────── | |
| EIP-712 type hashes are compile-time constants — alloy::sol! generates eip712_type_hash() at macro expansion time from the struct | |
| field names and types. This means you can't have a "dynamic schema" that produces valid EIP-712 signatures without pre-defining | |
| the Solidity struct shape in Rust code. This is why compile-time enum dispatch is the only sound approach here. | |
| ───────────────────────────────────────────────── | |
| On question 2 — your intuition is right, but I'd argue we should do both, and here's why: | |
| Domain-namespaced built-ins (newton.identity.kyc.age_gte(21)) are better for the common case: | |
| - Type-safe: age_gte knows it needs birthdate and reference_date fields, validates date formats, handles leap years. A generic | |
| get("birthdate") returns a raw string and pushes all that logic onto the policy author. | |
| - Discoverable: policy authors can browse newton.identity.kyc.* to see what's available for KYC. With generic get, they need to | |
| know the field names AND write comparison logic themselves. | |
| - Error messages are specific: "age_gte expects a positive valued age" vs a generic "field not found". | |
| A generic newton.identity.get(domain, field_name) is valuable as an escape hatch: | |
| - New domains can be used immediately in Rego policies even before domain-specific built-ins are written. A team adds a social | |
| domain struct with platform, handle, follower_count fields — they can write newton.identity.get("social", "follower_count") >= | |
| 1000 in Rego right away without waiting for newton.identity.social.follower_count_gte(). | |
| - It's trivial to implement if every domain schema implements a fn get_field(&self, name: &str) -> Option<Value> on the trait. | |
| Recommendation: Ship both. Domain-namespaced built-ins are the primary API. Generic get is the escape hatch for rapid prototyping | |
| and new domains. This is low incremental cost since the trait already needs a to_rego_fields() -> HashMap<String, Value> method | |
| anyway. | |
| You didn't answer question 3 yet — it's important for the implementation plan. Let me re-ask it along with the final batch: | |
| --- | |
| Batch 3: Architecture placement + integration | |
| 1. (Re-ask) Where should the domain-to-schema mapping live? | |
| My recommendation: split across libs/regorus, crates/core, and crates/chainio: | |
| ┌─────────────────────────────────────────┬────────────────────────────────────────────────────────┬──────────────────────────┐ | |
| │ Layer │ What lives here │ Why │ | |
| ├─────────────────────────────────────────┼────────────────────────────────────────────────────────┼──────────────────────────┤ | |
| │ │ Domain data structs (e.g., KycData, SocialData), Rego │ Regorus has zero deps on │ | |
| │ libs/regorus/src/extensions/identity.rs │ built-in functions, IdentityDomainData trait (for │ core/chainio. Rego │ | |
| │ │ get_field) │ extensions must live │ | |
| │ │ │ here. │ | |
| ├─────────────────────────────────────────┼────────────────────────────────────────────────────────┼──────────────────────────┤ | |
| │ │ Re-export of domain types, evaluate() updated to │ Already re-exports │ | |
| │ crates/core/src/rego/mod.rs │ accept domain-aware identity data │ IdentityData; becomes │ | |
| │ │ │ the bridge │ | |
| ├─────────────────────────────────────────┼────────────────────────────────────────────────────────┼──────────────────────────┤ | |
| │ │ EIP-712 sol structs per domain, IdentityDomainRegistry │ EIP-712 needs │ | |
| │ crates/chainio/src/identity_data.rs │ enum, fetch_identity_data() with domain dispatch, │ alloy::sol! which lives │ | |
| │ │ compute_identity_eip712_hash() per domain │ in chainio │ | |
| └─────────────────────────────────────────┴────────────────────────────────────────────────────────┴──────────────────────────┘ | |
| Does this layering make sense, or do you want it organized differently? | |
| 2. Should fetch_identity_data() return a typed enum or opaque JSON? | |
| Currently it returns Option<String> (the raw encrypted string from on-chain). After decryption, it gets parsed into IdentityData | |
| (KYC struct). With flexible schemas, two options: | |
| - Typed enum: fetch_identity_data() returns Option<IdentityDomainData> where IdentityDomainData is the compile-time enum. Requires | |
| the domain→struct mapping at fetch time. | |
| - Opaque JSON + late binding: fetch_identity_data() returns Option<(B256, String)> (domain + raw JSON string). The caller | |
| (operator/gateway) does domain dispatch when converting to Rego data. | |
| I lean toward opaque JSON + late binding since fetch_identity_data is an on-chain read that shouldn't need to know about Rego | |
| types, and the domain dispatch happens cleanly at the Rego injection point. | |
| 3. PR #437 (NEWT-692) introduced EncryptedIdentityData with a single string data field — is that PR merged? | |
| The ticket mentions it as related. If merged, the on-chain data is already an opaque blob (not the 8-field IdentityData struct), | |
| which simplifies our migration since we're just changing how the blob is deserialized after decryption. Need to confirm the | |
| current state of that PR to know what code path we're building on. | |
| ✻ Cooked for 34s | |
| ❯ 1. makse sense 2. make sense as well for your recommendation 3. PR #437 is merged: vrify here | |
| https://github.com/newt-foundation/newton-prover-avs/pull/437 | |
| --- | |
| Requirements Summary | |
| 1. Compile-time enum dispatch for domain-to-schema mapping. Adding a new domain = adding a Rust variant + sol struct + Rego | |
| built-ins + recompile. | |
| 2. Domain is always required — no backwards compatibility, no default domain. identity_domain must be specified in policy_params. | |
| 3. Both Rego APIs: domain-namespaced built-ins as primary (newton.identity.kyc.age_gte(21)) + generic newton.identity.get(domain, | |
| field) as escape hatch. | |
| 4. KYC as first domain example — fully implemented with all 8 existing built-ins migrated to newton.identity.kyc.* namespace. | |
| 5. Layering: regorus (data structs + Rego built-ins) → core (re-exports + evaluate bridge) → chainio (EIP-712 + on-chain fetch + | |
| domain registry). | |
| 6. Opaque JSON + late binding: fetch_identity_data() returns (B256, String), domain dispatch happens at Rego injection point. | |
| 7. Clear developer guide for adding new domains (in-repo, not external SDK — that's a follow-up). | |
| 8. Hard migration — remove old IdentityData struct, rename all newton.identity.* to newton.identity.kyc.*. | |
| --- | |
| One final question before I plan: | |
| The EncryptedIdentityData EIP-712 struct — PR #437 made it a single string data field. Is the EIP-712 signing now domain-agnostic | |
| (sign over the opaque blob regardless of domain), or does each domain still need its own EIP-712 struct type hash? | |
| If the signature is over EncryptedIdentityData { string data } universally, then the EIP-712 layer is already domain-agnostic and | |
| we don't need per-domain sol structs for signing — only for deserialization after decryption. That would significantly reduce | |
| scope. | |
| ★ Insight ───────────────────────────────────── | |
| EIP-712 type hashes serve as a semantic commitment — when a user signs IdentityData { string status; string birthdate; ... }, | |
| their wallet displays these field names and values. This is the core UX/security feature of EIP-712 vs raw hash signing. The | |
| question is whether this semantic commitment should happen at the encrypted blob level or the plaintext level. | |
| ───────────────────────────────────────────────── | |
| Option A: Single EncryptedIdentityData { string data } for all domains (current state post-PR #437) | |
| Pros: | |
| - EIP-712 layer is already done — zero work needed for signing/verification | |
| - Adding a new domain requires zero EIP-712 changes — just add deserialization + Rego built-ins | |
| - Simpler SDK: one signing flow for all domains | |
| - The signed blob is encrypted anyway — the user can't read it in their wallet regardless, so per-field EIP-712 display is moot | |
| for encrypted data | |
| Cons: | |
| - Wallet displays data: "0x8a3f..." (opaque blob) — user has no idea what they're signing | |
| - No type-level protection against submitting a social blob to a kyc domain slot (wrong domain, same EIP-712 type hash) | |
| - If we ever move to plaintext identity data (no encryption), the user experience degrades — they'd see a raw JSON string instead | |
| of structured fields | |
| Option B: Per-domain EIP-712 structs (KycIdentityData { string status; string birthdate; ... }, SocialIdentityData { string | |
| platform; string handle; ... }) | |
| Pros: | |
| - Wallet shows meaningful field names when signing plaintext data (future-proof for non-encrypted flows) | |
| - Type-level domain separation — a KYC signature can't be replayed as a social credential because the type hashes differ | |
| - Matches the Ethereum ecosystem norm: each data type gets its own EIP-712 struct | |
| - Stronger security invariant: the domain is cryptographically bound to the data shape, not just a routing parameter | |
| Cons: | |
| - More work: each new domain needs a sol! struct in chainio + corresponding EIP-712 signing logic | |
| - SDK complexity: each domain needs its own EIP-712 type definition client-side | |
| - Currently pointless: data is encrypted, so the wallet shows garbage either way | |
| - Creates a coupling between data schema evolution and signature compatibility — adding a field to KYC data invalidates all | |
| existing signatures (the ABI selector gotcha from lessons.md) | |
| My recommendation | |
| Keep Option A (single EncryptedIdentityData) for now. Reasons: | |
| 1. The data is encrypted. Per-field EIP-712 display is meaningless when the wallet shows data: "encrypted_blob". The security | |
| benefit of structured signing only materializes for plaintext flows, which don't exist yet. | |
| 2. Domain binding can be achieved via AAD. Instead of per-domain type hashes, bind the domain to the signature via the EIP-712 | |
| domain separator or AAD — e.g., include identity_domain in the signing context. This gives you cross-domain replay protection | |
| without per-domain structs. | |
| 3. Migration path is clean. If you later need per-domain structs (e.g., for a plaintext VC flow), you add them as a new signing | |
| path alongside the encrypted path. No breaking change — just a new option. | |
| 4. Scope. Per-domain EIP-712 structs multiply the work across every layer (Rust sol structs, SDK type definitions, wallet | |
| integration) for zero user-visible benefit today. The ticket is already 4 phases. | |
| The one thing I'd add: include identity_domain: bytes32 as a field in EncryptedIdentityData (or in the EIP-712 domain separator) | |
| so the domain is cryptographically bound to the signature. This prevents submitting a social blob with a KYC domain tag. | |
| Does that reasoning track, or do you want per-domain EIP-712 structs from day one? | |
| ✻ Churned for 32s | |
| ❯ that sounds reasonable. But do add a comment about considering migration to per-domain specific structs for reasons you specified | |
| above (with notes on the downside, reasoning about why we are going with "Single EncryptedIdentityData { string data } for all | |
| domains", at least to begin with | |
| ★ Insight ───────────────────────────────────── | |
| The full identity data flow has 6 touchpoints that need updating: | |
| 1. Gateway RPC (identity.rs types) — already domain-aware (identity_domain: FixedBytes<32>) | |
| 2. Gateway data provider (data_provider.rs) — encrypt_identity_data() takes typed IdentityData (needs to become domain-agnostic) | |
| 3. chainio (identity_data.rs) — IdentityData sol struct (8 KYC fields), fetch_identity_data() returns opaque String, | |
| EncryptedIdentityData (single string data) | |
| 4. Operator (core.rs) — decrypt_identity_data() returns IdentityData, resolve_identity_data() calls to_rego_data() | |
| 5. Core (rego/mod.rs) — evaluate() takes Option<IdentityData>, calls engine.with_newton_identity_extensions(data) | |
| 6. Regorus (extensions/identity.rs) — IdentityData struct, register_newton_identity_extensions(), 8 KYC built-ins | |
| ───────────────────────────────────────────────── | |
| --- | |
| I have all the context needed. Here's the implementation plan. I'll break it into ordered commits: | |
| Implementation Plan | |
| Commit 1: libs/regorus — Domain trait + KYC migration + generic get | |
| Files changed: | |
| - libs/regorus/src/extensions/identity.rs — The biggest change | |
| What: | |
| - Define IdentityDomainData trait with domain_name() -> &str, get_field(name) -> Option<Value>, reference_date() -> &str | |
| - Rename current IdentityData → KycIdentityData, implement trait | |
| - Rename all 8 built-ins from newton.identity.* → newton.identity.kyc.* | |
| - Add generic newton.identity.get(domain, field_name) built-in | |
| - Refactor register_newton_identity_extensions() → takes Box<dyn IdentityDomainData> + domain name, registers domain-namespaced | |
| built-ins | |
| - Add register_newton_identity_kyc_extensions() convenience method for KYC specifically | |
| - Update Engine::with_newton_identity_extensions() to accept the new trait object | |
| Commit 2: crates/core — Bridge types + evaluate() update | |
| Files changed: | |
| - crates/core/src/rego/mod.rs — Update re-exports, update evaluate() signature | |
| What: | |
| - Re-export KycIdentityData (was IdentityData) and IdentityDomainData trait | |
| - Update evaluate() to accept Option<Box<dyn IdentityDomainData>> instead of Option<IdentityData> | |
| - Update parse_and_evaluate_task() in crates/core/src/common/task.rs | |
| Commit 3: crates/chainio — Domain registry + EIP-712 update | |
| Files changed: | |
| - crates/chainio/src/identity_data.rs | |
| What: | |
| - Remove old IdentityData sol struct (the 8-field one) | |
| - Keep EncryptedIdentityData { string data } as the universal EIP-712 struct | |
| - Add IdentityDomain enum with Kyc variant (extensible) | |
| - Add parse_domain(domain: B256) -> Result<IdentityDomain> mapping keccak256("kyc") → Kyc | |
| - Add deserialize_identity_data(domain: IdentityDomain, json: &str, timestamp: String) -> Result<Box<dyn IdentityDomainData>> for | |
| domain dispatch | |
| - Add design decision comment about single EncryptedIdentityData vs per-domain EIP-712 structs | |
| - Update fetch_identity_data() to return Option<(B256, String)> (domain + raw data) | |
| Commit 4: crates/operator — Domain-aware decryption + Rego injection | |
| Files changed: | |
| - crates/operator/src/core.rs | |
| What: | |
| - Update PolicyTaskDataGenerator::decrypt_identity_data() to return raw JSON (not typed IdentityData) | |
| - Update resolve_identity_data() to use domain dispatch: fetch → get domain → decrypt → deserialize_identity_data(domain, json, | |
| timestamp) → return Box<dyn IdentityDomainData> | |
| - Update DataProviderGenerator impl | |
| Commit 5: crates/gateway — Update gateway-side identity handling | |
| Files changed: | |
| - crates/gateway/src/processor/data_provider.rs | |
| - crates/gateway/src/rpc/types/identity.rs | |
| - Gateway test files | |
| What: | |
| - Update PolicyTaskDataProvider::encrypt_identity_data() to be domain-agnostic (takes raw JSON + domain) | |
| - Update test mocks | |
| Commit 6: Tests + developer guide | |
| Files changed: | |
| - Unit tests across all changed files | |
| - docs/IDENTITY_DOMAIN_GUIDE.md — How to add a new identity domain (step-by-step for external devs) | |
| What: | |
| - KYC domain round-trip test (EIP-712 sign → on-chain → fetch → decrypt → Rego evaluate) | |
| - Generic newton.identity.get test with KYC data | |
| - Domain parse/dispatch tests | |
| - Developer guide: "Adding a new identity domain" with the 4 steps (sol struct, enum variant, Rego built-ins, SDK type) | |
| --- | |
| Key design decisions documented in code: | |
| 1. Single EncryptedIdentityData for all domains (with migration path comment for per-domain structs) | |
| 2. Domain is always required — no default fallback | |
| 3. Compile-time enum dispatch for domain→schema mapping | |
| 4. Both domain-namespaced built-ins (primary) and generic get (escape hatch) | |
| --- | |
| Does this plan look right? Any adjustments before I start implementing? | |
| ✻ Sautéed for 1m 6s | |
| ❯ make sure you document in / add to the relevant documentation for the discussusions and consdierations above (design rationales, | |
| etc.) | |
| Good — I now have the full architecture doc. The plan is solid and I have all the context needed. Let me update the plan to | |
| include documentation deliverables explicitly, then start implementing. | |
| Documentation deliverables added to the plan: | |
| 1. docs/IDENTITY_REGISTRY_VC_ARCHITECTURE.md — Update existing doc: | |
| - Update "System Overview" mermaid diagram to show domain dispatch | |
| - Update "Rego Identity Extensions" section to reflect domain-namespaced built-ins | |
| - Update "IdentityData struct" table to show it's now KYC-specific | |
| - Add "Domain-Flexible Architecture" section with design rationale | |
| - Update "Phases and Roadmap" to reflect this work as complete | |
| - Update "Known Gaps" section (remove "Single Identity Domain Per Policy" gap — now addressed) | |
| 2. docs/IDENTITY_DOMAIN_GUIDE.md — New developer guide: | |
| - "Design Decisions" section (single EncryptedIdentityData vs per-domain EIP-712, compile-time enum dispatch vs runtime, | |
| domain-namespaced vs generic get) | |
| - "Adding a New Identity Domain" step-by-step (4 steps across regorus/core/chainio) | |
| - Example: walking through a hypothetical social domain end-to-end | |
| - Reference to existing KYC domain as canonical example | |
| 3. Code-level design rationale comments at key decision points in: | |
| - crates/chainio/src/identity_data.rs — why single EncryptedIdentityData, migration path to per-domain structs | |
| - libs/regorus/src/extensions/identity.rs — why both namespaced built-ins and generic get | |
| - crates/chainio/src/identity_data.rs — why compile-time enum dispatch |
Author
Author
Good — typecheck passes. Here's my review of the current state.
Current Progress: Identity Module (Phase 3 — SDK Support)
The gist describes a 6-commit plan across newton-prover-avs to support flexible identity domains beyond KYC. Phases 1-2 (backend:
schema registry, Rego extensions) are complete. Phase 3 is the SDK client-side support, which is what these changes tackle.
What's been done
New files:
- src/modules/identity/index.ts — sendIdentityEncrypted and identityDomainHash
- src/types/identity.ts — SendIdentityEncryptedParams, SendIdentityEncryptedResponse, SendIdentityEncryptedRpcRequest,
IdentityDomainName
Wiring:
- src/const.ts — added newt_sendIdentityEncrypted RPC method
- src/index.ts — replaced the connectIdentityWithNewton placeholder stub with real sendIdentityEncrypted and identityDomainHash on
the wallet client actions; added standalone exports - src/types/index.ts — re-exports identity types
- README.md — new Identity Module section
Typecheck passes cleanly.
Issues and gaps
- No IDENTITY_REGISTRY addresses in const.ts — The identityRegistryAddress is a required param the caller must provide. Unlike
TASK_MANAGER_ADDRESSES and ATTESTATION_VALIDATOR, there's no per-chain lookup. This is either intentional (caller always provides)
or an oversight. If the registry is deployed at known addresses, we should add them to const.ts and default like other contracts. - Old exports removed without deprecation — connectIdentityWithNewton (the placeholder) was part of the public API surface in
v0.5.0. Removing it is a breaking change. If any consumers referenced it, this needs a major bump or at least a changelog note.
Same for registerUserData, linkApp, unlinkApp mentioned in CLAUDE.md's wallet actions list — none of these exist in code, so
CLAUDE.md's architecture section is stale. - No input validation on sendIdentityEncrypted — Per .claude/rules/security.md, the SDK should validate addresses with isAddress
before use. params.identityOwner and params.identityRegistryAddress are passed straight through without validation. The
identityOwner should also be checked against walletClient.account.address to prevent signing for a mismatched owner. - Error handling uses bare Error instead of SDKError — Lines 48-49 of the identity module throw new Error(...). The rest of the
SDK uses SDKError with error codes. Should be throw new SDKError('walletClient must have a chain configured', 'MISSING_CHAIN')
etc. - identityDomainHash on wallet client is odd — It's a pure function (no client state needed) but exposed on the wallet client
actions object. It's already exported standalone, which is the right pattern. Having it on the client extension adds API surface
without value. - No tests — src/modules/identity/ has no .test.ts file. At minimum: unit test for identityDomainHash (verify keccak256 output
matches Rust/Solidity), and a mock-based test for the EIP-712 signing flow in sendIdentityEncrypted. - IdentityDomainName type is narrow — Currently 'kyc' only. The gist's whole point is extensibility beyond KYC. The type should
either be a wider union reflecting known domains or just string — the | string escape hatch on identityDomain already handles
this, but having the union type with only one member is misleading about the current state. - Missing from types/task.ts re-export — src/types/index.ts doesn't re-export task.ts or policy.ts types (only core/*, identity,
privacy). This is pre-existing but worth noting since the identity types follow the same barrel export pattern.
Remaining items to ship Phase 3
┌────────────────────────────────────────────────────────────────────────────────────────────────────┬───────────┬──────────┐
│ Item │ Status │ Priority │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ sendIdentityEncrypted + identityDomainHash implementation │ Done │ — │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ RPC method constant │ Done │ — │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ Type definitions │ Done │ — │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ Wallet client action wiring │ Done │ — │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ Input validation (addresses, owner match) │ Missing │ High │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ SDKError with proper error codes │ Missing │ High │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ Unit tests │ Missing │ High │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ IDENTITY_REGISTRY addresses in const.ts (if deployed) │ TBD │ Medium │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ Remove identityDomainHash from wallet client actions │ Suggested │ Low │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ CLAUDE.md architecture section update (remove stale registerUserData/linkApp/unlinkApp references) │ Missing │ Medium │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ Docs sync to site/developers/reference/sdk-reference.mdx │ Missing │ Medium │
├────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼──────────┤
│ Changelog / breaking change note for connectIdentityWithNewton removal │ Missing │ Medium │
└────────────────────────────────────────────────────────────────────────────────────────────────────┴───────────┴──────────┘
Author
NEWT-710 Complete Review
Implementation Status: Done (Phases 1-3), Phase 4 mostly done
All code implementation is complete. The gaps I found and fixed during this review:
- newt_sendIdentityEncrypted was missing from rpc-api.mdx — now added
- identity-policy-reference.mdx used old newton.identity.* prefix — migrated to newton.identity.kyc.*
- integration-guide.mdx had 3 old-prefix code examples — fixed
Remaining non-blocking Phase 4 items
- Example policies for social/credit/professional domains — these domains don't exist yet, so this is aspirational
- Policy template generator — future consideration
What to watch at merge/deploy time
- Mainnet address: IDENTITY_REGISTRY in const.ts has no mainnet entry. Add it when IdentityRegistry deploys to mainnet.
- ABI sync: After #444 merges, verify newtonIdentityRegistryAbi.ts matches the deployed contract (it should — the ABI was built
from the #444 contract interface).
E2E Testing Plan for Newton Verifiable Credential
There's no automated e2e test today. Here's how to test the full flow manually against stagef:
Prerequisites:
- A funded wallet on Sepolia (for gas)
- A Newton API key with RpcWrite permission
- A deployed PolicyClient contract with a KYC-gated policy
Steps:
- Submit identity data (sendIdentityEncrypted)
const walletClient = createWalletClient({ chain: sepolia, transport: http(), account })
.extend(newtonWalletClientActions({ apiKey: API_KEY }))
const result = await walletClient.sendIdentityEncrypted({
identityOwner: account.address,
identityData: 'encrypted-kyc-blob',
identityDomain: 'kyc',
})
// Verify: result.inclusion_tx is a valid tx hash
// Verify: tx confirms on-chain
2. Link identity to PolicyClient (linkIdentityAsSignerAndUser)
const txHash = await walletClient.linkIdentityAsSignerAndUser({
policyClient: POLICY_CLIENT_ADDRESS,
identityDomains: [identityDomainHash('kyc')],
})
// Verify: tx confirms on-chain
// Verify: IdentityRegistry.policyClientLinks returns true
3. Submit a task with the KYC-gated policy
const task = await walletClient.submitEvaluationRequest({
intent: { from: account.address, to: '0x...', value: '0', data: '0x', chainId: 11155111, functionSignature: '0x' },
})
// Verify: task evaluates using newton.identity.kyc.* built-ins
// Verify: attestation is returned (allow or reject based on policy)
4. Unlink identity (unlinkIdentityAsUser)
const txHash = await walletClient.unlinkIdentityAsUser({
policyClient: POLICY_CLIENT_ADDRESS,
identityDomains: [identityDomainHash('kyc')],
})
// Verify: link is removed
// Verify: subsequent task with same policy now fails identity checks
5. Negative cases:
- Submit task without linking identity — should fail/reject
- Link with wrong domain — should not satisfy policy
- LinkAlreadyExists — try linking twice, second should revert
- Signer mismatch — sendIdentityEncrypted with different wallet than identityOwner
Automation: This could be scripted as a Vitest integration test that runs against a live stagef gateway + Sepolia fork (anvil).
The main blocker is needing real gateway responses — you'd need either a live stagef test or a mock gateway that exercises the
full flow.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Phase 1: Schema Registry — DONE
All items completed on this branch:
is encrypted anyway)
Phase 2: Generic Rego Extensions — DONE
All items completed (regorus PR #4, merged):
newton.identity.kyc.country_code(), newton.identity.kyc.iso_subdivision()
Phase 3: SDK and Client Support — NOT STARTED
Remaining items:
hardcodes the KYC field shape
struct) rather than the old IdentityData struct
Phase 4: Policy Tooling — PARTIALLY DONE
┌───────────────────────────────────────────────────────────┬───────────────────────────────────────────────────────┐
│ Item │ Status │
├───────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Rego policy reference docs with domain-specific built-ins │ Done (docs/REGORUS.md, docs/IDENTITY_DOMAIN_GUIDE.md) │
├───────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Example policies for social, credit, professional │ Not started (no non-KYC domains exist yet) │
├───────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────┤
│ Policy template generator │ Not started (stretch goal) │
└───────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────┘
Summary
┌────────────────────────────┬────────────────┬─────────────────────────────────────────────────────────┐
│ Phase │ Status │ Remaining Work │
├────────────────────────────┼────────────────┼─────────────────────────────────────────────────────────┤
│ 1. Schema Registry │ Done │ — │
├────────────────────────────┼────────────────┼─────────────────────────────────────────────────────────┤
│ 2. Generic Rego Extensions │ Done │ — │
├────────────────────────────┼────────────────┼─────────────────────────────────────────────────────────┤
│ 3. SDK + Client Support │ Not started │ newton-sdk updates (separate repo) │
├────────────────────────────┼────────────────┼─────────────────────────────────────────────────────────┤
│ 4. Policy Tooling │ Partially done │ Example policies for future domains, template generator │
└────────────────────────────┴────────────────┴─────────────────────────────────────────────────────────┘
★ Insight ─────────────────────────────────────
Phases 1-2 are the AVS-side work (this repo + regorus submodule) — that's complete and ready for review on PR #442. Phase 3 is
newton-sdk work (separate repo, separate PR). Phase 4's remaining items are blocked on Phase 3 since there's no point writing
example social/credit policies until the SDK can actually submit data for those domains. The PR can merge as-is — Phase 3-4 are
follow-up tickets.
─────────────────────────────────────────────────