Skip to content

Instantly share code, notes, and snippets.

@denniswon

denniswon/gist:32f8d9d25144dbc435f6da7c0f24f242

Created February 22, 2026 16:13
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save denniswon/32f8d9d25144dbc435f6da7c0f24f242 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save denniswon/32f8d9d25144dbc435f6da7c0f24f242 to your computer and use it in GitHub Desktop.
Download ZIP
E2E Test: Stagef Secrets Upload Flow
Raw
gistfile1.txt
Plan to implement │
│ │
│ E2E Test: Stagef Secrets Upload Flow │
│ │
│ Context │
│ │
│ We need a cross-service E2E test that validates the full API key + secrets upload pipeline across newton-dashboard-api and │
│ newton-prover-avs gateway on the Sepolia stagef environment. This is the first cross-service E2E test for the Newton platform. │
│ │
│ The flow: seed an API key in the shared newton_gateway DB -> encrypt secrets with the KMS RSA public key -> call gateway │
│ newt_storeEncryptedSecrets -> verify the secret was persisted. │
│ │
│ Approach │
│ │
│ Create a standalone Python script scripts/e2e_stagef_secrets.py that bypasses dashboard auth (no OTP flow) by seeding test data │
│ directly in the shared Aurora RDS, then exercises the gateway RPC endpoint. │
│ │
│ Why direct DB seeding (not dashboard API)? │
│ │
│ - Dashboard auth requires interactive email OTP (not automatable without email access) │
│ - The shared newton_gateway DB is the integration point — seeding directly tests that the gateway reads dashboard-created keys │
│ - The gateway's AuthManager picks up new keys immediately on cache miss (DB fallback path) │
│ │
│ Prerequisites │
│ │
│ - AWS CLI configured with access to stagef account (701849097212) │
│ - newton-dashboard-api-db-secret accessible in AWS Secrets Manager │
│ - Network access to stagef Aurora RDS (VPN or bastion) │
│ - Python packages: asyncpg, httpx, cryptography, boto3 │
│ │
│ Test Flow │
│ │
│ Step 1: Get DB credentials from AWS Secrets Manager │
│ │
│ boto3 -> secretsmanager.get_secret_value("newton-dashboard-api-db-secret") │
│ -> parse JSON -> {host, username, password, name/dbname} │
│ │
│ Step 2: Seed test data in shared newton_gateway DB │
│ │
│ Connect to stagef Aurora RDS via asyncpg and: │
│ │
│ 1. Upsert test user in user table: │
│ - id: deterministic UUID (e.g., UUID5 from "e2e-test-stagef") │
│ - public_address: 0xd1168701C5df97bdb65361aAF9318ae2c7e39312 (admin wallet) │
│ - is_active: true │
│ 2. Upsert test API key in api_keys table: │
│ - user_id: the test user's UUID │
│ - address: \xd1168701C5df97bdb65361aAF9318ae2c7e39312 (20-byte BYTEA) │
│ - api_key: generated gw_<random> token │
│ - name: e2e-test-stagef │
│ - permissions: ["rpc_write"] │
│ - is_active: true │
│ │
│ Step 3: Verify on-chain ownership (informational) │
│ │
│ Call Sepolia RPC to check NewtonPolicyClient(0x0168...7688).getOwner() matches 0xd1168701C5df97bdb65361aAF9318ae2c7e39312. This is │
│ the same check the gateway performs. If mismatch, abort with clear error. │
│ │
│ - Policy client: 0x0168649971b58bd65f0eb851e6acbbf776ea7688 │
│ - Policy data: 0x8a4f7a146f4f76364c44e065d7e409a955776eb1 │
│ - Chain ID: 11155111 (Sepolia) │
│ │
│ Step 4: Encrypt test secrets with KMS RSA public key │
│ │
│ Use Python cryptography library to encrypt a test JSON payload: │
│ │
│ from cryptography.hazmat.primitives.asymmetric import padding │
│ from cryptography.hazmat.primitives import hashes, serialization │
│ │
│ # Load the RSA public key (provided by user, KMS key a5132add-...) │
│ public_key = serialization.load_pem_public_key(pem_bytes) │
│ │
│ # Encrypt with RSAES-OAEP-SHA-256 (matches gateway's kms.decrypt algorithm) │
│ plaintext = json.dumps({"TEST_KEY": "test_value"}).encode() │
│ ciphertext = public_key.encrypt(plaintext, padding.OAEP( │
│ mgf=padding.MGF1(algorithm=hashes.SHA256()), │
│ algorithm=hashes.SHA256(), │
│ label=None │
│ )) │
│ secrets_b64 = base64.b64encode(ciphertext).decode() │
│ │
│ Note: RSA-4096 OAEP/SHA-256 max plaintext = 446 bytes. Keep test secrets small. │
│ │
│ Step 5: Call gateway newt_storeEncryptedSecrets │
│ │
│ POST https://gateway-avs.stagef.sepolia.newt.foundation/rpc │
│ Headers: Authorization: Bearer <seeded_api_key> │
│ Body: { │
│ "jsonrpc": "2.0", │
│ "method": "newt_storeEncryptedSecrets", │
│ "params": { │
│ "policy_client": "0x0168649971b58bd65f0eb851e6acbbf776ea7688", │
│ "policy_data_address": "0x8a4f7a146f4f76364c44e065d7e409a955776eb1", │
│ "secrets": "<base64_ciphertext>" │
│ }, │
│ "id": "e2e-test-1" │
│ } │
│ │
│ Expected: {"result": {"success": true, "schema": {...}, "error": null}} │
│ │
│ Step 6: Verify secret was persisted (direct DB query) │
│ │
│ SELECT policy_client_address, policy_data_address, secrets │
│ FROM policy_client_secret │
│ WHERE policy_client_address = '\x0168...' AND policy_data_address = '\x8a4f...' │
│ │
│ Verify the row exists and secrets is non-empty BYTEA. │
│ │
│ Step 7: Cleanup │
│ │
│ - Delete the seeded API key from api_keys (by name e2e-test-stagef) │
│ - Delete the test user from user table │
│ - Optionally delete the policy_client_secret row (or leave it — it's an upsert) │
│ │
│ File Structure │
│ │
│ scripts/e2e_stagef_secrets.py # Main E2E test script (NEW) │
│ requirements-dev.txt # Add: cryptography, boto3, asyncpg (if not already present) │
│ │
│ Key Files Referenced │
│ │
│ - app/core/blockchain/deployments/11155111-stagef.json — contract addresses │
│ - app/utils/api_key.py — generate_api_key() for consistent key format │
│ - docs/RPC_API.md — gateway RPC spec for newt_storeEncryptedSecrets │
│ - manual_test_email_login.py — existing test script pattern to follow │
│ │
│ Configuration │
│ │
│ All values hardcoded in the script (no .env needed): │
│ │
│ ┌────────────────────┬────────────────────────────────────────────────────────┐ │
│ │ Config │ Value │ │
│ ├────────────────────┼────────────────────────────────────────────────────────┤ │
│ │ Dashboard API │ https://dashboard.api.stagef.newt.foundation │ │
│ ├────────────────────┼────────────────────────────────────────────────────────┤ │
│ │ Gateway RPC │ https://gateway-avs.stagef.sepolia.newt.foundation/rpc │ │
│ ├────────────────────┼────────────────────────────────────────────────────────┤ │
│ │ DB Secret │ newton-dashboard-api-db-secret (AWS Secrets Manager) │ │
│ ├────────────────────┼────────────────────────────────────────────────────────┤ │
│ │ Admin wallet │ 0xd1168701C5df97bdb65361aAF9318ae2c7e39312 │ │
│ ├────────────────────┼────────────────────────────────────────────────────────┤ │
│ │ Policy client │ 0x0168649971b58bd65f0eb851e6acbbf776ea7688 │ │
│ ├────────────────────┼────────────────────────────────────────────────────────┤ │
│ │ Policy data │ 0x8a4f7a146f4f76364c44e065d7e409a955776eb1 │ │
│ ├────────────────────┼────────────────────────────────────────────────────────┤ │
│ │ Chain ID │ 11155111 │ │
│ ├────────────────────┼────────────────────────────────────────────────────────┤ │
│ │ KMS key ID │ a5132add-4e12-45a0-bfad-664a65e3b03b │ │
│ ├────────────────────┼────────────────────────────────────────────────────────┤ │
│ │ KMS RSA public key │ Embedded in script │ │
│ └────────────────────┴────────────────────────────────────────────────────────┘ │
│ │
│ Verification │
│ │
│ 1. Run: python scripts/e2e_stagef_secrets.py │
│ 2. Expected output: step-by-step progress with pass/fail for each stage │
│ 3. Gateway returns success: true with schema │
│ 4. DB query confirms secret row exists │
│ 5. Cleanup removes test artifacts │
│ │
│ Risks / Notes │
│ │
│ - On-chain ownership: If getOwner() of policy client 0x0168... does not return 0xd116..., the gateway will reject the request. The │
│ script verifies this upfront. │
│ - PolicyData schema: If 0x8a4f... has no secretsSchemaCid on-chain, the gateway cannot validate secrets. The script should handle │
│ this gracefully (the gateway returns the error in the response). │
│ - Network access: Requires VPN/bastion to reach stagef Aurora RDS. │
│ - Gateway key cache: New API keys are picked up immediately via DB fallback on cache miss — no 5-minute wait needed. │
│ - RSA-OAEP compatibility: Local encryption with the raw public key produces standard RSA-OAEP ciphertext that AWS KMS can decrypt │
│ with RSAES_OAEP_SHA_256. │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment