jordotech / 2026-05-09-eng-1492-direct-parquet-upload.md

Created May 9, 2026 21:20

ENG-1492 Direct Parquet Upload — Implementation Plan (13-task TDD)

ENG-1492 Direct Parquet Upload — Plan 1 Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Wire the PR-448 multipart upload endpoints into the full dataset pipeline (S3 combine → Parquet schema inference → Glue + collection dataset registration) as a 202-Accepted background pipeline, backed by a completion_status state machine with TTL-based DDB row retention.

Architecture: POST /complete returns 202 immediately. Work runs post-response via BackgroundTasks as a three-step chain (combine → infer → register) with per-step try/except rollback. Status polled via GET /multipart/{upload_id}. Terminal multipart_uploads_v1 rows expire via DynamoDB TTL (30d success/abort, 90d failure, 7d orphan). Terraform PR adds S3 AbortIncompleteMultipartUpload lifecycle rule + enables DDB TTL.

Tech Stack:

jordotech / 2026-04-17-eng-1131-encryption-key-mgmt-ui.md

Created April 17, 2026 13:57

ENG-1131: Encryption Key Management UI — Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add a self-service Encryption Key Management page to Organization Settings, gated by a new is_infra_admin flag, with DDB+SSM CRUD for Azure Key Vault credentials.

Architecture: Orthogonal is_infra_admin boolean on organization_x_members table (clj-pg-wrapper), exposed via JWT claim. platform-api gets 4 new /api/v1/admin/encryption/* endpoints using DDB+SSM hybrid storage (mirrors client_api_keys). Frontend adds /settings/encryption page with Configuration + Status tabs, plus a checkbox in the existing admin edit-member modal.

Tech Stack: Python/FastAPI (clj-pg-wrapper, platform-api), TypeScript/React (platform-frontend), Terraform/HCL, Azure SDK (azure-identity, azure-keyvault-keys), DynamoD

jordotech / incident-report.md

Last active April 14, 2026 17:41

Incident Report: ey-eu-west-1 Workflow Queue Pile-Up (2026-04-14)

Incident Report: ey-eu-west-1 Workflow Queue Pile-Up

Date: 2026-04-14 Duration: ~05:00 - 07:00 CST (11:00 - 13:00 UTC) Environment: ey-eu-west-1 Severity: S2 — User-facing workflow delays, incomplete executions Status: Investigating

Timeline

jordotech / discovery-data-scope-status.md

Created April 3, 2026 15:11

Discovery Institute — Data Scope Status Report

Date: 2026-04-03 Prepared for: Customer Success Collection: org_42caca5c-da0e-4c91-9bf3-d546266fd2e6_discovery-v1 Collection ID: c85d4618-b581-4413-a855-a4739125e705 Total chunks in Qdrant: 131,896

jordotech / ey-application-level-encryption.md

Last active March 26, 2026 22:12

EY Application-Level S3 Encryption - Technical Overview

Application-Level S3 Encryption: Technical Overview

Prepared for: EY Information Security Team Date: March 2026 Status: Proposed (LOE Review)

Executive Summary

jordotech / admin-feature-ENG-857-no-celery-design-20260326-134155.md

Last active March 26, 2026 21:16

Design: Application-Level S3 Encryption for EY Environments — LOE & Technical Design

Design: Application-Level S3 Encryption for EY Environments

Generated by /office-hours on 2026-03-26 Branch: feature/ENG-857-no-celery Repo: Faction-V/gofigure_terraform Status: APPROVED Mode: Intrapreneurship

Problem Statement

jordotech / ey-platform-access-control.md

Created March 25, 2026 17:21

Capitol AI — Data Access Controls for Customer Organizations

Overview

Capitol AI has implemented platform-level access control that restricts who can view and download an organization's files. Access is determined by the user's verified email domain — only users with authorized email addresses (e.g., @ey.com) can access the organization's data, even if other users have platform administrator privileges.

This control works alongside the existing External Key Management (EKM) encryption to provide multiple independent layers of data protection.

jordotech / per-org-s3-access-isolation.md

Created March 25, 2026 15:23

ENG-893: Per-Org S3 Access Isolation — Strategy & Test Results

ENG-893: Per-Org S3 Access Isolation via Email Domain Verification

Problem

Capitol.ai is a multi-tenant platform where organizations share the same AWS infrastructure. A Capitol.ai admin can add themselves to any organization (e.g., EY) and gain full access to that org's S3 files — uploads, workflow files, and generated outputs. Client orgs need assurance that only users with verified email domains can access their data.

Strategy

We implement email-domain-scoped IAM role assumption — a belt-and-suspenders approach combining STS AssumeRole with explicit IAM Deny policies.

jordotech / celery-to-async-worker.md

Last active March 21, 2026 20:58

ENG-857: Replacing Celery with Async Workers — Internal Briefing

Replacing Celery with Async Workers (ENG-857)

Branch: feature/ENG-857-no-celery (agentic-backend + terraform) Status: Deployed to HMG, ready for load testing

Pull Requests

App: https://github.com/Faction-V/agentic-backend/pull/410

jordotech / skills-repo-split-analysis.md

Created March 10, 2026 20:08

Analysis: Should skills/ live in a separate repo? (spoiler: no)

Should `skills/` Live in a Separate Repo?

TL;DR: The skills/ directory is 3.6MB / 282 files — smaller than a single retina screenshot. Splitting it into a separate repo adds real operational complexity to solve a non-problem, and partially reintroduces the version-coupling failures that caused the outages we're trying to fix.

Actual Size of `skills/`

Total: 3.6 MB, 282 files

jordotech jordotech