SSP Workflow Architecture - Judge API

SSP_WORKFLOW_ARCHITECTURE.md

SSP Workflow Architecture

Overview

The System Security Plan (SSP) workflow in Judge API is a sophisticated multi-stage pipeline that generates, transforms, and stores OSCAL-compliant SSP documents. This workflow leverages native Dapr activities for better resource isolation, error handling, and reliability.

Workflow Stages

1. Repository Cloning

Activity: CloneRepositoriesActivity

The workflow begins by cloning source code repositories associated with the input product. These repositories contain the system implementation that will be analyzed for compliance.

• Input: Repository URLs from the product configuration
• Process:
  • Clones repositories to a temporary workspace (/tmp/ssp-repos)
  • Creates tar.gz archive of all repositories
  • Uploads archive to blob storage for sharing between activities
• Output:
  • Local paths to cloned repositories
  • Blob storage key for the archived repositories

2. SSP Generation with AgentFlow + OscalSynth

Activity: SSPGeneratorActivity

This is the core activity that uses Claude AI (via AgentFlow) to analyze the codebase and generate an OSCAL-compliant SSP.

Process Flow

1. Repository Extraction: If repositories were archived to blob storage, extracts them to workspace
2. OSCAL SSP Creation: Uses oscal.CreateSSPHandler to create initial SSP structure with UUID
3. AgentFlow Workflow Execution:
   • Build Inventory Step: Analyzes repositories to identify system components
   • Implement AC-2 Step: Maps components to AC-2 control statements with evidence
   • Validate Coverage Step: Ensures all AC-2 statements are covered

Key Features

• Statement-level control implementation (not just control-level)
• Evidence tracking with gaps identification
• Implementation status determination (implemented/partial/planned)
• Component-based control mapping

Output Files

• OSCAL JSON ({uuid}.json): Standard OSCAL SSP format
• Archivista-compatible JSON (ssp.json): Transformed for Archivista storage
• Markdown (ssp.md): Human-readable SSP document

3. PDF Conversion

Activity: PDFConverterActivity

Converts the markdown SSP into a PDF report using Pandoc and pdflatex.

• Input: Markdown file path from SSP generator
• Process: Pandoc markdown → LaTeX → PDF conversion
• Output: PDF file path

4. Blob Storage Upload

Activity: BlobUploadActivity

Uploads the generated SSP document (PDF or JSON) to blob storage for long-term retention.

• Storage Key Format: ssp/{uuid}/{productId}.ssp.{ext}
• Content Types: application/pdf or application/json

5. Report Metadata Creation

Activity: CreateReportActivity

Creates a report entry in the database linking the SSP to the product.

• Report Type: ssp
• Reference: URI to the blob storage location

6. Archivista Storage with DSSE Signing

Activities: WitnessSignActivity + ArchivistaUploadActivity

This critical stage ensures the SSP is cryptographically signed and stored in Archivista for compliance attestation.

Process

1. DSSE Envelope Creation:

   • Creates in-toto subject with SHA256 digest of SSP JSON
   • Wraps SSP in witness predicate with type https://witness.dev/attestations/ssp/v0.1
   • Sets payload type to oscal/ssp to trigger Archivista's SSP parser

2. Signing:

   • Uses AWS KMS for cryptographic signing (default: local development key)
   • Can include timestamp authorities for non-repudiation

3. Archivista Upload:

   • Uploads signed DSSE envelope to Archivista
   • Archivista's storeSSP function parses the OSCAL structure
   • Stores SSP components in relational database tables

Data Transformation & Schema Compatibility

The Critical Transformation Layer

The workflow includes a transformation step between the raw OSCAL SSP and the Archivista-compatible format. This is handled in the generateArchivistaJSON function.

Why Transformation is Needed

1. Schema Mismatches: AgentFlow/OscalSynth may produce SSP JSON that doesn't 100% match Archivista's expected schema
2. Custom Extensions: Judge may add custom fields (like evidence tracking) not in standard OSCAL
3. Versioning Issues: Different OSCAL versions may have structural differences

Transformation Strategy

// Determine which data to use for Archivista storage
dataForArchivista := sspReportResult.SSPJSONData
if sspReportResult.ArchivistaCompatibleData != "" {
    // Use transformed data if available
    dataForArchivista = sspReportResult.ArchivistaCompatibleData
}

Handling Schema Mismatches

1. Preferred: Fix upstream in Archivista's schema parser
2. Alternative: Fix downstream in AgentFlow/OscalSynth output
3. Band-aid: Apply transformations during conversion

Archivista Storage Structure

When the DSSE envelope with payload type oscal/ssp is received, Archivista:

1. Parses SSP Structure: Unmarshals JSON into ssp.Root struct
2. Creates Database Records:
   • SSPMetadata: Title, version information
   • SystemSecurityPlan: Main SSP record with UUID
   • SSPSystemCharacteristics: System description and properties
   • SSPSystemImplementation: Components and their relationships
   • SSPComponent: Individual system components
   • SSPControlImplementation: How controls are satisfied

Error Handling & Resilience

Activity-Level Isolation

Each activity runs in the Dapr sidecar with:

• Automatic retry logic for transient failures
• Resource isolation preventing crashes from affecting judge-api
• Built-in timeout management

Non-Critical Failures

Some operations are marked as non-critical:

• PDF conversion failures don't fail the workflow
• Archivista storage failures are logged but don't block completion

Critical Failures

These will fail the workflow:

• Repository cloning failures (if repos provided)
• SSP generation failures
• Report metadata creation failures

Configuration & Environment

Key Environment Variables

• GATEWAY_URL: GraphQL gateway for queries
• ARCHIVISTA_URL: Archivista service endpoint
• SSP_ENABLED: Master switch for SSP functionality
• ANTHROPIC_API_KEY: Claude AI API key for AgentFlow
• WORKFLOW_SIGNER_KMS_URI: KMS key for DSSE signing

Helm Configuration

ssp:
  enabled: true
  agentflowModel: "claude-sonnet-4-20250514"

Storage Locations

Temporary Files

• Cloned repositories: /tmp/ssp-repos/
• SSP output: /tmp/ssp-output/
• Workspace: /tmp/ssp-workspace-{timestamp}/

Persistent Storage

• Blob Storage: s3://judge/ssp/{uuid}/{productId}.ssp.{ext}
• Archivista: DSSE envelopes with gitoid references
• Database: Report metadata linking to blob storage

Security Considerations

1. Cryptographic Signing: All SSPs are signed using AWS KMS keys
2. In-toto Attestations: SSPs wrapped in DSSE envelopes with proper subjects
3. SHA256 Digests: Content integrity verified through cryptographic hashes
4. Tenant Isolation: SSPs stored with tenant ID for multi-tenancy support

Monitoring & Debugging

Key Log Patterns

• [SSP Workflow {instanceID}]: Workflow-level operations
• SSPGeneratorActivity: AgentFlow execution details
• ArchivistaUploadActivity: Storage operations
• Failed to unmarshal ssp payload: Schema mismatch indicators

Common Issues & Solutions

1. "Failed to unmarshal ssp payload":

   • Check SSP JSON structure against ssp.Root schema
   • Verify OSCAL compliance
   • Review transformation logic

2. "PDF conversion failed":

   • Verify pandoc/pdflatex installation
   • Check markdown formatting

3. "Archivista upload failed":

   • Verify authentication token
   • Check network connectivity
   • Review DSSE envelope structure

Future Improvements

1. Extended Control Coverage: Currently focused on AC-2, expand to full control catalog
2. Enhanced Evidence Collection: Integrate with runtime attestations
3. Schema Versioning: Better handling of OSCAL version differences
4. Caching: Repository analysis results for performance
5. Incremental Updates: Support for updating existing SSPs

Technical Debt

• TODO: Remove ArchivistaCompatibleData workaround once schema alignment is complete
• TODO: Implement proper OSCAL version negotiation
• TODO: Add support for multiple control implementations beyond AC-2