clsandoval/00_COMPETITOR_OVERVIEW.md

Created January 16, 2026 08:06

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Select an option

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/clsandoval/9f9030fc195316fb45ee584ff50aec0d.js"></script>
Save clsandoval/9f9030fc195316fb45ee584ff50aec0d to your computer and use it in GitHub Desktop.

Download ZIP

Decision AI Analysis v3 - Competitor Overview, Pack Manifest Format, Artifact Output Patterns

Raw

00_COMPETITOR_OVERVIEW.md

Competitor Overview

Comprehensive analysis of 26 competitor repositories analyzed for Decision AI product positioning

Executive Summary

This document provides a structured overview of all competitors analyzed during our research phase. Each competitor is categorized by market segment, with detailed profiles including value propositions, target audiences, key features, and user journey diagrams.

Key Finding: The market is fragmented across multiple niches. No single competitor addresses our full vision of unified context across platforms + modular AI sessions (deployable Claude instances) + trust-focused data science. This represents our blue ocean opportunity.

CURRENT STATE vs FUTURE VISION

CRITICAL: This section clearly distinguishes what EXISTS today versus what is PLANNED for the future.

What EXISTS Today (January 2025)

Component	Status	Description
Discord Bot	IMPLEMENTED	Primary user interface
Workflow Executor	IMPLEMENTED	Claude API with workflow_tools
Fly.io Deployment	IMPLEMENTED	Dynamic machine creation via fly_app_tools
ACP Protocol	IMPLEMENTED	Inter-session communication via SSE
Session Templates	IMPLEMENTED	Supabase database records
Builder Claude	IMPLEMENTED	Containerization service with meta-skills

What is PLANNED (Future Vision)

Component	Status	Description
Decision Packs	PLANNED	GitHub repos as deployable units with pack.yaml manifests
Pack Registry	PLANNED	Searchable index of available packs
Pack Marketplace	PLANNED	Web UI for discovery and deployment
Voice Sessions	PLANNED	Hands-free Discord voice interaction
Memory Layer	PLANNED	Persistent cross-session memory

Builder as Claude Factory: Our Key Differentiator

What makes Decision AI unique: Builder Claude doesn't just containerize code—it constructs entire intelligent environments.

┌─────────────────────────────────────────────────────────────────────────────┐
│                    BUILDER AS CLAUDE FACTORY (CURRENT)                       │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   INPUT: User's code repository                                              │
│          (any framework: Marimo, Streamlit, FastAPI, etc.)                  │
│                                                                             │
│   BUILDER CLAUDE CONSTRUCTS:                                                 │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │  1. Docker image with user's code                                    │   │
│   │  2. .claude/ directory with:                                         │   │
│   │     ├── CLAUDE.md (execution rules + purpose)                        │   │
│   │     └── skills/ (generated from repo analysis)                       │   │
│   │  3. ACP server for communication                                     │   │
│   │  4. GitHub session repo (source of truth)                            │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│   OUTPUT: Complete Claude Code environment deployed on Fly.io               │
│                                                                             │
│   KEY INSIGHT: Each build = complete Claude Code environment                │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

User Intent Priority (Decision Flow)

Does repo have .claude/?
├── YES → Inherit/merge (user customizations WIN)
│         - Preserve existing skills, hooks, CLAUDE.md
│         - Merge base execution rules
│         - Add missing infrastructure skills
│
└── NO → Did user specify skill preferences?
    ├── YES → Follow their guidance exactly
    │
    └── NO → Generate from scratch using meta-skills:
             a. Analyze repo (dependencies, code patterns, purpose)
             b. Detect framework (Marimo, Streamlit, FastAPI, etc.)
             c. Generate domain-specific skills
             d. Create CLAUDE.md with execution rules

Git as Source of Truth

Each session gets its own GitHub repository. This replaces container-as-artifact with git as the unit of reproducibility.

┌─────────────────────────────────────────────────────────────────────────────┐
│                    GIT AS SOURCE OF TRUTH (CURRENT)                          │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   Session Repo Pattern:                                                      │
│   • New GitHub repo: github.com/org/session-mmm-{hex}                       │
│   • All changes tracked: git add -A && git commit after work                │
│   • Versioning via tags: git tag "template/my-analysis-v1"                  │
│   • Full history: Browsable on GitHub, diffable                             │
│                                                                             │
│   Benefits:                                                                  │
│   • Transparency: Readable source code, not opaque binary images            │
│   • Reproducibility: git clone --branch tag = exact state                   │
│   • Shareability: Link to GitHub repo = shareable, forkable                 │
│   • Auditable: Every change logged with timestamps, diffs                   │
│                                                                             │
│   Build Result Format:                                                       │
│   {                                                                          │
│     "status": "complete",                                                   │
│     "app_name": "template-my-thing",                                        │
│     "image_ref": "registry.fly.io/template-my-thing:v1",                    │
│     "git_repo": "github.com/org/template-my-thing",                         │
│     "git_ref": "snapshot/v1"                                                │
│   }                                                                          │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Current Decision Orchestrator Architecture

What ACTUALLY exists today:

┌─────────────────────────────────────────────────────────────────────────────┐
│                    DECISION ORCHESTRATOR - CURRENT STATE                     │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   Discord Bot (Primary Interface)                                            │
│   └── User sends message                                                     │
│       └── Workflow Executor runs with Claude API                             │
│           └── Uses workflow_tools:                                           │
│               ├── CUSTOM_FLY_LAUNCH_BUILDER (spawn builder session)         │
│               ├── CUSTOM_FLY_LAUNCH_SESSION (launch from template)          │
│               ├── CUSTOM_ACP_SEND_MESSAGE (talk to sessions)                │
│               ├── CUSTOM_FLY_STOP_SESSION (destroy session)                 │
│               ├── CUSTOM_FLY_GET_SESSION_STATUS (check health)              │
│               ├── CUSTOM_FLY_LIST_SESSIONS (inventory)                      │
│               ├── CUSTOM_FLY_LIST_TEMPLATES (available templates)           │
│               ├── CUSTOM_FLY_SAVE_TEMPLATE (persist as reusable)            │
│               └── human_interaction_tools (wait_for_human_decision)         │
│                                                                             │
│   Builder Sessions (Ephemeral)                                               │
│   └── Name pattern: mmm-builder-{hex}                                        │
│       └── Lifetime: ~5-30 minutes                                            │
│       └── Purpose: Clone, analyze, generate .claude/, build, deploy          │
│       └── Death: After deployment (cleanup via CUSTOM_FLY_STOP_SESSION)     │
│                                                                             │
│   User Work Sessions (Persistent)                                            │
│   └── Name pattern: mmm-{hex}                                                │
│       └── Lifetime: User determines (hours to days)                          │
│       └── Purpose: Interactive analysis, experimentation                     │
│       └── Environment: Framework runtime + Claude Agent SDK/ACP             │
│                                                                             │
│   Templates (Supabase: session_templates table)                              │
│   └── Columns: name, system_prompt, tools_config, mcp_config, metadata      │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Decision AI-Specific Tool Inventory (CURRENT)

Builder Lifecycle Tools

Tool	Purpose	Returns
`CUSTOM_FLY_LAUNCH_BUILDER`	Spawn ephemeral builder session	app_name, acp_url, build_id
`CUSTOM_ACP_SEND_MESSAGE`	Send instructions to builder/session	Claude's response
`CUSTOM_FLY_STOP_SESSION`	Destroy session	success/error

Session Lifecycle Tools

Tool	Purpose	Returns
`CUSTOM_FLY_LAUNCH_SESSION`	Launch from pre-saved template	app_name, urls, session_repo
`CUSTOM_FLY_GET_SESSION_STATUS`	Check session health	status, urls, session_repo
`CUSTOM_FLY_LIST_SESSIONS`	Inventory active sessions	session list

Template Management Tools

Tool	Purpose	Returns
`CUSTOM_FLY_LIST_TEMPLATES`	Show available templates	template list
`CUSTOM_FLY_SAVE_TEMPLATE`	Persist build as reusable template	success + metadata
`CUSTOM_FLY_DELETE_TEMPLATE`	Remove saved template	success/error

What These Tools DON'T Do:

No predefined Dockerfile schemas—Builder Claude generates them
No rigid skill parameters—Claude-in-Builder decides what to use
No forced framework choices—Claude analyzes and detects

Market Landscape Map

┌─────────────────────────────────────────────────────────────────────────────────────────────┐
│                              COMPETITIVE LANDSCAPE 2025                                      │
├─────────────────────────────────────────────────────────────────────────────────────────────┤
│                                                                                              │
│  ORCHESTRATION & FRAMEWORKS                    OBSERVABILITY & EVALUATION                   │
│  ┌──────────────┐  ┌──────────────┐           ┌──────────────┐  ┌──────────────┐           │
│  │   CrewAI     │  │  LangGraph   │           │   Langfuse   │  │  Braintrust  │           │
│  │  Multi-Agent │  │  Stateful    │           │  LLM Tracing │  │   AI Evals   │           │
│  │  Tool Mgmt   │  │  Workflows   │           │  & Prompts   │  │  Regression  │           │
│  └──────────────┘  └──────────────┘           └──────────────┘  └──────────────┘           │
│                                                                                              │
│  PLATFORMS & BUILDERS                          MEMORY & KNOWLEDGE                           │
│  ┌──────────────┐  ┌──────────────┐           ┌──────────────┐  ┌──────────────┐           │
│  │   LLMStack   │  │  VoltAgent   │           │  ChatMemory  │  │    Glean     │           │
│  │   No-Code    │  │  TypeScript  │           │  4-Tier Mem  │  │  Enterprise  │           │
│  │   Builder    │  │  Full-Stack  │           │  Hierarchy   │  │  Knowledge   │           │
│  └──────────────┘  └──────────────┘           └──────────────┘  └──────────────┘           │
│                                                                                              │
│  CHAT & MULTI-PLATFORM                         MARKETING ANALYTICS (MMM)                    │
│  ┌──────────────┐  ┌──────────────┐           ┌──────────────┐  ┌──────────────┐           │
│  │   Dust.tt    │  │   Clawdbot   │           │   PyMC-      │  │   Google     │           │
│  │  Slack AI    │  │  8-Platform  │           │  Marketing   │  │   LWMMM      │           │
│  │  Assistants  │  │  Personal AI │           │  Bayesian    │  │              │           │
│  └──────────────┘  └──────────────┘           └──────────────┘  └──────────────┘           │
│                                                                                              │
│  TEMPLATE & DEPLOYMENT PLATFORMS (Inspiration for Future Pack System)                        │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐                     │
│  │   Replit     │  │   Railway    │  │   Render     │  │   Vercel     │                     │
│  │  Templates   │  │  Templates   │  │  Blueprints  │  │  Templates   │                     │
│  │  Full Proj   │  │  One-Click   │  │  IaC Deploy  │  │  Frontend    │                     │
│  └──────────────┘  └──────────────┘  └──────────────┘  └──────────────┘                     │
│                                                                                              │
└─────────────────────────────────────────────────────────────────────────────────────────────┘

Gap Analysis: Our Opportunity

What NO Competitor Does

Gap	Description	Our Approach
Builder as Claude Factory	No one generates complete Claude environments from repo analysis	Meta-skills construct .claude/ dynamically
Git as Source of Truth	Competitors use opaque container images	Session repos track all changes via git
Unified Context	No one offers context continuity across Discord/Slack/Teams/CLI	Interface Primitives + Shared Memory (PLANNED)
Domain-Specific Evals	No one has insight recovery benchmarks for analytics	MMM Insight Recovery Experiments
Trust-First Data Science	No one combines Bayesian causal + LLM + governance	Trust Differentiators
ACP Protocol	No standard for inter-Claude communication	Our implemented protocol

What We Should Adopt

Pattern	From	Why
ToolCollection	CrewAI	Best-in-class tool management
Thread-as-Boundary	Dust.tt	Essential for chat context
Statistical Evals	Braintrust	Right approach to AI testing
4-Tier Memory	ChatMemory	Complete hierarchy
Manifest Format	Awesome Skills	Proven skill structure
Streaming Progress	Replit	Great deploy UX
Bayesian Foundation	PyMC-Marketing	Trust through uncertainty

Architectural Philosophy: Embodied vs Puppeteer

Repo2Run Pattern (Puppeteer)

🧠 (External LLM) ─────► 📦 (Dumb container)
- LLM remote-controls container
- Container has no intelligence
- Intelligence only during build time
- After build: container is static code

Our Approach (Embodied)

┌──────────────────────┐
│ 🧠 (Claude INSIDE)   │
│ 📦 (Container/body)  │
└──────────────────────┘
- Claude inhabits the container
- Container is Claude's body
- Intelligence at runtime
- Interactive collaboration with user

Trade-offs

Aspect	Repo2Run	Decision AI
External rollback	Excellent	Requires orchestrator
Deterministic outputs	Yes	No (but adaptive)
Runtime adaptation	No	Yes
Domain expertise	None	Skills loaded in session
User collaboration	None	Interactive
Multi-repo composition	Hard	Flexible merging
Framework support	Python-only	Framework-agnostic

Complete Competitor List

#	Competitor	Category	What They Do (One-Liner)
1	CrewAI	Framework	Multi-agent orchestration with role-based collaboration and tool collections
2	LangGraph	Framework	Stateful graph-based workflows for LLM applications
3	Swarm	Framework	Lightweight multi-agent handoffs (educational, by OpenAI)
4	Claude-Flow	Framework	Enterprise multi-agent swarms with neural learning and MCP
5	AutoGen	Framework	Microsoft's multi-agent conversation framework
6	Pydantic-AI	Framework	Type-safe Python agents with structured outputs
7	VoltAgent	Platform	TypeScript full-stack agent framework with VoltOps observability
8	LLMStack	Platform	No-code visual builder for AI agents and workflows
9	BotSharp	Platform	.NET/C# agent framework with plugin architecture
10	Composio	SDK	500+ app integrations for AI agents
11	Langfuse	Observability	LLM tracing, prompt management, and evaluation
12	Braintrust	Observability	Statistical AI evaluation with regression detection
13	AgentOps	Observability	Agent session replay and cost tracking
14	ChatMemory	Memory	4-tier hierarchical memory for AI assistants
15	Glean	Knowledge	Enterprise permission-aware knowledge search
16	Dust.tt	Chat	Thread-aware Slack AI assistants
17	Clawdbot	Chat	8-platform personal AI assistant (desktop)
18	KIRA	Chat	Privacy-first desktop AI coworker
19	Runbear	Chat	Tiered Slack/Teams bot platform
20	Awesome Claude Skills	Skills	Open-source skill manifest patterns
21	PyMC-Marketing	MMM	Bayesian causal marketing mix modeling
22	Meta Robyn	MMM	Automated MMM with Pareto optimization
23	Replit Templates	Templates	Full project templates with instant deployment
24	Railway Templates	Templates	One-click deployable app templates
25	Render Blueprints	Templates	Infrastructure-as-code deployment templates
26	Vercel Templates	Templates	Frontend/fullstack starter templates

Master Comparison Table

Competitor	Category	Primary Language	Deploy Model	Key Differentiator	Pricing
CrewAI	Framework	Python	Library	Role-based multi-agent + ToolCollection	OSS
LangGraph	Framework	Python	Library	Stateful graphs + checkpointing	OSS + Cloud
Swarm	Framework	Python	Library	Minimal primitives (educational)	OSS
Claude-Flow	Framework	TypeScript	Enterprise	54+ agents + neural learning	OSS
AutoGen	Framework	Python	Library	Conversational multi-agent	OSS
Pydantic-AI	Framework	Python	Library	Type safety + structured outputs	OSS
VoltAgent	Platform	TypeScript	Hybrid	Full-stack + VoltOps console	OSS + Cloud
LLMStack	Platform	Python	Self-hosted	No-code visual builder	OSS + Cloud
BotSharp	Platform	C#	Enterprise	.NET ecosystem + plugins	OSS
Composio	SDK	TypeScript	Multi-framework	500+ integrations	Freemium
Langfuse	Observability	TypeScript	Self-hosted	Tracing + prompt management	OSS + Cloud
Braintrust	Observability	Python	Cloud	Statistical evals + regression	Freemium
AgentOps	Observability	Python	Cloud	Session replay + cost tracking	Freemium
ChatMemory	Memory	Python	Library	4-tier hierarchy + pgvector	OSS
Glean	Knowledge	-	Enterprise	Permission-aware search	Enterprise
Dust.tt	Chat	-	Cloud	Thread-aware Slack AI	Tiered
KIRA	Chat	Python	Desktop	Privacy-first, local-only	OSS
Clawdbot	Chat	TypeScript	Desktop	8-platform personal AI	OSS
Runbear	Chat	-	Cloud	Tiered bot platform	Tiered
Awesome Skills	Skills	-	-	Manifest format pattern	OSS
PyMC-Marketing	MMM	Python	Library	Bayesian causal inference	OSS
Robyn	MMM	R	Library	Automated Pareto optimization	OSS
LightweightMMM	MMM	Python	Library	Google's Bayesian MMM	OSS
Nielsen	MMM	-	Service	Industry standard	Enterprise
Replit Agent	Deploy	-	Cloud	Zero-friction deploy	Freemium
Hex AI	Artifacts	-	Cloud	Professional notebooks	Tiered
v0.dev	Artifacts	-	Cloud	AI-generated UI preview	Freemium

Document generated for Decision AI competitive analysis - January 2025 26 competitors analyzed across 9 categories Updated to reflect actual current state + Builder as Claude Factory architecture

Raw

01_PACK_MANIFEST_FORMAT.md

Session Templates & Future Pack Architecture

This document describes the CURRENT template system in Decision Orchestrator and the FUTURE VISION for "Decision Packs" as deployable repositories.

CRITICAL: Current State vs. Future Vision

Aspect	CURRENT (Implemented)	FUTURE (Planned)
Storage	Supabase database records	GitHub repositories
Format	JSON in database columns	pack.yaml manifests + code
Deployment	Dynamic Fly.io machines	Pre-built container images
Discovery	Query by name	Trigger-based matching
Versioning	metadata.version field	Git tags
Status	IMPLEMENTED	VISION - NOT YET BUILT

CURRENT STATE: Session Templates (IMPLEMENTED)

Template Structure

-- Current schema (from supabase/migrations/)
CREATE TABLE session_templates (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name TEXT UNIQUE NOT NULL,
    system_prompt TEXT NOT NULL,
    tools_config JSONB DEFAULT '{}',
    mcp_config JSONB DEFAULT '{}',
    metadata JSONB DEFAULT '{}',
    is_default BOOLEAN DEFAULT FALSE,
    created_at TIMESTAMPTZ DEFAULT now(),
    updated_at TIMESTAMPTZ DEFAULT now()
);

How Templates Work Today

┌─────────────────────────────────────────────────────────────────────────────┐
│                    CURRENT TEMPLATE SYSTEM (IMPLEMENTED)                     │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   Supabase: session_templates table                                          │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │ id          │ UUID (primary key)                                     │   │
│   │ name        │ "marketing-analyst"                                    │   │
│   │ system_prompt│ "You are a marketing analysis assistant..."          │   │
│   │ tools_config│ {"allowed_tools": ["search", "calculate"]}            │   │
│   │ mcp_config  │ {"servers": [...]}                                     │   │
│   │ metadata    │ {"version": "1.0", "author": "team"}                  │   │
│   │ is_default  │ false                                                  │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│   How it's used:                                                             │
│   1. User requests session via Discord                                       │
│   2. Workflow executor queries session_templates by name                     │
│   3. System prompt and configs are loaded                                   │
│   4. Fly.io machine is created with this configuration                       │
│   5. Claude instance runs with the template's instructions                   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Example Template Record

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "marketing-analyst",
  "system_prompt": "You are a marketing analytics assistant specializing in budget optimization and ROI analysis...",
  "tools_config": {
    "allowed_tools": ["search_web", "calculate", "create_chart"],
    "restrictions": []
  },
  "mcp_config": {
    "servers": []
  },
  "metadata": {
    "version": "1.0",
    "category": "analytics",
    "author": "pymc-labs"
  },
  "is_default": false
}

CURRENT STATE: Builder Claude's Role (IMPLEMENTED)

Builder Claude doesn't just use templates—it generates complete Claude environments from repository analysis.

Builder as Claude Factory

┌─────────────────────────────────────────────────────────────────────────────┐
│                    BUILDER AS CLAUDE FACTORY (CURRENT)                       │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   When Builder receives a repo URL:                                          │
│                                                                             │
│   1. CLONE repository to /workspace/app                                      │
│                                                                             │
│   2. ANALYZE the repo:                                                       │
│      ├── Detect framework (Marimo, Streamlit, FastAPI, etc.)                │
│      ├── Check for existing .claude/ directory                              │
│      └── Identify dependencies and purpose                                  │
│                                                                             │
│   3. CONSTRUCT production .claude/ using META-SKILLS:                        │
│      ├── claude-factory.md (master skill)                                   │
│      ├── skill-creation.md (generate domain-specific skills)                │
│      ├── claude-md-templates.md (CLAUDE.md generation)                      │
│      └── merge-strategy.md (if repo has existing .claude/)                  │
│                                                                             │
│   4. GENERATE Dockerfile using framework skills:                             │
│      └── dockerfile-gen.md (framework-specific patterns)                    │
│                                                                             │
│   5. SET UP ACP server (always required)                                     │
│                                                                             │
│   6. DEPLOY via Fly: fly deploy --remote-only                               │
│                                                                             │
│   7. CREATE GitHub session repo as source of truth                           │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Meta-Skills (What Builder Uses)

Meta-Skill	Purpose	Location
claude-factory	Orchestrate entire .claude/ construction	`.claude/skills/meta/`
skill-creation	Generate new skills from repo analysis	`.claude/skills/meta/`
claude-md-templates	Templates for CLAUDE.md generation	`.claude/skills/meta/`
merge-strategy	Combine repo's .claude/ with base	`.claude/skills/meta/`
dockerfile-gen	Framework-specific Dockerfile patterns	`.claude/skills/building/`

Production Image Structure (What Builder Creates)

production-image/
├── app/                    # User's code
├── acp-server/             # Claude Agent SDK ACP server (REQUIRED)
│   ├── server.py
│   ├── tools/
│   └── pyproject.toml
└── .claude/
    ├── CLAUDE.md           # Execution rules + purpose (REQUIRED)
    └── skills/             # Generated domain-specific skills (optional)

Git as Source of Truth (CURRENT)

Each session gets its own GitHub repository. This is IMPLEMENTED, not future vision.

Session Repo Pattern

┌─────────────────────────────────────────────────────────────────────────────┐
│                    GIT AS SOURCE OF TRUTH (IMPLEMENTED)                      │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   For each snapshot/template, Builder CREATES a new GitHub repo:            │
│                                                                             │
│   1. gh repo create org/template-{name} --private                           │
│   2. Push the snapshot to that repo                                         │
│   3. Return the repo URL + app name to Orchestrator                         │
│                                                                             │
│   Build Result Format:                                                       │
│   {                                                                          │
│     "status": "complete",                                                   │
│     "app_name": "template-my-thing",                                        │
│     "image_ref": "registry.fly.io/template-my-thing:v1",                    │
│     "git_repo": "github.com/org/template-my-thing",                         │
│     "git_ref": "snapshot/v1"                                                │
│   }                                                                          │
│                                                                             │
│   Orchestrator handles saving metadata to Supabase.                          │
│   Git repo IS the source of truth.                                          │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Multi-Repo Sessions (Combined State)

When user clones repos A, B, C and tinkers:

Session workspace has A/, B/, C/ directories (each with .git)
On snapshot: Remove nested .git directories (flatten)
Initialize single root .git (new Repo D)
Snapshot manifest records provenance (where each part came from)
Dockerfile becomes simple COPY . . (not multiple clones)
Result: Combined state becomes the template source

Configuration Spectrum (CURRENT)

The system slides along a spectrum based on use case:

HARDCODED ◄─────────────────────────────────────────► DYNAMIC
(batch)                                              (R&D)

┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐
│ Static   │  │Template  │  │ Custom   │  │ Blank    │
│ Script   │  │ Launch   │  │ Build    │  │ Slate    │
└──────────┘  └──────────┘  └──────────┘  └──────────┘

Position	Type	Builder	Claude	Example
Far left	Static	Not used	Not needed	Pre-built script, just execute
Left-center	Template	Runs once	In session	Pre-saved image, launch and work
Right-center	Custom Build	Runs now	In builder & session	"Build this repo, then I'll analyze it"
Far right	Blank Slate	Interactive	Full	"Install Python, let me tinker"

Key: Same architecture at all points, different configuration. Not different systems.

FUTURE VISION: Decision Packs (NOT YET IMPLEMENTED)

NOTE: Everything below this line describes a FUTURE architecture that does not yet exist in the codebase.

The Pack Vision

A Decision Pack would be a complete, deployable repository containing:

Dockerfile for containerization
All dependencies and tooling
Agent/skill logic and instructions
Manifest/configuration
Ready to deploy on Fly.io

┌─────────────────────────────────────────────────────────────────────────────┐
│                    FUTURE: DECISION PACK = DEPLOYABLE REPOSITORY             │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  Current (Template):                    Future Vision (Pack):               │
│  ──────────────────                     ───────────────────────             │
│                                                                             │
│  ┌─────────────────────┐               ┌─────────────────────────────────┐ │
│  │ Supabase record:    │               │ github.com/org/pack-mmm         │ │
│  │ - name              │               │                                  │ │
│  │ - system_prompt     │               │ ├── Dockerfile        ← BUILD   │ │
│  │ - tools_config      │               │ ├── fly.toml          ← DEPLOY  │ │
│  │ - mcp_config        │               │ ├── requirements.txt  ← DEPS    │ │
│  └─────────────────────┘               │ ├── pack.yaml         ← MANIFEST│ │
│                                        │ ├── .claude/                    │ │
│  Just database config.                 │ │   ├── CLAUDE.md     ← BRAIN   │ │
│  Session created                       │ │   └── skills/                 │ │
│  dynamically.                          │ ├── acp-server/       ← COMMS   │ │
│                                        │ └── app/              ← CODE    │ │
│                                        └─────────────────────────────────┘ │
│                                                                             │
│                                        Complete. Self-contained. Deployable.│
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Proposed Pack Manifest (pack.yaml) - FUTURE

# pack.yaml - Pack Manifest (PROPOSED - NOT YET IMPLEMENTED)
name: marketing-mmm
version: 2.1.0
description: Marketing Mix Modeling with Bayesian inference and budget optimization

# Identity
author: decision-ai
license: Apache-2.0
repository: github.com/decision-ai/pack-mmm

# Discovery triggers - WHEN should this pack be suggested?
triggers:
  keywords:
    - "marketing.*budget"
    - "ROI|ROAS"
    - "attribution"
    - "media mix"
  file_patterns:
    - "*.csv"           # Marketing data files
    - "*.parquet"
  contexts:
    - analytics
    - marketing
    - planning

# Capabilities - WHAT does this pack provide?
provides:
  - bayesian-inference
  - budget-optimization
  - uncertainty-quantification
  - channel-attribution

# Dependencies - WHAT other packs does this need?
requires:
  - name: data-loader
    version: ">=1.0"
  - name: visualization
    version: ">=2.0"
    optional: true

# Runtime requirements
runtime:
  base_image: python:3.11-slim
  memory: 2gb
  cpu: shared-2x
  gpu: false

# Environment variables needed
environment:
  required:
    - ANTHROPIC_API_KEY
  optional:
    - DATABASE_URL
    - REDIS_URL

# Health check
health:
  path: /healthz
  interval: 30s
  timeout: 5s

Roadmap: From Current to Future

Phase 0: Current State (IMPLEMENTED)

Supabase session_templates table
Dynamic Fly.io machine creation
System prompts loaded from database
ACP communication between sessions
Builder Claude with meta-skills
Git repos as source of truth for sessions

Phase 1: Enhanced Templates (PLANNED)

Add git_repo_url field to session_templates
If git_url present, clone and extract .claude/
Template versioning via git refs

Phase 2: Pack System (FUTURE VISION)

Git repositories as deployable units
pack.yaml manifest for discovery
Container registry for pre-built images
Pack marketplace

Migration Path

┌─────────────────────────────────────────────────────────────────────────────┐
│                        EVOLUTION PATH                                        │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  TODAY (Current)             NEAR-TERM                    FUTURE            │
│  ──────────────              ─────────                    ──────            │
│                                                                             │
│  session_templates           session_templates            Pack repos        │
│  (Supabase)                  + git_repo_url field         (GitHub)          │
│       │                          │                           │              │
│       ▼                          ▼                           ▼              │
│  system_prompt               system_prompt OR            .claude/CLAUDE.md  │
│  in database                 .claude/ from repo          in repository      │
│       │                          │                           │              │
│       ▼                          ▼                           ▼              │
│  Dynamic Fly.io            Clone repo →                Pre-built images     │
│  machine creation          Build → Deploy              from registry        │
│                                                                             │
│  Builder Claude            Builder Claude              Pack discovery       │
│  generates .claude/        respects repo .claude/      + pre-built         │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Summary

Aspect	Current (Templates)	Future (Packs)
Storage	Supabase database + Git session repos	GitHub pack repositories
Format	JSON records + generated .claude/	YAML manifests + code
Deployment	Builder generates, Fly deploys	Pre-built containers
Discovery	Query by name	Trigger-based matching
Versioning	Git tags on session repos	Git tags on pack repos
Distribution	Supabase + GitHub	Container registry
Status	IMPLEMENTED	VISION

This document reflects the actual state of Decision Orchestrator as of January 2025. The pack system described is a future vision based on roadmap documents in the codebase.

Raw

02_ARTIFACT_OUTPUT_PATTERNS.md

Artifact Output Patterns

How AI agents produce, structure, and present actionable outputs to users

The Core Insight

An artifact is a discrete, actionable unit of agent output. Unlike conversational text, artifacts are:

Structured and machine-parseable
Designed for a specific next action
Often rendered specially in UI
Versioned and comparable

The best agents don't just respond—they produce artifacts that enable action.

Artifact Taxonomy

┌─────────────────────────────────────────────────────────────────────────────┐
│                          ARTIFACT TYPES                                      │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  CODE ARTIFACTS                         DOCUMENT ARTIFACTS                   │
│  ──────────────                         ──────────────────                   │
│  ┌─────────────────┐                    ┌─────────────────┐                 │
│  │ Type: code      │                    │ Type: document  │                 │
│  │ Lang: python    │                    │ Format: markdown│                 │
│  │ Executable: yes │                    │ Sections: yes   │                 │
│  │ Testable: yes   │                    │ Exportable: yes │                 │
│  └─────────────────┘                    └─────────────────┘                 │
│                                                                             │
│  VISUAL ARTIFACTS                       DATA ARTIFACTS                       │
│  ────────────────                       ──────────────                       │
│  ┌─────────────────┐                    ┌─────────────────┐                 │
│  │ Type: visual    │                    │ Type: data      │                 │
│  │ Format: SVG     │                    │ Format: JSON    │                 │
│  │ Renderable: yes │                    │ Schema: defined │                 │
│  │ Interactive: ?  │                    │ Queryable: yes  │                 │
│  └─────────────────┘                    └─────────────────┘                 │
│                                                                             │
│  COMPOSITE ARTIFACTS                    ACTION ARTIFACTS                     │
│  ───────────────────                    ────────────────                     │
│  ┌─────────────────┐                    ┌─────────────────┐                 │
│  │ Type: report    │                    │ Type: action    │                 │
│  │ Contains:       │                    │ Executable: yes │                 │
│  │  - text         │                    │ Reversible: ?   │                 │
│  │  - charts       │                    │ Confirmable: yes│                 │
│  │  - tables       │                    └─────────────────┘                 │
│  │  - code         │                                                        │
│  └─────────────────┘                                                        │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

The Artifact Lifecycle

┌─────────────────────────────────────────────────────────────────────────────┐
│                      ARTIFACT LIFECYCLE                                      │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  ┌────────────┐    ┌────────────┐    ┌────────────┐    ┌────────────┐      │
│  │  GENERATE  │───►│  VALIDATE  │───►│  PRESENT   │───►│   ACTION   │      │
│  └────────────┘    └────────────┘    └────────────┘    └────────────┘      │
│        │                 │                 │                 │              │
│        ▼                 ▼                 ▼                 ▼              │
│  Agent creates     Schema check      UI renders       User acts on         │
│  structured        Type validation   Special display  artifact             │
│  output            Completeness      Action buttons   (copy, run, etc.)    │
│                                                                             │
│  ════════════════════════════════════════════════════════════════════════  │
│                                                                             │
│                         ITERATION LOOP                                       │
│                                                                             │
│     ┌─────────────────────────────────────────────────────────┐            │
│     │                                                          │            │
│     │    User: "Make the chart blue"                          │            │
│     │         │                                                │            │
│     │         ▼                                                │            │
│     │    ┌─────────┐     ┌─────────┐     ┌─────────┐         │            │
│     │    │Artifact │────►│ Update  │────►│Artifact │         │            │
│     │    │  v1     │     │ Command │     │  v2     │         │            │
│     │    └─────────┘     └─────────┘     └─────────┘         │            │
│     │                                         │               │            │
│     │                                         │               │            │
│     │    Both versions preserved for comparison/rollback      │            │
│     │                                                          │            │
│     └─────────────────────────────────────────────────────────┘            │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Structured Output Schema Pattern

The key insight: artifacts need schemas for reliability.

┌─────────────────────────────────────────────────────────────────────────────┐
│                    SCHEMA-DRIVEN ARTIFACTS                                   │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  WITHOUT SCHEMA                          WITH SCHEMA                         │
│  ──────────────                          ───────────                         │
│                                                                             │
│  Agent output:                           Agent output:                       │
│  "The ROI is about 2.5x,                 {                                   │
│   give or take, and                        "artifact_type": "analysis",     │
│   you should probably                      "metrics": {                     │
│   increase digital spend"                    "roi": {                       │
│                                                "value": 2.5,               │
│  Problems:                                     "unit": "multiple",         │
│  • Can't extract metrics                       "confidence": 0.87          │
│  • Can't compare versions                    }                              │
│  • Can't feed to downstream                },                               │
│  • UI can't render specially               "recommendations": [             │
│                                              {                              │
│                                                "action": "increase_spend", │
│                                                "channel": "digital",       │
│                                                "amount_pct": 15            │
│                                              }                              │
│                                            ]                                │
│                                          }                                  │
│                                                                             │
│                                          Benefits:                          │
│                                          • Machine-parseable               │
│                                          • Validatable                     │
│                                          • Comparable                      │
│                                          • UI can render richly            │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Decision AI Artifacts: Build Results

In Decision AI, build results are a key artifact type. Builder Claude produces structured build results:

┌─────────────────────────────────────────────────────────────────────────────┐
│                    BUILD RESULT ARTIFACT (DECISION AI)                       │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  After Builder Claude completes a build:                                     │
│                                                                             │
│  ## Build Result                                                             │
│                                                                             │
│  | Field | Value |                                                           │
│  |-------|-------|                                                           │
│  | **App Name** | `mmm-session-abc123` |                                    │
│  | **App URL** | `https://mmm-session-abc123.fly.dev` |                     │
│  | **Image** | `registry.fly.io/mmm-session-abc123:deployment-xyz` |        │
│  | **Framework** | `marimo` |                                               │
│  | **Git Repo** | `github.com/org/session-mmm-abc123` |                     │
│  | **Git Ref** | `snapshot/v1` |                                            │
│                                                                             │
│  ### Services Config                                                         │
│  ```json                                                                     │
│  [                                                                           │
│    {"protocol": "tcp", "internal_port": 8080,                               │
│     "ports": [{"port": 443, "handlers": ["tls", "http"]}]},                 │
│    {"protocol": "tcp", "internal_port": 3017,                               │
│     "ports": [{"port": 3017, "handlers": ["tls"]}]}                         │
│  ]                                                                           │
│  ```                                                                         │
│                                                                             │
│  ### Save as Template                                                        │
│  CUSTOM_FLY_SAVE_TEMPLATE(                                                  │
│    slug="mmm-analysis",                                                     │
│    name="MMM Analysis Session",                                             │
│    image_ref="registry.fly.io/mmm-session-abc123:deployment-xyz",           │
│    services=[...],                                                          │
│    framework="marimo",                                                      │
│    description="Marketing mix modeling with PyMC"                           │
│  )                                                                           │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Artifact Presentation Patterns

How artifacts are displayed to users:

┌─────────────────────────────────────────────────────────────────────────────┐
│                    PRESENTATION PATTERNS                                     │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  PATTERN 1: INLINE EXPANSION                                                 │
│  ───────────────────────────                                                 │
│                                                                             │
│     Conversation:                                                            │
│     ┌────────────────────────────────────────┐                              │
│     │ User: "Generate a budget allocation"   │                              │
│     │                                        │                              │
│     │ Agent: "Here's the optimized budget:"  │                              │
│     │                                        │                              │
│     │ ┌────────────────────────────────────┐ │                              │
│     │ │ [ARTIFACT: Budget Table]           │ │ ◄── Rendered inline          │
│     │ │                                    │ │                              │
│     │ │ Channel  │ Current │ Recommended  │ │                              │
│     │ │ ─────────┼─────────┼────────────  │ │                              │
│     │ │ TV       │ $500K   │ $420K        │ │                              │
│     │ │ Digital  │ $300K   │ $380K        │ │                              │
│     │ │                                    │ │                              │
│     │ │ [Copy] [Export CSV] [Apply]       │ │ ◄── Action buttons           │
│     │ └────────────────────────────────────┘ │                              │
│     └────────────────────────────────────────┘                              │
│                                                                             │
│  PATTERN 2: SIDE PANEL                                                       │
│  ─────────────────────                                                       │
│                                                                             │
│     ┌─────────────────────┬─────────────────────────────────┐               │
│     │   Conversation      │     Artifact Panel               │               │
│     │                     │                                  │               │
│     │  User: "Show me     │  ┌───────────────────────────┐  │               │
│     │   the analysis"     │  │  [LIVE PREVIEW]           │  │               │
│     │                     │  │                           │  │               │
│     │  Agent: "I've       │  │   Chart renders here      │  │               │
│     │   created a viz..." │  │   Updates in real-time    │  │               │
│     │                     │  │   as conversation         │  │               │
│     │  User: "Make it     │  │   progresses              │  │               │
│     │   a bar chart"      │  │                           │  │               │
│     │                     │  └───────────────────────────┘  │               │
│     │  Agent: "Updated."  │                                  │               │
│     └─────────────────────┴─────────────────────────────────┘               │
│                                                                             │
│  PATTERN 3: ARTIFACT GALLERY                                                 │
│  ───────────────────────────                                                 │
│                                                                             │
│     Session artifacts:                                                       │
│     ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐                     │
│     │ Chart   │  │ Code    │  │ Report  │  │ Config  │                     │
│     │ v3      │  │ v2      │  │ v1      │  │ v1      │                     │
│     │ [view]  │  │ [view]  │  │ [view]  │  │ [view]  │                     │
│     └─────────┘  └─────────┘  └─────────┘  └─────────┘                     │
│         ▲                                                                   │
│         └── Click to expand, compare with previous versions                 │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Action Buttons Pattern

Every artifact should have clear next actions:

┌─────────────────────────────────────────────────────────────────────────────┐
│                      ACTION BUTTON TAXONOMY                                  │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  CODE ARTIFACTS                                                              │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │  [Copy to Clipboard] [Run in Sandbox] [Insert into File] [Explain] │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  DATA ARTIFACTS                                                              │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │  [Download CSV] [Download JSON] [Open in Spreadsheet] [Visualize]  │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  BUILD ARTIFACTS (Decision AI specific)                                      │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │  [Open Session] [Save as Template] [View Logs] [Stop Session]      │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  ANALYSIS ARTIFACTS                                                          │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │  [Export Report] [Schedule Re-run] [Share with Team] [Add to Dash] │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  RECOMMENDATION ARTIFACTS                                                    │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │  [Apply Changes] [Modify Parameters] [Reject] [Request Explanation]│   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Streaming & Progressive Rendering

For long-running artifact generation:

┌─────────────────────────────────────────────────────────────────────────────┐
│                   PROGRESSIVE ARTIFACT RENDERING                             │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  USER REQUEST: "Generate a comprehensive marketing report"                   │
│                                                                             │
│  TIME ────────────────────────────────────────────────────────────────►     │
│                                                                             │
│  t=0s     ┌─────────────────────────────────────────────┐                  │
│           │  [ARTIFACT: Report]                          │                  │
│           │                                              │                  │
│           │  Status: Generating...                       │                  │
│           │  ████░░░░░░░░░░░░░░░░░░░░  15%              │                  │
│           │                                              │                  │
│           │  Sections:                                   │                  │
│           │  ✓ Executive Summary (ready)                │                  │
│           │  ⟳ Channel Analysis (in progress)           │                  │
│           │  ○ Recommendations (pending)                 │                  │
│           │                                              │                  │
│           │  [View Available Sections]                   │                  │
│           └─────────────────────────────────────────────┘                  │
│                                                                             │
│  t=60s    ┌─────────────────────────────────────────────┐                  │
│           │  [ARTIFACT: Report]                          │                  │
│           │                                              │                  │
│           │  Status: Complete ✓                          │                  │
│           │  ████████████████████████  100%             │                  │
│           │                                              │                  │
│           │  All 5 sections ready                        │                  │
│           │                                              │                  │
│           │  [Download PDF] [Share] [Schedule Update]    │                  │
│           └─────────────────────────────────────────────┘                  │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Key Design Principles

Principle	Implementation
Schema-First	Define artifact structure before generation
Progressive Display	Show what's ready while generating rest
Clear Actions	Every artifact has obvious next steps
Versioning	Track changes, enable comparison & rollback
Export Flexibility	Multiple formats for different consumers
Inline Context	Keep artifacts near relevant conversation

Common Pitfalls

┌─────────────────────────────────────────────────────────────────────────────┐
│                       ARTIFACT ANTI-PATTERNS                                 │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  PROBLEM: Wall of Text                  SOLUTION: Structured Artifact        │
│  ─────────────────────                  ────────────────────────             │
│                                                                             │
│  "Here's your analysis:                 ┌──────────────────────────┐        │
│   The ROI for TV is 2.1x               │ [Analysis Artifact]      │        │
│   which is lower than digital          │                          │        │
│   at 3.2x but social is only           │ ROI Summary Table        │        │
│   1.8x so you should..."               │ Key Insight: Digital > TV│        │
│                                         │ [See Full Report]        │        │
│  Can't scan, extract, or act            └──────────────────────────┘        │
│                                         Scannable, actionable               │
│                                                                             │
│  PROBLEM: No Next Steps                 SOLUTION: Action Buttons             │
│  ──────────────────────                 ────────────────────────             │
│                                                                             │
│  "Here's the code."                     ┌──────────────────────────┐        │
│                                         │ [Code Artifact]          │        │
│  User: "Now what?"                      │                          │        │
│                                         │ [Run] [Copy] [Test]      │        │
│                                         └──────────────────────────┘        │
│                                                                             │
│  PROBLEM: Lost Artifacts                SOLUTION: Artifact Gallery           │
│  ───────────────────────                ────────────────────────             │
│                                                                             │
│  User: "Where's that chart             Session artifacts persisted          │
│   you made earlier?"                    and browsable in sidebar            │
│                                                                             │
│  Scroll, scroll, scroll...              One-click to find any output        │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Great artifacts are not just outputs—they're starting points for the next action. The difference between "here's some text" and "here's a structured artifact with clear next steps" is the difference between a chatbot and a productive AI assistant.