Agent Context Compaction Strategies

compaction-comparison.md

Context Window Management: Agent Comparison

Summary

Agent	Trigger Threshold	Compression Strategy	Auxiliary Model	Token Tracking
Phosphor	80% (configurable)	LLM summary; optional embedded dlgo model	Yes — main, small, or embedded (dlgo)	(chars + 3) / 4 estimate
Oh-My-Pi	80% (configurable)	3 strategies: context-full, handoff, snapcompact (bitmap images)	Optional handoff task	Model-aware token budget
Hermes-Agent	50% (configurable)	Pluggable ContextEngine (lossy LLM summary default)	Yes (configurable auxiliary model/provider)	API-reported + rough char estimate
OpenCode	Budget-based (window - buffer)	Checkpoint (summary + serialized context)	No	Local estimate vs. reserved headroom

Trigger Comparison

Dimension	Phosphor	Oh-My-Pi	Hermes	OpenCode
Primary trigger	Threshold: currentTokens / contextWindow >= threshold	6 scenarios: threshold, overflow, incomplete, idle, mid-turn, manual	Dual: agent compressor (50%) + gateway hygiene (85%)	Request size vs. contextWindow - buffer
Overflow recovery	AggressiveBuild retry on overflow	Context promotion first, then compaction; agent.continue() retry	Gateway safety net at 85%	One overflow-triggered compaction attempt
Pre-compaction pruning	PruneAndBuild(msgs, 200)	Tool-output pruning + useless-result elision	Phase 1: prune old tool results (>200 chars)	Deferred
Mid-turn check	None	Yes (midTurnEnabled)	None	None
Idle maintenance	None	Yes (runIdleCompaction())	None	None

Summary Generation

Dimension	Phosphor	Oh-My-Pi	Hermes	OpenCode
Summary method	LLM generates summary message	LLM summary or snapcompact bitmap archival	Auxiliary LLM with structured template	Hidden checkpoint with rolling summary
Summary template	templates/summary.md (state, files, context, strategy, next steps)	prompts/compaction-summary-context.md	Structured: Goal, Constraints, Progress, Decisions, Files, Next Steps	Structured rolling summary + token-bounded serialized context
Iterative update	Yes — previous summary prepended	New summary each compaction	Yes — previous summary passed for update	Yes — repeated compactions update prior summary
Summary model	Configurable: main, small, embedded	Main model or none (snapcompact)	Auxiliary model (configurable)	Not specified

Configuration Surface

# Phosphor
options:
  disable_auto_summarize: false
  summarize_threshold: 0.8
  summarize_model: main    # 'main', 'small', or 'embedded'
  summarize_prune_chars: 200
  summarize_prune_aggressive_chars: 50

# Oh-My-Pi
compaction:
  enabled: true
  strategy: context-full    # or: handoff, snapcompact
  threshold: 0.8
  midTurnEnabled: true
  autoContinue: true
  dropUseless: true
  snapcompact:
    shape: auto
    maxFrames: 80

# Hermes
compression:
  enabled: true
  threshold: 0.50
  target_ratio: 0.20
  protect_last_n: 20
  codex_gpt55_autoraise: true
auxiliary:
  compression:
    model: null
    provider: auto

# OpenCode
compaction:
  auto: true
  prune: false
  keep.tokens: 20000
  buffer: 4000

Key Differentiators

• Oh-My-Pi snapcompact: Unique bitmap-image archival — no API call needed. Model-aware frame shapes (Claude: 11on16-bw, Gemini: 8on22-bw). Cheapest compaction but requires vision-capable model.

• Hermes dual-layer: Gateway hygiene (85%) as safety net + agent compressor (50%) as primary. Anthropic prompt caching integrated with compaction.

• Hermes auxiliary model: Dedicated summary model keeps cost low and offloads the main model. Summary budget scales with content size.

• OpenCode checkpoint model: Durable full transcript preserved; compaction creates a hidden checkpoint that replaces the model-visible representation. Context Epoch baseline rebuilds after compaction.

• Phosphor: Configurable summary model (main, small, embedded dlgo). Supports iterative compaction (previous summary prepended), pre-compaction pruning (PruneAndBuild at 200-char limit), and overflow recovery with aggressive pruning retry. Threshold-based trigger remains at 80%.

What's Missing in Phosphor vs. Others

Feature	Available In
Multiple compaction strategies	Oh-My-Pi
Pre-compaction tool-output pruning	Phosphor, Oh-My-Pi, Hermes
Auxiliary/cheaper summary model	Phosphor, Hermes
Dual-layer safety net	Hermes
Context Epoch / checkpoint model	OpenCode
Mid-turn compaction	Oh-My-Pi
Idle maintenance	Oh-My-Pi
Bitmap archival (no API)	Oh-My-Pi
Anthropic prompt caching	Hermes
Iterative summary updates	Phosphor, Hermes, OpenCode

hermes-agent-compaction.md

Hermes-Agent Context Compression

Overview

Hermes uses a dual-layer compression system with a pluggable ContextEngine architecture. Compression is managed by the ContextCompressor (default engine) inside the agent loop, with a safety-net "gateway session hygiene" pass that fires at a higher threshold.

Architecture

┌───────────────────────────────────────────────┐
│  Gateway Session Hygiene (85%)                 │  Safety net for large sessions
│  ──────────────────────────────────────────────  (pre-agent, rough estimate)
│                                                 │
│  Agent ContextCompressor (50%, configurable)   │  Primary compression
│  ──────────────────────────────────────────────  (in-loop, real API tokens)
└───────────────────────────────────────────────┘

Gateway Layer (85%)

• Runs in gateway/run.py before the agent processes a message
• Uses rough character-based token estimation when API-reported tokens aren't available
• Fires only when len(history) >= 4 and compression is enabled
• Prevents API failures from sessions that escaped the agent's compressor

Agent Layer (50% default)

• Runs inside the agent tool loop via context_compressor.py
• Uses real API-reported token counts for accurate triggering
• Fully configurable threshold, tail budget, and auxiliary summary model

Pluggable Context Engine

The ContextEngine ABC (agent/context_engine.py) allows swapping the default ContextCompressor for alternative engines (e.g., "Lossless Context Management"). Selection via context.engine in config.yaml. Plugins are never auto-activated.

context:
  engine: "compressor"   # default — built-in lossy summarization
  engine: "lcm"          # plugin providing lossless context

Compression Algorithm (4 Phases)

Phase 1: Prune Old Tool Results

Cheap pre-pass: tool results >200 chars outside the protected tail are replaced with:

[Old tool output cleared to save context space]

Phase 2: Determine Boundaries

The message list is split into three regions:

[0..2]       ← Head: system prompt + first exchange (always protected)
[3..N]        ← Middle: turns to be summarized
[N..end]      ← Tail: protected by token budget or minimum message count

Tail protection is token-budget based: walks backward from end, accumulating tokens until threshold_tokens × target_ratio budget is exhausted. Falls back to protect_last_n count (default 20) if budget would protect fewer messages.

Boundaries are aligned with _align_boundary_backward() to keep tool_call/ tool_result pairs intact.

Phase 3: Generate Structured Summary

The middle turns are sent to the auxiliary LLM with a structured template:

## Goal
## Constraints & Preferences
## Progress
  ### Done
  ### In Progress
  ### Blocked
## Key Decisions
## Relevant Files
## Next Steps
## Critical Context

Summary token budget: content_tokens × 0.20, minimum 2,000, maximum min(context_length × 0.05, 12,000).

Iterative re-compression: Previous summary is passed to the LLM with instructions to update it, preserving information across multiple compactions.

Phase 4: Assemble Compressed Messages

Result: head messages + summary message + tail messages. Orphaned tool pairs are cleaned up (_sanitize_tool_pairs()).

Configuration

compression:
  enabled: true
  threshold: 0.50              # Triggers at 50% of context window
  target_ratio: 0.20           # Tail token budget = threshold_tokens × target_ratio
  protect_last_n: 20           # Minimum protected tail messages
  codex_gpt55_autoraise: true  # Raise trigger to 85% for gpt-5.5 on Codex OAuth

auxiliary:
  compression:
    model: null     # Override summary model (default: auto-detect)
    provider: auto  # Provider for summarization

prompt_caching:
  cache_ttl: "5m"   # TTL for Anthropic prompt caching

Codex gpt-5.5 Autoraise

The Codex OAuth route hard-caps gpt-5.5 at 272K context. Default 50% threshold would fire at ~136K — half the usable window. The autoraise bumps the trigger to 85% (~231K). Opt-out via hermes config set compression.codex_gpt55_autoraise false.

Anthropic Prompt Caching

Reduces input token costs by ~75% using Anthropic's cache_control breakpoints.

Strategy system_and_3: Uses all 4 allowed breakpoints:

1. System prompt (stable across all turns)
2. 3rd-to-last non-system message
3. 2nd-to-last non-system message
4. Last non-system message

Cache breaks on compaction but re-establishes within 1-2 turns.

Important Constraints

• Summary model context window: Must be ≥ main model's context length, or the summary call will fail with a context-length error.
• No intermediate pressure warnings: Removed because they caused models to "give up" prematurely on complex tasks.
• Orphaned tool pairs: Tool results referencing removed calls are removed; orphaned calls get stub results injected.

Before/After Example

Before (45 messages, ~95K tokens): Full conversation history with tool calls and results.

After (25 messages, ~45K tokens): System prompt with compaction note, a single structured summary message, then the protected tail messages.

oh-my-pi-compaction.md

Oh-My-Pi Compaction

Overview

Oh-My-Pi uses a multi-layered compaction system that replaces the conversation history with a dense summary before the context window fills. Unlike Phosphor's single LLM-summary approach, OMP supports three compaction strategies and a rich pipeline of pre-compaction optimizations.

Compaction Strategies

Strategy	Description
context-full (default)	Full LLM-based summarization: sends the truncated history to the model with a structured summary prompt
handoff	Delegates summarization to a separate "handoff" task; the new session receives a handoff message instead of a compaction entry
snapcompact	Local, deterministic archival: converts discarded history into pixel-font bitmap images (PNG frames) — no API call needed; requires a vision-capable model

Snapcompact Detail

Snapcompact is OMP's unique differentiator:

• Discarded messages are serialized, whitespace-collapsed, and rendered onto fixed-width PNG frames using public-domain pixel fonts
• Frame shape is model-aware: Claude gets 11on16-bw at 1932px (under Anthropic's 4,784 visual-token cap), Gemini gets 8on22-bw at 2048px (Gemini's 1,120-token image budget), OpenAI-compatible gets 8on22-bw at 1568px
• Archive persists as bounded source text + rendered frames in CompactionEntry.preserveData.snapcompact
• Later compactions re-render from the source text, not by carrying old PNGs forward
• maxFrames defaults to 80; large mids foveate internally (HQ/LQ/HQ)
• Configurable via snapcompact.shape (can force eval variants like 8x8r, doc-8on16-bw, etc.)

Triggers

OMP fires compaction in six scenarios:

#	Trigger	Reason	Will Retry
1	/compact [instructions]	manual	—
2	Context overflow error (429/400)	"overflow"	yes
3	Incomplete output (stopReason === "length")	"incomplete"	yes
4	Post-turn threshold exceeded	"threshold"	no (auto-continue optional)
5	Mid-turn threshold (before next request)	"threshold"	no
6	Idle maintenance	"idle"	—

Overflow & Incomplete Recovery

• Overflow recovery: Removes the failing assistant error message, tries context promotion (switch to a larger model) first. If promotion fails and compaction is enabled, runs context-full compaction. Then retries via agent.continue().
• Incomplete-output recovery: Removes the truncated assistant message, tries promotion, then runs auto-maintenance (supports handoff strategy). Retries via agent.continue().

Threshold Maintenance

• Post-turn: runs after a successful turn when adjusted context tokens exceed the threshold. With handoff strategy, schedules a post-prompt auto-handoff task. With autoContinue !== false, schedules an auto-continue prompt from prompts/system/auto-continue.md.
• Mid-turn: runs inline before the next provider request when midTurnEnabled !== false. Suppresses handoff session resets; falls back to context-full compaction.

Pre-Compaction Optimizations

Tool-Output Pruning (pruneToolOutputs)

Before compaction decisions, OMP prunes large tool results:

• Protects newest 40,000 tool-output tokens
• Requires at least 20,000 total estimated savings
• Never blanks results below 50 tokens (placeholder would grow context)
• Skips skill tool results, skill:// path reads, and active plan references
• Pruned results become [Output truncated - N tokens]

Useless-Result Elision

Tools can flag results as contextually useless (zero-match searches, timed-out polls):

• Flagged results are blanked to [Uneventful result elided]
• Bypasses the protect-recent window
• Excluded from summarizer input during serialization
• Flagged pairs are never removed from history (only blanked in-place)

Compaction Entry Model

OMP stores compaction as first-class session entries:

// CompactionEntry
{
  type: "compaction",
  summary: string,
  shortSummary?: string,
  firstKeptEntryId: string,   // compaction boundary
  tokensBefore: number,
  details?: string,
  preserveData?: {...},      // snapcompact archive
  fromExtension?: string,
}

When rebuilding context (buildSessionContext):

1. Latest compaction → compactionSummary user message (via template)
2. Entries from firstKeptEntryId to compaction point are re-included
3. Later entries are appended

Cut-Point Logic

prepareCompaction() only considers entries since the last compaction entry:

1. Find previous compaction index
2. boundaryStart = prevCompactionIndex + 1
3. Adapt keepRecentTokens using measured usage ratio
4. Run findCutPoint() over the boundary window

Valid cut points: user, assistant, bashExecution, hookMessage, branchSummary, compactionSummary, custom_message, branch_summary. Never cut at toolResult.

Branch Summaries

When navigating /tree, abandoned branch context is captured as BranchSummaryEntry:

{
  type: "branch_summary",
  fromId: string,
  summary: string,
  details?: string,
  fromExtension?: string,
}

Converted to branchSummary messages via prompts/branch-summary-context.md.

Configuration

{
  "compaction": {
    "enabled": true,
    "strategy": "context-full",    // "context-full", "handoff", "snapcompact"
    "threshold": 0.8,              // fraction of context window
    "midTurnEnabled": true,
    "autoContinue": true,
    "dropUseless": true,
    "snapcompact": {
      "shape": "auto",
      "systemPrompt": false,
      "toolResults": false,
      "maxFrames": 80
    }
  }
}

Display Transcript

OMP's TUI shows compaction as a slim inline divider (── 📷 compacted · ctrl+o ──). Scrollback above the divider stays intact (no visual restart). Only LLM context resets at the compaction boundary.

opencode-compaction.md

OpenCode Compaction

Overview

OpenCode's V2 session model uses request-budget compaction: before each provider turn, the runner estimates the complete model-visible request against the model's context window minus a configured buffer headroom. When budget is exceeded and older turns are available, compaction fires.

How It Works

Trigger

Before each provider turn:

estimated_request_tokens >= (model_context_window - compaction.buffer)

The compaction.buffer is the greater of the model's output allowance and the configured buffer. This headroom prevents compaction from firing right at the absolute limit.

Compaction Checkpoint

Compaction replaces the active model representation with a hidden checkpoint containing:

1. Structured rolling summary — a continuation-friendly summary of compacted turns
2. Token-bounded serialized recent context — recent messages kept for continuity

The full transcript remains durable; only the model-visible representation changes. Provider-native assistant, reasoning, and tool messages are never carried across the compaction boundary (avoiding signature and encrypted-reasoning failures when the prefix changes).

Event Durability

session.next.compaction.started.1    — durably identifies the attempt
session.next.compaction.ended.1     — durably stores final summary and serialized context

• Only compaction.ended.1 projects a model-visible compaction message
• On the next provider attempt, the runner observes the completed compaction and directly renders a fresh Context Epoch baseline
• A failed or interrupted attempt leaves the previous history boundary active

Repeated Compaction

Repeated compactions update the previous structured summary with newly compacted messages rather than summarizing from scratch — information is preserved across cycles.

Overflow Recovery

When a provider rejects a request as context overflow before durable assistant output or tool execution, the runner attempts one overflow-triggered compaction. A completed checkpoint rebuilds the same logical provider turn with one remaining attempt. A second overflow, unavailable compaction, or overflow after durable output becomes the terminal failure — recovery never loops or replays partial side effects.

Manual Compaction

• TUI: /compact (alias: /summarize), keybind <leader>c
• API: POST /api/session/:sessionID/compact
• SDK: session.summarize({ path, body })
• Plugin hook: experimental.session.compacting — triggers before LLM generates the continuation summary, allowing injection of domain-specific context

Tool-Result Pruning

Configurable via compaction.prune: old tool output is deleted to save tokens. Defaults to false. Pruning is separate from compaction but works alongside it.

Configuration

{
  "compaction": {
    "auto": true,          // Enable automatic compaction
    "prune": false,        // Enable old tool-result pruning
    "keep": {
      "tokens": 10000     // Token budget for recent context to keep
    },
    "buffer": 4000         // Headroom reserve tokens
  }
}

Key Design Decisions

Feature	Description
Durable transcripts	Full conversation history is always persisted; compaction only changes model-visible representation
Checkpoint model	Compaction creates a hidden checkpoint (summary + serialized context), not a simple message replacement
Context Epoch	After compaction, a fresh baseline is rendered from the completed checkpoint
No overflow looping	Only one overflow-triggered compaction is attempted; a second overflow is terminal
Plugin hooks	session.compacting hook allows plugin-level context injection

phosphor-compaction.md

Context Window Compaction (Auto-Summarization)

Overview

Phosphor automatically summarizes a session's conversation history when the context window is approaching capacity. This "compaction" prevents context overflow by replacing the full message history with a concise summary produced by the model itself.

Configuration

summarize_model

The summarize_model option in phosphor.json controls which model handles compaction. Valid values:

Value	Behavior
"main" (default, alias for large)	Use the session's large model
"large"	Use the configured large model
"small"	Use the configured small model
"embedded"	Use the local dlgo GGUF model (requires embedded_models config)

These live in phosphor.json under the options key:

{
  "options": {
    "disable_auto_summarize": false,
    "summarize_threshold": 0.8,
    "summarize_model": "main",
    "summarize_prune_chars": 200,
    "summarize_prune_aggressive_chars": 50
  }
}

summarize_prune_chars

Number of characters to keep for each tool output during standard pre-compaction pruning. Defaults to 200. Tool results longer than this value are truncated and appended with \n... [truncated]. Non-tool messages are passed through unchanged. Lower values produce more aggressive context reduction; higher values retain more tool output detail.

summarize_prune_aggressive_chars

Number of characters to keep during overflow recovery pruning. Defaults to 50. Used as a fallback when the standard pruned input still exceeds the model's context window — the engine retries with this more aggressive truncation.

embedded_models

{
  "embedded_models": {
    "inference": {
      "enabled": true,
      "model_repo": "Qwen3.5-0.8B",
      "gpu": false
    }
  }
}

• model_repo: HuggingFace repo ID for auto-download (e.g., "Qwen3.5-0.8B", "Phi-3-mini", "Gemma-2-2B")
• model_path: Local path to a GGUF file (alternative to model_repo)
• gpu: Enable GPU acceleration (requires build with Vulkan support); falls back to CPU if GPU is unavailable.

Trigger Conditions

Auto-summarization is triggered at two checkpoints during SessionAgent.Run():

1. Pre-Request Threshold Check

Before sending the LLM request, the agent estimates the current token count of all messages and checks whether it exceeds the configured threshold fraction of the model's context window:

fraction = currentTokens / contextWindow

If fraction >= threshold (default 0.8, i.e. 80%), summarization fires.

2. Post-User-Message Overflow Check

After adding the new user message to the session, the agent re-estimates total tokens. If the total would exceed the model's context window, it force-summarizes before sending the request:

if totalTokens > contextWindow:
    force summarize

This is a safety net — it catches cases where a single large user prompt would push the conversation past the model's limit.

How Summarization Works

When triggered, SessionAgent.Summarize() executes:

1. Loads conversation history via getSessionMessages().
2. Prunes tool outputs — verbose tool results are truncated to summarize_prune_chars (default 200) characters (pkg/agent/prune.go) to reduce context pressure before summarization.
3. Loads summary prompt from the profile system (pkg/agent/prompt/templates/summarization.md.tpl), with optional iterative context (previous summary prepended).
4. Creates a summary message (IsSummaryMessage: true) in the session.
5. Sends the pruned history to the configured model (main/large, small, or embedded).
6. Streams the summary — text deltas are appended to the summary message in real-time.
7. Updates session state:

• SummaryMessageID is set to the new summary message's ID.
• CurrentTokens is recalculated from the summary content using approxTokenCount().
• CompletionTokens is updated from the response usage.
• PromptTokens is reset to 0 (the summary replaces the full history).

8. Processes queued messages — if user sent new prompts during summarization, the first queued message is processed immediately after.

Overflow Recovery

If the model rejects the request due to context overflow, the compaction engine retries with aggressive pruning (tool outputs truncated to 'summarize_prune_aggressive_chars', default 50). This handles edge cases where the standard pruning wasn't enough.

Iterative Summaries

On subsequent compaction cycles, the previous summary is prepended to the prompt, allowing the model to compound knowledge across multiple summarization passes.

Embedded Model Path

When summarize_model: "embedded", the summarizeEmbedded() path handles:

1. Model download: Auto-downloads GGUF models from HuggingFace at startup (ensureEmbeddedModel).
2. GPU acceleration: If gpu: true, attempts Vulkan GPU offload; falls back to CPU if no GPU is available.
3. Otel instrumentation: Tracks gen_ai.usage.input_tokens and gen_ai.usage.output_tokens for observability.

Profile-Based Prompt Customization

The summarization prompt is loaded via the 3-tier profile system:

1. Workspace: .phosphor/profiles/<profile>/summarization.md.tpl
2. Global: %LOCALAPPDATA%/phosphor/profiles/<profile>/summarization.md.tpl
3. Default: pkg/agent/prompt/templates/summarization.md.tpl (embedded)

Users can customize the prompt per-profile to tailor summary structure or focus.