managing-long-running-tasks-claude

SKILL.md

name	managing-long-running-tasks
description	Patterns for orchestrating multi-session tasks with background agents. Use when a task requires many agent iterations, has rate limits, or needs state tracking across sessions.

Managing Long-Running Tasks

State File

Create a state file at task start:

{
  "total": 100,
  "completed": ["item1", "item2"],
  "pending": ["item3", "item4", ...],
  "errors": {"item5": "rate limited"}
}

Update after each agent completes. Never infer state from output files.

Agent Batches

• Batch size: 30-50 items max per agent
• Pass explicit lists: Give agents the exact items to process, not "figure out what's remaining"
• Verify before continuing: Check counts match expected before launching next agent

Agent Prompts

Include in every agent prompt:

1. Explicit item list - The exact items to process this batch
2. Output file path - Where to write results
3. Schema - Full schema, every time (agents have no memory)
4. Boundaries - "ONLY process items from this list, nothing else"
5. Exit conditions - Specific criteria, not just "DO NOT STOP"

Example structure:

Process these 30 items: [list]
Output file: /path/to/results.json
Schema: {full schema}
ONLY add items from the list above.
Stop when: all 30 processed OR rate limited 3x in a row.

Between Agents

After each agent completes:

1. Run deduplication on output file
2. Update state file (move completed items)
3. Verify counts: completed + pending + errors = total
4. Launch next agent with remaining items

Rate Limits

When agents hit rate limits:

• Log to errors in state file
• Wait before relaunching
• Retry failed items in next batch

Completion

Task is complete when:

• pending is empty
• All items in completed or errors
• Final verification: output file count matches expected