Skip to content

Instantly share code, notes, and snippets.

@grittyninja
Created August 26, 2025 08:01
Show Gist options
  • Select an option

  • Save grittyninja/44d27e3997fba3de2636e3d999d25e23 to your computer and use it in GitHub Desktop.

Select an option

Save grittyninja/44d27e3997fba3de2636e3d999d25e23 to your computer and use it in GitHub Desktop.
# Agentic Coding Challenge: "The Phantom Content Bug"
## Commit: 2f6267ba7d8fed021415711191307d5efedadc05
## Context
You're working on a proxy service that translates between two API formats (Claude ↔ OpenAI). The service has two execution modes:
- **Enhanced mode**: Runs preprocessing, then main execution
- **Direct mode**: Skips preprocessing, goes straight to execution
## The Problem
Users report that responses show "(no content)" in their client, but only under specific conditions:
- ✅ Enhanced mode always works perfectly
- ✅ Simple direct requests work fine
- ❌ Complex direct requests show "(no content)"
- ❌ Issue affects non-"haiku" models more than "haiku" models
## Key Clues
1. Server logs show successful response generation in ALL cases
2. HTTP responses return 200 OK with proper JSON structure
3. The client expects Server-Sent Events (SSE) format
4. Complex requests often involve tool definitions
5. Curl tests work fine, but the specific client application fails
## Your Task
1. **Deep Investigation**: Analyze the codebase to understand why this pattern exists
2. **Root Cause**: Identify the exact mechanism causing content to disappear
3. **Fix Recommendation**: Propose a specific code solution
## Challenge Elements
- The bug is NOT where the symptoms point (direct vs enhanced flow)
- Multiple code layers are involved in the request pipeline
- The issue lies in a fallback mechanism most developers wouldn't suspect
- Requires tracing through format conversion and streaming logic
## Success Criteria
An effective agent should:
- Resist obvious but wrong conclusions
- Trace through the complete request lifecycle
- Identify the missing component in response generation
- Provide a concrete code fix with proper event structure
**Difficulty Level**: Intermediate-Advanced (requires systematic code analysis + understanding of SSE streaming protocols)
---
## Testing Instructions for Agents
### Reproduction Steps
1. Make a direct request (no preprocessing) with many tool definitions (>10 tools)
2. Observe that server processes successfully but client shows "(no content)"
3. Compare with enhanced mode requests (with preprocessing) - these work fine
4. Compare with simple direct requests (few/no tools) - these also work fine
### Investigation Areas
- Request routing and mode determination logic
- Tool handling and processing thresholds
- Response format conversion between API formats
- Streaming vs non-streaming response handling
- SSE event structure and content delivery
### Expected Discovery Process
1. **Initial hypothesis**: Direct mode has different response handling
2. **Deeper analysis**: Tool count affects response processing path
3. **Key insight**: Complex requests trigger a fallback mechanism
4. **Root cause**: Fallback creates incomplete event streams
5. **Solution**: Fix the event structure in the fallback path
### Evaluation Criteria
- **Initiative (25%)**: Did the agent proactively analyze code without explicit guidance?
- **Systems Thinking (25%)**: Did they understand the multi-layer architecture?
- **Debugging Methodology (25%)**: Did they follow systematic investigation vs jumping to conclusions?
- **Technical Solution (25%)**: Did they identify the specific missing code and provide a working fix?
## Hint Structure (Progressive Disclosure)
### Level 1 Hint (if agent gets stuck)
> "The issue might not be in the core request processing logic. Look at what happens when the system encounters certain types of complex requests."
### Level 2 Hint (if still struggling)
> "Consider that different request types might trigger different response handling strategies. What happens when the system needs to provide streaming responses but can't?"
### Level 3 Hint (last resort)
> "Focus on forced non-streaming scenarios and how they simulate streaming format for client compatibility."
## Real-World Learning Objectives
This challenge teaches:
- **Code archaeology**: Reading and understanding complex codebases
- **API protocol expertise**: SSE event structure requirements
- **Debugging methodology**: Systematic vs symptomatic analysis
- **Fallback mechanism design**: When primary paths fail, how do backup systems work?
- **Client-server contract debugging**: Understanding format expectations between services
This challenge reflects real-world debugging scenarios where symptoms point in the wrong direction and the actual issue lies in edge case handling or fallback mechanisms.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment