Sprite Console: Paste Interception Specification

Goal

When a user pastes an image into Claude CLI running in a sprite, it should work exactly as if Claude CLI was running locally. No extra steps, no visible difference.

Current Behavior (Local Claude CLI)

User copies image (screenshot, from browser, from Finder)
User pastes in Claude CLI (Cmd+V / Ctrl+V)
Terminal converts clipboard to file path: /var/folders/.../image.png
Claude CLI reads the file from that path
Image appears in conversation

Desired Behavior (Sprite)

User copies image on local machine
User pastes in Claude CLI running in sprite (Cmd+V / Ctrl+V)
[NEW] Sprite console intercepts, uploads file, rewrites path
Claude CLI reads the file from remote path: /tmp/pasted-image.png
Image appears in conversation

The user should not notice any difference.

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│  Terminal (Ghostty)                                                 │
│  - User presses Cmd+V                                               │
│  - Clipboard contains image → converted to temp file path           │
│  - Sends to sprite console: "/var/folders/abc/image.png"            │
└─────────────────────────────────┬───────────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────────┐
│  Sprite Console Client (INTERCEPTION POINT)                         │
│                                                                     │
│  Input buffer: "/var/folders/abc/image.png"                         │
│                                                                     │
│  1. Detect: looks like local file path                              │
│  2. Check: file exists at /var/folders/abc/image.png                │
│  3. Upload: POST to sprite API with file contents                   │
│  4. Receive: remote path "/tmp/pasted-a1b2c3.png"                   │
│  5. Rewrite: replace local path with remote path in buffer          │
│  6. Send to sprite: "/tmp/pasted-a1b2c3.png"                        │
│                                                                     │
└─────────────────────────────────┬───────────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────────┐
│  Sprite VM                                                          │
│  - Receives: "/tmp/pasted-a1b2c3.png"                               │
│  - File exists at that path (just uploaded)                         │
│  - Claude CLI reads it successfully                                 │
│  - ✓ Works exactly like local                                       │
└─────────────────────────────────────────────────────────────────────┘

Detection Logic

The sprite console client needs to identify when pasted text is a local file path that should be uploaded.

Patterns to Match

# macOS paths
/Users/*
/var/folders/*
/private/var/*
/tmp/* (local tmp, not sprite tmp)
~/*

# Linux paths
/home/*
/tmp/* (if running console from Linux)

# Windows paths
C:\Users\*
%USERPROFILE%\*

Detection Algorithm

def should_intercept_paste(text: str) -> bool:
    # Must look like an absolute file path
    if not looks_like_file_path(text):
        return False

    # Must exist locally
    if not os.path.exists(text):
        return False

    # Must be a file (not directory)
    if not os.path.isfile(text):
        return False

    # Optionally: only intercept certain file types
    # (images, PDFs, common document types)

    return True

def looks_like_file_path(text: str) -> bool:
    # Clean up: might have quotes, escapes, whitespace
    text = text.strip().strip('"\'')

    # macOS
    if text.startswith('/Users/'):
        return True
    if text.startswith('/var/folders/'):
        return True
    if text.startswith('/private/'):
        return True

    # Linux
    if text.startswith('/home/'):
        return True

    # Windows
    if re.match(r'^[A-Z]:\\', text):
        return True

    return False

Edge Cases

Path with spaces: /Users/name/My Files/image.png
- May be escaped: /Users/name/My\ Files/image.png
- May be quoted: "/Users/name/My Files/image.png"
- Must handle all formats
Multiple paths in one paste: Rare, but handle by processing each
Path that looks local but isn't: /Users/someone/file.txt when file doesn't exist
- Don't intercept, pass through as-is
Non-file paths: /Users/someone (directory)
- Don't intercept directories (or optionally zip and upload?)
Very large files: User pastes path to 1GB video
- Should have size limit with clear error message
- Suggest using explicit transfer for large files

Upload Mechanism

API Endpoint

POST /v1/sprites/{sprite_id}/files/upload
Authorization: Bearer {console_session_token}
Content-Type: multipart/form-data

------boundary
Content-Disposition: form-data; name="file"; filename="image.png"
Content-Type: image/png

<binary data>
------boundary
Content-Disposition: form-data; name="prefix"

pasted-
------boundary--

Response

{
  "path": "/tmp/pasted-a1b2c3d4.png",
  "size": 145832,
  "mime_type": "image/png"
}

Error Response

{
  "error": "file_too_large",
  "message": "File exceeds 50MB limit. Use 'sprite upload' for large files.",
  "max_size": 52428800
}

File Storage in Sprite

Location

Files uploaded via paste go to: /tmp/pasted-{uuid}.{ext}

Using /tmp/ because:

Standard location for temporary files
Auto-cleaned on sprite restart
Excluded from checkpoints by default
No special permissions needed

Naming

Format: pasted-{8-char-uuid}.{original-extension}

Examples:

pasted-a1b2c3d4.png
pasted-e5f6g7h8.jpg
pasted-i9j0k1l2.pdf

Cleanup

Options (choose one):

On sprite pause/stop: All /tmp/pasted-* files deleted
TTL-based: Delete after 1 hour
Session-based: Delete when console session ends
Manual: User runs cleanup command

Recommendation: On sprite stop - simplest, predictable, matches expectation that temp files are temporary.

Timing Considerations

Latency Budget

Local paste feels instant (<50ms). For sprite paste to feel similar:

Step	Target	Notes
Detect local path	<1ms	Simple regex
Check file exists	<5ms	Local filesystem
Read file	<100ms	Depends on size
Upload to sprite	<500ms	Network, depends on size
Receive response	<50ms	Small JSON
Total	<700ms	For typical image (<5MB)

For files <1MB, this should feel nearly instant. For files 1-10MB, slight delay but acceptable. For files >10MB, should show progress indicator.

Progress Feedback

For files taking >500ms to upload, show progress:

⠋ Uploading image.png (2.3 MB)...

Or integrate with terminal title/status line.

User Experience Details

Success Case

User pastes: /Users/aezell/Desktop/screenshot.png
User sees:   /tmp/pasted-a1b2c3d4.png    (after ~200ms)
Claude sees: /tmp/pasted-a1b2c3d4.png
Result:      Image loads correctly

Failure Cases

File doesn't exist locally:

User pastes: /Users/aezell/Desktop/deleted.png
User sees:   /Users/aezell/Desktop/deleted.png  (passed through unchanged)
Claude sees: /Users/aezell/Desktop/deleted.png
Result:      Claude shows "file not found" (expected behavior)

File too large:

User pastes: /Users/aezell/Movies/huge-video.mp4
User sees:   ⚠️ File too large (1.2GB). Use 'sprite upload' for files >50MB.
             /Users/aezell/Movies/huge-video.mp4
Claude sees: /Users/aezell/Movies/huge-video.mp4
Result:      User understands they need different approach

Upload fails (network error):

User pastes: /Users/aezell/Desktop/image.png
User sees:   ⚠️ Upload failed: connection timeout. Retrying...
             (retry succeeds)
             /tmp/pasted-a1b2c3d4.png

Configuration Options

# ~/.config/sprite/console.yaml

paste_interception:
  enabled: true                    # Master switch
  max_file_size: 52428800          # 50MB in bytes
  show_progress: true              # Show upload progress for large files
  file_types:                      # Only intercept these types (empty = all)
    - image/*
    - application/pdf
    - text/*

Most users shouldn't need to configure anything - defaults should "just work."

Security Considerations

Accidental sensitive file paste: User might paste path to sensitive file
- Mitigation: Files go to /tmp/, not persisted
- Optional: Prompt for confirmation for non-image files?
Path traversal: Malicious path like /../../../etc/passwd
- Mitigation: Validate path is absolute, file exists, is a regular file
Large file DoS: User pastes many large files
- Mitigation: Rate limiting, size limits, cleanup on session end
File type validation: Is the file actually what it claims to be?
- For images: Verify magic bytes match extension
- Optional: Full content-type sniffing

Implementation Phases

Phase 1: Basic Interception (MVP)

Detect macOS paths (/Users/*, /var/folders/*)
Upload images only (png, jpg, gif, webp)
Simple replacement, no progress
Size limit: 10MB

Phase 2: Enhanced UX

Progress indicator for large files
Support more file types
Windows path support
Better error messages

Phase 3: Polish

Configuration options
Retry logic
Rate limiting
Metrics/logging

Testing Checklist

Sprite Console: Clipboard Image Support Design

Problem Statement

When a user pastes an image into Claude Code running inside a sprite:

The terminal/Claude receives a local file path (e.g., /Users/aezell/Library/Messages/Attachments/.../Screenshot.png)
This path exists on the user's Mac, not in the sprite
Claude cannot access the file to analyze it

Current Architecture

┌─────────────────────────────────────────────────────────────┐
│  User's Mac                                                 │
│  ┌─────────────────┐    ┌─────────────────────────────┐    │
│  │ Clipboard       │    │ Ghostty Terminal            │    │
│  │ (image data or  │───▶│ - Receives paste event      │    │
│  │  file reference)│    │ - Converts to text (path)   │    │
│  └─────────────────┘    │ - Sends to sprite console   │    │
│                         └──────────────┬──────────────┘    │
└────────────────────────────────────────┼────────────────────┘
                                         │ TTY stream (text only)
                                         ▼
┌─────────────────────────────────────────────────────────────┐
│  Sprite VM                                                  │
│  ┌─────────────────────────────────────────────────────┐   │
│  │ PTY receives: "/Users/aezell/.../Screenshot.png"     │   │
│  │ (just text - no file data!)                          │   │
│  └─────────────────────────────────────────────────────┘   │
│                         │                                   │
│                         ▼                                   │
│  ┌─────────────────────────────────────────────────────┐   │
│  │ Claude Code                                          │   │
│  │ - Sees local path                                    │   │
│  │ - Cannot access file                                 │   │
│  │ - ❌ Fails                                           │   │
│  └─────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘

Key Insight

The sprite console client (running on the user's Mac) is the only component that has access to both:

The local filesystem (where the pasted image lives)
The connection to the sprite

Therefore, the sprite console client must handle the file transfer.

Proposed Solutions

Option 1: Paste Interception in Sprite Console Client

How it works:

User pastes in Ghostty (Cmd+V)
Ghostty sends paste content to sprite console client
Sprite console client inspects the paste:
- If it's a local file path (e.g., starts with /Users/, C:\, or ~)
- AND the file exists locally
- AND it's an image file
Sprite console uploads the file to sprite via API
Sprite console replaces the pasted path with the remote path
TTY receives the remote path instead

Architecture:

┌─────────────────────────────────────────────────────────────┐
│  Sprite Console Client (new behavior)                       │
│                                                             │
│  paste_input = "/Users/aezell/.../image.png"                │
│                                                             │
│  if is_local_path(paste_input) and file_exists(paste_input):│
│      remote_path = upload_to_sprite(paste_input)            │
│      send_to_tty(remote_path)  # e.g., "/tmp/image.png"     │
│  else:                                                      │
│      send_to_tty(paste_input)  # pass through as-is         │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Pros:

Completely transparent to user
Works with any application in the sprite (not just Claude)
No changes needed inside sprite

Cons:

Requires sprite console client changes
Need to define where files are uploaded (e.g., /.sprite/uploads/)
Need to handle cleanup of uploaded files

Option 2: Custom OSC Escape Sequence for File Request

How it works:

Sprite console supports a new OSC code: OSC 9999;file-request;<local-path>
When the sprite receives a local path it can't access, it sends this OSC
Sprite console client receives it, uploads the file, sends back the remote path

Sequence:

Sprite → Client:  ESC]9999;file-request;/Users/aezell/.../image.pngESC\
Client → Sprite:  (uploads file via API)
Client → Sprite:  ESC]9999;file-ready;/tmp/uploaded-image.pngESC\

Implementation in Claude hook:

# When Claude sees a local path, trigger file request
def request_file_transfer(local_path):
    tty = find_tty()
    escape = f'\x1b]9999;file-request;{local_path}\x1b\\'
    os.write(tty, escape.encode())
    # Wait for file-ready response...

Pros:

Explicit request mechanism
Can be triggered programmatically
Works for any file, not just images

Cons:

Requires coordination between sprite and client
More complex async handling
Needs response mechanism

Option 3: Support Kitty File Transfer Protocol

How it works:

Kitty has a well-documented file transfer protocol
If sprite console implements this, kitten transfer just works
Users can transfer files bidirectionally

Usage:

# From inside sprite, pull file from local machine:
kitten transfer --direction=receive /Users/aezell/.../image.png /tmp/image.png

# Or from local machine, push to sprite:
kitten transfer --direction=upload ./image.png sprite:/tmp/image.png

Pros:

Standard protocol, well-documented
Works with existing kitten tool
Bidirectional (can also download from sprite)

Cons:

More complex to implement fully
May be overkill for just image paste

Option 4: Clipboard Sync with Image Extraction

How it works:

Sprite console maintains clipboard sync between local and sprite
When clipboard contains image DATA (not path), it's synced to sprite
Image is saved to a known location: /.sprite/clipboard/image.png
A hook or alias expands [clipboard] to this path

Architecture:

Local clipboard change detected
       │
       ▼
┌─────────────────────────────────────────┐
│ Sprite Console Client                   │
│                                         │
│ if clipboard_contains_image():          │
│     image_data = get_clipboard_image()  │
│     upload_to_sprite(                   │
│         data=image_data,                │
│         path="/.sprite/clipboard/img"   │
│     )                                   │
│     notify_sprite("clipboard_updated")  │
│                                         │
└─────────────────────────────────────────┘

Inside sprite:

# Alias or hook can reference:
cat /.sprite/clipboard/image.png

# Or in Claude:
"Analyze this image: /.sprite/clipboard/image.png"

Pros:

Simple mental model: "clipboard is synced"
Predictable location for images
Could support text clipboard too

Cons:

Requires proactive sync (battery/bandwidth)
Only one clipboard item at a time
Doesn't solve the "path paste" problem directly

Option 5: Web-Based Upload UI

How it works:

Sprite exposes authenticated HTTP endpoint for uploads
User can drag-drop files to a web UI
Or use CLI: sprite upload ./image.png
Files appear in /.sprite/uploads/

Pros:

Works without terminal modifications
Good for large files
Can have nice UI

Cons:

Separate workflow from paste
Not seamless

Recommended Approach

Phase 1: Paste Interception (Option 1)

Modify sprite console client to detect local file path pastes
Upload file to sprite via existing API
Replace pasted text with remote path
Store uploads in /.sprite/uploads/ or /.sprite/tmp/

Phase 2: Custom OSC for Explicit Requests (Option 2)

Add OSC 9999;file-request and OSC 9999;file-ready
Enables programmatic file transfer from inside sprite
Hooks and scripts can use this

Phase 3: Full Kitty Protocol (Option 3)

For users who want bidirectional transfers
Power-user feature

Implementation Details for Phase 1

Sprite Console Client Changes

// Pseudocode for paste interception
func handlePaste(content string) string {
    // Check if it looks like a local file path
    if isLocalPath(content) && fileExists(content) {
        // Upload to sprite
        remotePath, err := uploadToSprite(content)
        if err != nil {
            log.Warn("Failed to upload pasted file:", err)
            return content // Fall back to original
        }
        return remotePath
    }
    return content
}

func isLocalPath(s string) bool {
    // Mac
    if strings.HasPrefix(s, "/Users/") { return true }
    // Linux
    if strings.HasPrefix(s, "/home/") { return true }
    // Windows
    if matched, _ := regexp.MatchString(`^[A-Z]:\\`, s); matched { return true }
    return false
}

Sprite API Endpoint

POST /v1/sprites/{id}/files
Authorization: Bearer <token>
Content-Type: multipart/form-data

file: <binary data>
path: /tmp/uploaded-image.png  (optional, auto-generate if not provided)

Response:
{
  "path": "/tmp/uploaded-abc123.png",
  "size": 12345
}

File Cleanup

Options:

TTL-based: Files in /.sprite/uploads/ auto-deleted after 1 hour
Session-based: Deleted when sprite pauses/stops
Checkpoint-aware: Excluded from checkpoints by default
Manual: User runs sprite cleanup uploads

Questions to Resolve

Where should uploaded files go?
- /.sprite/uploads/ (dedicated directory)
- /.sprite/tmp/ (temporary, auto-cleaned)
- /tmp/ (standard temp, may conflict)
How to handle filename collisions?
- Add UUID suffix: image-abc123.png
- Use content hash: image-sha256prefix.png
- Timestamp: image-20240110-123456.png
Size limits?
- Max file size for paste upload (e.g., 50MB?)
- Should large files require explicit transfer command?
Security considerations?
- Validate file types?
- Scan for malware?
- Rate limiting?
Should this be opt-in or default?
- Default on: Most users expect paste to work
- Could have config to disable if privacy concern

Summary

The core insight is that the sprite console client must handle file transfers because it's the only component with access to both local files and the sprite. The recommended approach is:

Start with paste interception - detect local paths and upload automatically
Add OSC escape sequences - for programmatic file requests
Consider Kitty protocol - for power users who want full file transfer capabilities

This would make pasting images into Claude "just work" without any user intervention.

aezell/paste-interception-spec.md

Sprite Console: Paste Interception Specification

Goal

Current Behavior (Local Claude CLI)

Desired Behavior (Sprite)

Architecture

Detection Logic

Patterns to Match

Detection Algorithm

Edge Cases

Upload Mechanism

API Endpoint

Response

Error Response

File Storage in Sprite

Location

Naming

Cleanup

Timing Considerations

Latency Budget

Progress Feedback

User Experience Details

Success Case

Failure Cases

Configuration Options

Security Considerations

Implementation Phases

Phase 1: Basic Interception (MVP)

Phase 2: Enhanced UX

Phase 3: Polish

Testing Checklist

Sprite Console: Clipboard Image Support Design

Problem Statement

Current Architecture

Key Insight

Proposed Solutions

Option 1: Paste Interception in Sprite Console Client

Option 2: Custom OSC Escape Sequence for File Request

Option 3: Support Kitty File Transfer Protocol

Option 4: Clipboard Sync with Image Extraction

Option 5: Web-Based Upload UI

Recommended Approach

Implementation Details for Phase 1

Sprite Console Client Changes

Sprite API Endpoint

File Cleanup

Questions to Resolve

Summary