Dummy implementation of OpenAI Responses API endpoints on Kubernetes Gateway API

README.md

Kubernetes Gateway API implementation that covers the OpenAI Responses API endpoints based on the current documentation.

Core Response Operations:

• POST /v1/responses - Create new responses with input, model, and tools
• GET /v1/responses/{id} - Retrieve response by ID
• GET /v1/responses/{id} (with Accept: text/event-stream) - Stream responses in real-time
• POST /v1/responses/{id} - Continue/extend existing responses
• DELETE /v1/responses/{id} - Delete responses

Specialized Endpoints:

• POST /v1/responses/parse - Parse endpoint for structured outputs
• POST /v1/responses/{id}/mcp/approve - MCP (Model Context Protocol) approval

File Management:

• POST /v1/files - Upload files (PDFs, images, etc.)
• GET /v1/files - List uploaded files
• GET /v1/files/{id} - Get file details
• DELETE /v1/files/{id} - Delete files

Key Features

Gateway Configuration:

• HTTPS/HTTP listeners on ports 443/80
• TLS termination with certificate management
• Hostname-based routing (api.openai-dummy.com)

Advanced Routing:

• Regex patterns for dynamic IDs
• Content-type based routing for streaming
• Header-based request differentiation

Production-like Features:

• Enhanced security headers
• Resource limits and requests
• Network policies for security
• ServiceMonitor for Prometheus integration
• Comprehensive error handling

Built-in Tools Support:

• Web search, file search, code interpreter, and computer use capabilities
• MCP (Model Context Protocol) integration

Response Formats:

• JSON responses for standard endpoints
• Server-Sent Events for streaming
• Structured output parsing
• File upload handling

Documentation:

• Interactive HTML documentation at /docs
• Complete API reference with examples
• Built-in authentication guidance

This implementation covers the OpenAI Responses API specification, including the latest features like streaming responses, built-in tools integration, and MCP support that were introduced in the March 2025 release.

dummy-openai-api-k8s-gateway-api.yaml

# Complete OpenAI Responses API Implementation with Kubernetes Gateway API
---
# Namespace
apiVersion: v1
kind: Namespace
metadata:
  name: openai-api
  labels:
    name: openai-api

---
# Gateway Class (if not already available in cluster)
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: openai-gateway-class
  namespace: openai-api
spec:
  controllerName: gateway.networking.k8s.io/gateway-controller
  description: "Gateway class for OpenAI API services"

---
# Gateway
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: openai-api-gateway
  namespace: openai-api
spec:
  gatewayClassName: openai-gateway-class
  listeners:
  - name: https
    port: 443
    protocol: HTTPS
    hostname: api.openai-dummy.com
    tls:
      mode: Terminate
      certificateRefs:
      - name: openai-api-tls-cert
        kind: Secret
  - name: http
    port: 80
    protocol: HTTP
    hostname: api.openai-dummy.com

---
# HTTPRoute for Complete Responses API
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: openai-responses-route
  namespace: openai-api
spec:
  parentRefs:
  - name: openai-api-gateway
    namespace: openai-api
  hostnames:
  - api.openai-dummy.com
  rules:
  # Create a new response
  - matches:
    - path:
        type: Exact
        value: /v1/responses
      method: POST
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: create-response
  
  # Get a specific response by ID
  - matches:
    - path:
        type: RegularExpression
        value: /v1/responses/[a-zA-Z0-9_-]+$
      method: GET
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: get-response
  
  # Stream a response by ID
  - matches:
    - path:
        type: RegularExpression
        value: /v1/responses/[a-zA-Z0-9_-]+$
      method: GET
      headers:
      - name: Accept
        value: text/event-stream
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: stream-response
  
  # Continue a response (add more input)
  - matches:
    - path:
        type: RegularExpression
        value: /v1/responses/[a-zA-Z0-9_-]+$
      method: POST
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: continue-response
  
  # Delete a response
  - matches:
    - path:
        type: RegularExpression
        value: /v1/responses/[a-zA-Z0-9_-]+$
      method: DELETE
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: delete-response
  
  # Parse endpoint for structured outputs
  - matches:
    - path:
        type: Exact
        value: /v1/responses/parse
      method: POST
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: parse-response
  
  # MCP (Model Context Protocol) endpoints
  - matches:
    - path:
        type: RegularExpression
        value: /v1/responses/[a-zA-Z0-9_-]+/mcp/approve
      method: POST
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: mcp-approve
  
  # Files upload for responses (supporting file operations)
  - matches:
    - path:
        type: Exact
        value: /v1/files
      method: POST
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: upload-file
  
  # Get file details
  - matches:
    - path:
        type: RegularExpression
        value: /v1/files/[a-zA-Z0-9_-]+$
      method: GET
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: get-file
  
  # Delete file
  - matches:
    - path:
        type: RegularExpression
        value: /v1/files/[a-zA-Z0-9_-]+$
      method: DELETE
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: delete-file
  
  # List files
  - matches:
    - path:
        type: Exact
        value: /v1/files
      method: GET
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: list-files
  
  # Health check endpoint
  - matches:
    - path:
        type: Exact
        value: /health
      method: GET
    backendRefs:
    - name: openai-responses-service
      port: 8080
      weight: 100
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-API-Route
          value: health-check

---
# Service
apiVersion: v1
kind: Service
metadata:
  name: openai-responses-service
  namespace: openai-api
  labels:
    app: openai-responses-api
spec:
  selector:
    app: openai-responses-api
  ports:
  - name: http
    port: 8080
    targetPort: 8080
    protocol: TCP
  type: ClusterIP

---
# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: openai-responses-api
  namespace: openai-api
  labels:
    app: openai-responses-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: openai-responses-api
  template:
    metadata:
      labels:
        app: openai-responses-api
    spec:
      containers:
      - name: openai-responses-api
        image: nginx:alpine
        ports:
        - containerPort: 8080
        env:
        - name: PORT
          value: "8080"
        - name: API_VERSION
          value: "v1"
        volumeMounts:
        - name: config
          mountPath: /etc/nginx/conf.d
        - name: html
          mountPath: /usr/share/nginx/html
        resources:
          requests:
            memory: "256Mi"
            cpu: "100m"
          limits:
            memory: "512Mi"
            cpu: "500m"
      volumes:
      - name: config
        configMap:
          name: openai-responses-nginx-config
      - name: html
        configMap:
          name: openai-responses-html

---
# ConfigMap for nginx configuration with all endpoints
apiVersion: v1
kind: ConfigMap
metadata:
  name: openai-responses-nginx-config
  namespace: openai-api
data:
  default.conf: |
    upstream backend {
        server 127.0.0.1:8081;
    }
    
    server {
        listen 8080;
        server_name localhost;
        
        # Security headers
        add_header 'X-Content-Type-Options' 'nosniff' always;
        add_header 'X-Frame-Options' 'DENY' always;
        add_header 'X-XSS-Protection' '1; mode=block' always;
        
        # CORS headers
        add_header 'Access-Control-Allow-Origin' '*' always;
        add_header 'Access-Control-Allow-Methods' 'GET, POST, DELETE, OPTIONS' always;
        add_header 'Access-Control-Allow-Headers' 'Authorization, Content-Type, X-API-Route, Accept' always;
        
        # Handle preflight requests
        if ($request_method = 'OPTIONS') {
            return 204;
        }
        
        # Main routing logic
        location / {
            set $route $http_x_api_route;
            set $response_id "";
            
            # Extract response ID from path if present
            if ($uri ~ "^/v1/responses/([a-zA-Z0-9_-]+)") {
                set $response_id $1;
            }
            
            # Extract file ID from path if present
            if ($uri ~ "^/v1/files/([a-zA-Z0-9_-]+)") {
                set $response_id $1;
            }
            
            # Route handlers
            if ($route = "create-response") {
                return 200 '{
                    "id": "resp_abc123def456",
                    "object": "response",
                    "created_at": 1735603200,
                    "model": "gpt-4o",
                    "status": "in_progress",
                    "output": [],
                    "usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}
                }';
            }
            
            if ($route = "get-response") {
                return 200 '{
                    "id": "resp_abc123def456",
                    "object": "response",
                    "created_at": 1735603200,
                    "model": "gpt-4o",
                    "status": "completed",
                    "output": [
                        {
                            "id": "msg_xyz789",
                            "type": "message",
                            "role": "assistant",
                            "content": [
                                {
                                    "type": "text",
                                    "text": "Hello! This is a dummy response from the OpenAI Responses API."
                                }
                            ]
                        }
                    ],
                    "usage": {"prompt_tokens": 15, "completion_tokens": 25, "total_tokens": 40}
                }';
            }
            
            if ($route = "stream-response") {
                add_header Content-Type "text/event-stream";
                add_header Cache-Control "no-cache";
                add_header Connection "keep-alive";
                return 200 'data: {"id":"resp_abc123def456","object":"response.delta","created_at":1735603200,"delta":{"content":[{"type":"text","text":"Hello"}]}}

data: {"id":"resp_abc123def456","object":"response.delta","created_at":1735603201,"delta":{"content":[{"type":"text","text":" from streaming!"}]}}

data: [DONE]

';
            }
            
            if ($route = "continue-response") {
                return 200 '{
                    "id": "resp_abc123def456",
                    "object": "response",
                    "created_at": 1735603200,
                    "model": "gpt-4o",
                    "status": "in_progress",
                    "output": [
                        {
                            "id": "msg_xyz789",
                            "type": "message",
                            "role": "assistant",
                            "content": [
                                {
                                    "type": "text",
                                    "text": "Continuing the conversation..."
                                }
                            ]
                        }
                    ],
                    "usage": {"prompt_tokens": 30, "completion_tokens": 15, "total_tokens": 45}
                }';
            }
            
            if ($route = "delete-response") {
                return 200 '{
                    "id": "resp_abc123def456",
                    "object": "response",
                    "deleted": true
                }';
            }
            
            if ($route = "parse-response") {
                return 200 '{
                    "id": "resp_parse123",
                    "object": "response",
                    "created_at": 1735603200,
                    "model": "gpt-4o",
                    "status": "completed",
                    "parsed_output": {
                        "name": "John Doe",
                        "age": 30,
                        "email": "[email protected]"
                    },
                    "usage": {"prompt_tokens": 20, "completion_tokens": 10, "total_tokens": 30}
                }';
            }
            
            if ($route = "mcp-approve") {
                return 200 '{
                    "id": "mcpr_approval123",
                    "object": "mcp_approval_response",
                    "approved": true,
                    "created_at": 1735603200
                }';
            }
            
            if ($route = "upload-file") {
                return 200 '{
                    "id": "file-abc123",
                    "object": "file",
                    "bytes": 1024,
                    "created_at": 1735603200,
                    "filename": "example.pdf",
                    "purpose": "responses"
                }';
            }
            
            if ($route = "get-file") {
                return 200 '{
                    "id": "file-abc123",
                    "object": "file",
                    "bytes": 1024,
                    "created_at": 1735603200,
                    "filename": "example.pdf",
                    "purpose": "responses"
                }';
            }
            
            if ($route = "delete-file") {
                return 200 '{
                    "id": "file-abc123",
                    "object": "file",
                    "deleted": true
                }';
            }
            
            if ($route = "list-files") {
                return 200 '{
                    "object": "list",
                    "data": [
                        {
                            "id": "file-abc123",
                            "object": "file",
                            "bytes": 1024,
                            "created_at": 1735603200,
                            "filename": "example.pdf",
                            "purpose": "responses"
                        },
                        {
                            "id": "file-def456",
                            "object": "file",
                            "bytes": 2048,
                            "created_at": 1735603300,
                            "filename": "document.txt",
                            "purpose": "responses"
                        }
                    ],
                    "has_more": false
                }';
            }
            
            if ($route = "health-check") {
                return 200 '{
                    "status": "healthy",
                    "version": "1.0.0",
                    "timestamp": "2025-08-12T12:00:00Z"
                }';
            }
            
            # Default response for unrecognized routes
            return 400 '{
                "error": {
                    "message": "Invalid route or method. Check the OpenAI Responses API documentation.",
                    "type": "invalid_request_error",
                    "param": null,
                    "code": "invalid_route"
                }
            }';
        }
        
        # Static documentation endpoint
        location /docs {
            root /usr/share/nginx/html;
            index index.html;
        }
    }

---
# ConfigMap for static HTML documentation
apiVersion: v1
kind: ConfigMap
metadata:
  name: openai-responses-html
  namespace: openai-api
data:
  index.html: |
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <title>OpenAI Responses API - Dummy Implementation</title>
        <style>
            body { font-family: Arial, sans-serif; margin: 40px; background: #f5f5f5; }
            .container { max-width: 1200px; margin: 0 auto; background: white; padding: 30px; border-radius: 8px; box-shadow: 0 2px 10px rgba(0,0,0,0.1); }
            h1 { color: #333; border-bottom: 3px solid #10a37f; padding-bottom: 10px; }
            h2 { color: #10a37f; margin-top: 30px; }
            .endpoint { background: #f8f9fa; padding: 15px; border-left: 4px solid #10a37f; margin: 10px 0; border-radius: 4px; }
            .method { font-weight: bold; color: #007bff; }
            .path { font-family: monospace; background: #e9ecef; padding: 2px 6px; border-radius: 3px; }
            code { background: #e9ecef; padding: 2px 6px; border-radius: 3px; font-family: monospace; }
            .example { background: #f1f3f4; padding: 15px; border-radius: 4px; margin: 10px 0; }
        </style>
    </head>
    <body>
        <div class="container">
            <h1>🤖 OpenAI Responses API - Dummy Implementation</h1>
            <p>This is a complete dummy implementation of the OpenAI Responses API using Kubernetes Gateway API.</p>
            
            <h2>📋 Core Response Endpoints</h2>
            
            <div class="endpoint">
                <div><span class="method">POST</span> <span class="path">/v1/responses</span></div>
                <p>Create a new response with input, model, and optional tools</p>
            </div>
            
            <div class="endpoint">
                <div><span class="method">GET</span> <span class="path">/v1/responses/{response_id}</span></div>
                <p>Retrieve a response by its ID</p>
            </div>
            
            <div class="endpoint">
                <div><span class="method">GET</span> <span class="path">/v1/responses/{response_id}</span> <code>Accept: text/event-stream</code></div>
                <p>Stream a response in real-time using Server-Sent Events</p>
            </div>
            
            <div class="endpoint">
                <div><span class="method">POST</span> <span class="path">/v1/responses/{response_id}</span></div>
                <p>Continue a response by adding more input</p>
            </div>
            
            <div class="endpoint">
                <div><span class="method">DELETE</span> <span class="path">/v1/responses/{response_id}</span></div>
                <p>Delete a response and free associated resources</p>
            </div>
            
            <h2>🔧 Specialized Endpoints</h2>
            
            <div class="endpoint">
                <div><span class="method">POST</span> <span class="path">/v1/responses/parse</span></div>
                <p>Parse input with structured output format (JSON Schema)</p>
            </div>
            
            <div class="endpoint">
                <div><span class="method">POST</span> <span class="path">/v1/responses/{response_id}/mcp/approve</span></div>
                <p>Approve MCP (Model Context Protocol) actions</p>
            </div>
            
            <h2>📁 File Management</h2>
            
            <div class="endpoint">
                <div><span class="method">POST</span> <span class="path">/v1/files</span></div>
                <p>Upload files for use with responses (PDFs, images, etc.)</p>
            </div>
            
            <div class="endpoint">
                <div><span class="method">GET</span> <span class="path">/v1/files</span></div>
                <p>List all uploaded files</p>
            </div>
            
            <div class="endpoint">
                <div><span class="method">GET</span> <span class="path">/v1/files/{file_id}</span></div>
                <p>Get details about a specific file</p>
            </div>
            
            <div class="endpoint">
                <div><span class="method">DELETE</span> <span class="path">/v1/files/{file_id}</span></div>
                <p>Delete a file</p>
            </div>
            
            <h2>🛠️ System Endpoints</h2>
            
            <div class="endpoint">
                <div><span class="method">GET</span> <span class="path">/health</span></div>
                <p>Health check endpoint for monitoring</p>
            </div>
            
            <h2>📝 Example Usage</h2>
            
            <div class="example">
                <h3>Create a Response</h3>
                <pre><code>curl -X POST https://api.openai-dummy.com/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{
    "model": "gpt-4o",
    "input": "What is the capital of France?",
    "tools": [
      {"type": "web_search"},
      {"type": "file_search"}
    ]
  }'</code></pre>
            </div>
            
            <div class="example">
                <h3>Stream a Response</h3>
                <pre><code>curl -X GET https://api.openai-dummy.com/v1/responses/resp_abc123def456 \
  -H "Accept: text/event-stream" \
  -H "Authorization: Bearer your-api-key"</code></pre>
            </div>
            
            <h2>🎯 Built-in Tools</h2>
            <ul>
                <li><strong>Web Search:</strong> Access real-time information from the web</li>
                <li><strong>File Search:</strong> Search through uploaded documents</li>
                <li><strong>Code Interpreter:</strong> Execute Python code in a sandboxed environment</li>
                <li><strong>Computer Use:</strong> Interact with user interfaces (preview)</li>
                <li><strong>MCP (Model Context Protocol):</strong> Connect to external systems</li>
            </ul>
            
            <h2>🔒 Authentication</h2>
            <p>Include your API key in the Authorization header:</p>
            <code>Authorization: Bearer your-api-key-here</code>
            
            <p><em>This is a dummy implementation for testing and development purposes.</em></p>
        </div>
    </body>
    </html>

---
# TLS Certificate Secret (replace with actual certificate data)
apiVersion: v1
kind: Secret
metadata:
  name: openai-api-tls-cert
  namespace: openai-api
type: kubernetes.io/tls
data:
  # Base64 encoded certificate and key
  # Replace with actual certificate data
  tls.crt: LS0tLS1CRUdJTi0tLS0t  # Your certificate here
  tls.key: LS0tLS1CRUdJTi0tLS0t  # Your private key here

---
# ReferenceGrant for cross-namespace access
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: allow-gateway-to-service
  namespace: openai-api
spec:
  from:
  - group: gateway.networking.k8s.io
    kind: Gateway
    namespace: openai-api
  to:
  - group: ""
    kind: Service

---
# HorizontalPodAutoscaler for auto-scaling
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: openai-responses-api-hpa
  namespace: openai-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: openai-responses-api
  minReplicas: 3
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

---
# ServiceMonitor for Prometheus monitoring (optional)
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: openai-responses-api-monitor
  namespace: openai-api
  labels:
    app: openai-responses-api
spec:
  selector:
    matchLabels:
      app: openai-responses-api
  endpoints:
  - port: http
    path: /health
    interval: 30s

---
# NetworkPolicy for security (optional)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: openai-responses-api-netpol
  namespace: openai-api
spec:
  podSelector:
    matchLabels:
      app: openai-responses-api
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: gateway-system
    ports:
    - protocol: TCP
      port: 8080
  egress:
  - {} # Allow all egress for this demo