Skip to content

Instantly share code, notes, and snippets.

@danielezonca

danielezonca/envoy-ai-gateway-composability-analysis.md

Created February 10, 2026 09:52
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save danielezonca/02c8c139f294663a9060b90ea851371f to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save danielezonca/02c8c139f294663a9060b90ea851371f to your computer and use it in GitHub Desktop.
Download ZIP
Envoy AI Gateway: Composability Feasibility Analysis - Istio Gateway Support
Raw
envoy-ai-gateway-composability-analysis.md

Envoy AI Gateway: Composability Feasibility Analysis

Analysis of making the Envoy AI Gateway composable across different gateway implementations (Envoy Gateway, Istio Gateway, and others).

Repository: envoyproxy/ai-gateway

Table of Contents

1. Executive Summary

Feasibility: HIGH. The Envoy AI Gateway already has a clean architectural boundary between its data plane and control plane. The data plane (ext_proc server, translators, auth handlers, config model) has zero dependencies on Envoy Gateway and can work with any Envoy-based proxy that supports the ext_proc filter — including Istio Gateway.

The work required falls into three categories:

  1. No changes needed: Data plane components (ext_proc, translators, auth, config model) — already portable
  2. Refactoring needed: Extract gateway-agnostic logic from the current Envoy Gateway controller into a shared library
  3. New code needed: Istio Gateway adapter (controller + EnvoyFilter generation)

The current codebase's explicit decoupling of filterapi.Config from Kubernetes and Envoy Gateway types makes this significantly easier than it might otherwise be.

2. Current Architecture: Data Plane vs Control Plane

┌─────────────────────────────────────────────────────────────────┐
│                     CONTROL PLANE                                │
│                                                                  │
│  cmd/controller/main.go          Entry point                     │
│  internal/controller/            K8s controllers                 │
│    ├── ai_gateway_route.go       AIGatewayRoute → HTTPRoute +   │
│    │                             egv1a1.HTTPRouteFilter          │
│    ├── ai_service_backend.go     AIServiceBackend reconciler     │
│    ├── backend_security_policy.go BSP → egv1a1.SecurityPolicy   │
│    ├── gateway.go                Gateway → filterapi.Config      │
│    ├── gateway_mutator.go        Sidecar injection webhook       │
│    ├── mcp_route.go              MCPRoute → egv1a1.Backend      │
│    └── mcp_route_security_policy.go MCP security policies       │
│  internal/extensionserver/       Envoy Gateway extension hooks   │
│    ├── extensionserver.go        EnvoyGatewayExtensionServer     │
│    ├── post_translate_modify.go  xDS cluster/listener/route mods │
│    ├── post_route_modify.go      Per-route modifications         │
│    ├── post_cluster_modify.go    Per-cluster modifications       │
│    └── mcproute.go               MCP resource generation         │
│                                                                  │
├──────────────────────── BOUNDARY ────────────────────────────────┤
│            filterapi.Config YAML (filesystem)                    │
├──────────────────────────────────────────────────────────────────┤
│                                                                  │
│                      DATA PLANE                                  │
│                                                                  │
│  cmd/extproc/                    ext_proc binary entry point     │
│  internal/extproc/               ext_proc gRPC server            │
│  internal/translator/            Schema translation              │
│  internal/filterapi/             Config model + file watcher     │
│  internal/backendauth/           Provider auth handlers          │
│  internal/apischema/             Provider data models            │
│  internal/mcpproxy/              MCP proxy                       │
│                                                                  │
└──────────────────────────────────────────────────────────────────┘

The boundary between control plane and data plane is a plain YAML file (filterapi.Config) stored in a Kubernetes Secret and mounted into the ext_proc container. The ext_proc process watches this file for changes and reloads configuration dynamically.

3. Data Plane Independence Assessment

Every data plane component was verified for Envoy Gateway dependencies:

Component Path EG Imports Gateway API Imports K8s Imports Verdict
ext_proc server internal/extproc/ None None None Fully portable
Translators internal/translator/ None None None Fully portable
Config model internal/filterapi/ None None None Fully portable
Config watcher internal/filterapi/watcher.go None None None Fully portable
Backend auth internal/backendauth/ None None None Fully portable
API schemas internal/apischema/ None None None Fully portable
MCP proxy internal/mcpproxy/ None None None Fully portable
ext_proc binary cmd/extproc/ None None None Fully portable

The filterapi.Config format is explicitly designed to be gateway-agnostic (comment in filterconfig.go):

"This configuration must be decoupled from the Envoy Gateway types as well as its implementation details. Also, the configuration must not be tied with k8s so it can be tested and iterated without the need for the k8s cluster."

Conclusion: The entire data plane can be used as-is with Istio Gateway or any other Envoy-based proxy. No changes required.

4. Control Plane Coupling Points

4.1 Extension Server gRPC Interface

Location: internal/extensionserver/

The extension server implements Envoy Gateway's proprietary EnvoyGatewayExtensionServer gRPC service:

egextension.RegisterEnvoyGatewayExtensionServer(grpcServer, extensionServer)

Three hooks are implemented:

Hook File What It Does
PostTranslateModify() post_translate_modify.go Injects ext_proc filter into clusters/listeners, creates UDS cluster, creates EPP clusters, wires MCP
PostClusterModify() post_cluster_modify.go Converts clusters to ORIGINAL_DST for InferencePool endpoint picking
PostRouteModify() post_route_modify.go Disables auto host rewrite, adds InferencePool metadata to routes

Impact: This entire package is Envoy Gateway-specific. Istio has no equivalent gRPC extension interface. The equivalent Istio mechanism is the EnvoyFilter CRD or Gateway API policy attachment.

4.2 Controller Resource Generation

Controllers watch AI Gateway CRDs and generate Envoy Gateway-specific resources:

Controller AI Gateway CRD Creates EG-Specific Resources
ai_gateway_route.go AIGatewayRoute egv1a1.HTTPRouteFilter (host rewriting, 404 responses)
mcp_route.go MCPRoute egv1a1.Backend
backend_security_policy.go BackendSecurityPolicy egv1a1.SecurityPolicy, egv1a1.BackendTrafficPolicy
mcp_route_security_policy.go MCPRoute egv1a1.SecurityPolicy, egv1a1.BackendTrafficPolicy, egv1a1.HTTPRouteFilter
gateway.go Gateway (standard) filterapi.Config stored in K8s Secret

Complete list of Envoy Gateway types used:

egv1a1.HTTPRouteFilter          egv1a1.Backend
egv1a1.SecurityPolicy           egv1a1.BackendTrafficPolicy
egv1a1.CustomResponseBody       egv1a1.HTTPURLRewriteFilter
egv1a1.HTTPDirectResponseFilter egv1a1.ExtAuth
egv1a1.JWT                      egv1a1.OIDC
egv1a1.APIKeyAuth               egv1a1.RemoteJWKS
egv1a1.KubernetesContainerSpec  egv1a1.BackendRef
egv1a1.BackendCluster           egv1a1.BackendEndpoint
egv1a1.IPEndpoint               egv1a1.BackendHTTPHostnameModifier
egv1a1.GRPCExtAuthService       egv1a1.ResponseOverride

Impact: Every controller except gateway.go generates EG-specific resources. However, gateway.go contains the gateway-agnostic logic that builds filterapi.Config from AI Gateway CRDs — this is the reusable core.

4.3 Sidecar Injection

Location: internal/controller/gateway_mutator.go

A mutating webhook that injects the ext_proc container into Envoy Gateway pods. This mechanism is entirely EG-specific.

Impact: Istio has its own sidecar injection. For standalone deployment, this isn't needed at all.

5. Envoy Gateway vs Istio Gateway: Extension Mechanisms

Concern Envoy Gateway Istio Gateway Gateway API Standard
ext_proc filter injection Extension Server gRPC hooks (PostTranslateModify) EnvoyFilter CRD Not available (no filter injection API)
Route configuration egv1a1.HTTPRouteFilter EnvoyFilter or VirtualService HTTPRoute filter extensions (limited)
Security policies egv1a1.SecurityPolicy (JWT, OIDC, ExtAuth, APIKey) RequestAuthentication, AuthorizationPolicy Not standardized
Backend traffic policy egv1a1.BackendTrafficPolicy DestinationRule Not standardized
Backend definitions egv1a1.Backend ServiceEntry or K8s Service BackendRef (standard)
Sidecar injection Custom mutating webhook Istio's native injection (istio-proxy) Not applicable
Rate limiting egv1a1.BackendTrafficPolicy with rate limit config EnvoyFilter or Istio rate limit service Not standardized
InferencePool Extension resources via PostClusterModify EnvoyFilter for ORIGINAL_DST + metadata InferencePool (Gateway API Inference Extension)

Key insight: Both Envoy Gateway and Istio Gateway use Envoy as the data plane proxy. The ext_proc filter is an Envoy feature, not a gateway implementation feature. The difference is only in how the ext_proc filter gets configured in the Envoy instance.

6. Proposed Composable Architecture

6.1 Core AI Gateway (Portable)

Everything that is gateway-implementation-agnostic:

ai-gateway-core/
├── api/v1alpha1/                  AI Gateway CRDs (unchanged)
│   ├── AIGatewayRoute
│   ├── AIServiceBackend
│   ├── BackendSecurityPolicy
│   ├── MCPRoute
│   ├── QuotaPolicy
│   └── GatewayConfig
├── internal/extproc/              ext_proc gRPC server (unchanged)
├── internal/translator/           Schema translation (unchanged)
├── internal/filterapi/            Config model + watcher (unchanged)
├── internal/backendauth/          Provider auth handlers (unchanged)
├── internal/apischema/            Provider data models (unchanged)
├── internal/mcpproxy/             MCP proxy (unchanged)
├── internal/controller/
│   └── config_builder.go          NEW: extracted from gateway.go
│                                  Builds filterapi.Config from CRDs
│                                  Gateway-agnostic logic only
└── cmd/extproc/                   ext_proc binary (unchanged)

6.2 Gateway Adapter Interface

Each gateway implementation provides an adapter that fulfills these responsibilities:

// Conceptual interface — the adapter responsibilities
type GatewayAdapter interface {
    // Config Generation
    // Watch AI Gateway CRDs and generate filterapi.Config YAML
    // Store it where the ext_proc process can read it
    ReconcileConfig(ctx context.Context, route *AIGatewayRoute) (*filterapi.Config, error)

    // ext_proc Filter Wiring
    // Inject ext_proc filter into the gateway's Envoy configuration
    // (EG: extension server hooks, Istio: EnvoyFilter CRD)
    EnsureExtProcFilter(ctx context.Context, gateway *Gateway) error

    // Security Policy Translation
    // Translate BackendSecurityPolicy to gateway-native auth resources
    // (EG: SecurityPolicy, Istio: RequestAuthentication+AuthorizationPolicy)
    ReconcileSecurityPolicy(ctx context.Context, bsp *BackendSecurityPolicy) error

    // Backend Registration
    // Register AI backends with the gateway's service discovery
    // (EG: Backend CRD, Istio: ServiceEntry or K8s Service)
    ReconcileBackend(ctx context.Context, backend *AIServiceBackend) error

    // ext_proc Deployment Management
    // Deploy and manage the ext_proc process
    // (sidecar injection, standalone deployment, or both)
    EnsureExtProcDeployment(ctx context.Context, gateway *Gateway) error
}

This is not necessarily a Go interface in code — it's the contract each adapter must fulfill. Implementation can be via separate controllers, extension servers, or any other mechanism.

6.3 Adapter Responsibilities

┌────────────────────────────────────────────────────────────┐
│                AI Gateway Core (Portable)                   │
│                                                             │
│  CRDs ──→ Config Builder ──→ filterapi.Config YAML         │
│                                                             │
│  ext_proc binary (reads YAML, serves gRPC)                 │
│  ├── Translators (OpenAI ↔ Bedrock/Anthropic/Gemini/...)  │
│  ├── Auth handlers (AWS SigV4, GCP, Azure, API keys)      │
│  └── MCP proxy                                              │
│                                                             │
└──────────────────────┬─────────────────────────────────────┘
                       │
            filterapi.Config (YAML)
                       │
     ┌─────────────────┼─────────────────┐
     │                 │                 │
     ▼                 ▼                 ▼
┌──────────┐    ┌──────────┐    ┌──────────────┐
│ EG       │    │ Istio    │    │ Standalone   │
│ Adapter  │    │ Adapter  │    │ (no gateway) │
│          │    │          │    │              │
│ Tasks:   │    │ Tasks:   │    │ Tasks:       │
│ ① Ext    │    │ ① Envoy  │    │ ① Generate   │
│   Server │    │   Filter │    │   static     │
│   hooks  │    │   CRD    │    │   Envoy      │
│ ② EG     │    │ ② Istio  │    │   config     │
│   CRDs   │    │   AuthN/ │    │ ② Run ext_   │
│   (Sec   │    │   AuthZ  │    │   proc       │
│   Policy │    │ ③ Svc    │    │   standalone │
│   etc.)  │    │   Entry  │    │              │
│ ③ Side-  │    │   or K8s │    │              │
│   car    │    │   Svc    │    │              │
│   inject │    │ ④ Side-  │    │              │
│          │    │   car or │    │              │
│          │    │   deploy │    │              │
└──────────┘    └──────────┘    └──────────────┘

7. Required Codebase Changes

7.1 Changes to Core (Minimal)

The goal is to minimize changes to the existing codebase. The main extraction is the config builder logic from internal/controller/gateway.go:

Current state: gateway.go contains both:

  • Gateway-agnostic logic: building filterapi.Config from AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy CRDs
  • EG-specific logic: storing config in K8s Secrets, formatting for EG sidecar mount paths

Proposed change: Extract the filterapi.Config building logic into a shared function/package that both the EG adapter and Istio adapter can call:

// internal/controller/config_builder.go (NEW - extracted from gateway.go)

// BuildFilterConfig constructs a filterapi.Config from AI Gateway CRDs.
// This is gateway-implementation-agnostic.
func BuildFilterConfig(
    route *v1alpha1.AIGatewayRoute,
    backends map[string]*v1alpha1.AIServiceBackend,
    securityPolicies map[string]*v1alpha1.BackendSecurityPolicy,
) (*filterapi.Config, error) {
    // Extracted from gateway.go reconcileAIGatewayRoute()
    // Builds backends, models, LLM request costs
    // Returns portable filterapi.Config
}

Files affected in core:

  • internal/controller/gateway.go — extract config-building logic (refactor, not rewrite)
  • No other core files need changes

7.2 Envoy Gateway Adapter (Refactor Existing)

The existing code stays largely in place. The only change is that gateway.go calls the extracted BuildFilterConfig() instead of inline logic:

internal/controller/gateway.go          → calls BuildFilterConfig() (minor refactor)
internal/controller/ai_gateway_route.go → unchanged (still creates egv1a1.HTTPRouteFilter)
internal/extensionserver/               → unchanged (still implements EG extension hooks)

7.3 Istio Gateway Adapter (New)

A new adapter package (potentially in a separate repo) that:

7.3.1 Config Generation Controller

Watches AI Gateway CRDs and generates filterapi.Config:

// Watches: AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy
// Calls: core.BuildFilterConfig()
// Outputs: filterapi.Config YAML stored in K8s Secret or ConfigMap

7.3.2 EnvoyFilter Generation

Creates Istio EnvoyFilter CRDs to inject the ext_proc filter:

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: ai-gateway-extproc
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  # Router-level ext_proc filter
  - applyTo: HTTP_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: envoy.filters.network.http_connection_manager
            subFilter:
              name: envoy.filters.http.router
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.filters.http.ext_proc
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor
          grpc_service:
            envoy_grpc:
              cluster_name: ai-gateway-extproc
            timeout: 30s
          processing_mode:
            request_header_mode: SEND
            request_body_mode: BUFFERED
            response_header_mode: SEND
            response_body_mode: BUFFERED
          message_timeout: 10s
          failure_mode_allow: false
          allow_mode_override: true

  # ext_proc cluster (UDS or network)
  - applyTo: CLUSTER
    match:
      context: GATEWAY
    patch:
      operation: ADD
      value:
        name: ai-gateway-extproc
        type: STATIC
        connect_timeout: 10s
        typed_extension_protocol_options:
          envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
            "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
            explicit_http_config:
              http2_protocol_options: {}
        load_assignment:
          cluster_name: ai-gateway-extproc
          endpoints:
          - lb_endpoints:
            - endpoint:
                address:
                  # For sidecar mode (UDS):
                  pipe:
                    path: /etc/ai-gateway-extproc-uds/run.sock
                  # For standalone mode (network):
                  # socket_address:
                  #   address: ai-gateway-extproc.ai-gateway.svc.cluster.local
                  #   port_value: 1063

7.3.3 Security Policy Translation

Maps BackendSecurityPolicy to Istio-native resources:

BackendSecurityPolicy Field Istio Equivalent
JWT authentication RequestAuthentication CRD
OIDC RequestAuthentication + token exchange
API Key AuthorizationPolicy with header match or EnvoyFilter
ExtAuth AuthorizationPolicy with custom action

Note: Backend-level auth (AWS SigV4, GCP credentials, Azure tokens) is handled by the ext_proc process itself via internal/backendauth/ — this is portable and needs no Istio-specific mapping.

7.3.4 Gateway API HTTPRoute

Both Envoy Gateway and Istio support standard Gateway API HTTPRoute. The Istio adapter can create HTTPRoute resources referencing the AI backends, similar to what the EG controller does today. The difference is what supplementary resources are created alongside:

Concern EG Creates Istio Creates
Route HTTPRoute + egv1a1.HTTPRouteFilter HTTPRoute (standard filters only)
Host rewrite egv1a1.HTTPURLRewriteFilter EnvoyFilter patch or HTTPRoute filter
404 fallback egv1a1.HTTPDirectResponseFilter EnvoyFilter patch

8. Milestone 1: Chat Completion on Istio Gateway

Goal: Enable OpenAI-compatible chat completion through Istio Gateway with schema translation to any supported backend provider.

Scope

  • Single AIGatewayRoute with one or more AIServiceBackend backends
  • Chat completion endpoint only (/v1/chat/completions)
  • Backend auth via ext_proc (API keys, AWS, GCP, Azure)
  • Both streaming and non-streaming responses
  • Standalone ext_proc deployment (simplest to start)

What's Needed

  1. ext_proc binary: Already works as-is. Deploy as a standalone K8s Deployment + Service.

  2. filterapi.Config generation: Extract BuildFilterConfig() from gateway.go. Write a minimal Istio controller that:

    • Watches AIGatewayRoute + AIServiceBackend CRDs
    • Calls BuildFilterConfig()
    • Stores result in a ConfigMap mounted by the ext_proc Deployment

  3. EnvoyFilter for ext_proc injection: Generate an EnvoyFilter CRD (as shown in section 7.3.2) that adds the ext_proc filter to the Istio gateway's Envoy config. For standalone deployment, use network address instead of UDS.

  4. HTTPRoute generation: Create standard HTTPRoute pointing client traffic to the AI backend services, with the ext_proc filter intercepting and translating.

What's NOT Needed for Milestone 1

  • Sidecar injection (standalone deployment only)
  • Security policies (BackendSecurityPolicy → Istio AuthN/AuthZ)
  • MCPRoute support
  • InferencePool / endpoint picker integration
  • QuotaPolicy / rate limiting
  • Host rewrite filters (ext_proc handles path translation)

Architecture for Milestone 1

Client
  │
  ▼
┌──────────────────────┐
│ Istio Gateway        │
│ (Envoy proxy)        │
│                      │
│ EnvoyFilter adds:    │
│  ext_proc filter ──────────► ┌──────────────────────┐
│  (gRPC over network) │       │ ext_proc Deployment  │
│                      │       │                      │
│                      │       │ Reads:               │
│                      │       │  filterapi.Config    │
│                      │       │  (from ConfigMap)    │
│                      │       │                      │
│                      │       │ Translates:          │
│                      │       │  OpenAI ↔ Provider   │
│                      │       │                      │
│                      │       │ Auth:                │
│                      │       │  AWS/GCP/Azure/Keys  │
└──────┬───────────────┘       └──────────────────────┘
       │
       ▼
┌──────────────────────┐
│ Backend Provider     │
│ (OpenAI, Bedrock,    │
│  Vertex AI, etc.)    │
└──────────────────────┘

Estimated Changes

Area Change Type Scope
internal/controller/gateway.go Refactor Extract BuildFilterConfig() to shared function
Istio adapter controller New code ~300-500 lines (config generation + EnvoyFilter)
Istio adapter Helm chart New code Deployment, Service, RBAC, EnvoyFilter template
Core data plane None Zero changes

9. Milestone 2: Full Feature Parity

Goal: Support all features available on Envoy Gateway, on Istio Gateway.

Features to Add (in suggested order)

Phase 2a: Security & Auth

  • BackendSecurityPolicy → Istio RequestAuthentication + AuthorizationPolicy
  • Client-facing auth (JWT, OIDC, API keys) via Istio policies

Phase 2b: Sidecar Deployment

  • Istio sidecar injection for ext_proc (via annotation-based injection or custom injector)
  • UDS communication (lower latency than network)

Phase 2c: MCPRoute

  • MCP proxy deployment alongside ext_proc
  • EnvoyFilter for MCP-specific routing
  • MCP session management

Phase 2d: InferencePool Integration

  • Gateway API Inference Extension (InferencePool CRD) support
  • Endpoint picker ext_proc integration
  • ORIGINAL_DST cluster configuration via EnvoyFilter

Phase 2e: Rate Limiting & Quotas

  • QuotaPolicy enforcement
  • Token-based rate limiting via Envoy's rate limit filter
  • Dynamic metadata for cost tracking

Phase 2f: Advanced Features

  • Request/response header and body mutations per route
  • Model name overrides
  • Priority-based backend selection
  • Retry/failover across providers

Additional Adapter Work for Each Phase

Phase Istio Resources to Generate
2a RequestAuthentication, AuthorizationPolicy
2b Sidecar injection config, PeerAuthentication
2c Additional EnvoyFilter patches, MCP Service
2d EnvoyFilter for ORIGINAL_DST, EPP cluster config
2e EnvoyFilter for rate limit filter, rate limit Service
2f Extended EnvoyFilter patches

10. Risk Assessment

Low Risk

  • Data plane portability: Already proven — zero EG dependencies
  • ext_proc compatibility: Istio uses Envoy, ext_proc is an Envoy feature
  • filterapi.Config: Explicitly designed to be portable
  • Backend auth: Handled entirely in ext_proc, no gateway dependency

Medium Risk

  • EnvoyFilter stability: Istio considers EnvoyFilter a "break glass" API. Future Istio versions may deprecate or change it. Mitigation: monitor Istio's Gateway API extension efforts; prefer Gateway API standard mechanisms where available.
  • Two ext_proc filter levels: The current architecture uses router-level AND upstream-level ext_proc filters. Istio's EnvoyFilter can inject both, but upstream filter injection requires careful match context configuration.
  • Streaming response handling: The ModeOverride from BUFFERED to STREAMED must work correctly with Istio's Envoy configuration. This should work since it's a standard ext_proc feature, but needs testing.

Higher Risk

  • InferencePool integration: The ORIGINAL_DST cluster type with header-based endpoint selection is an advanced Envoy feature. Configuring this via EnvoyFilter is more fragile than via Envoy Gateway's extension hooks. The Gateway API Inference Extension (inference.networking.k8s.io) is in early stages and may not have Istio support yet.
  • MCP Gateway: The current MCP implementation requires complex listener/route manipulation (separate backend listener on port 10088, credential injection filters, etc.). Replicating this via EnvoyFilter is possible but complex.
  • Feature drift: As the EG adapter and Istio adapter evolve independently, maintaining feature parity requires discipline. A shared test suite against both adapters would mitigate this.

11. Key Files Reference

Core (Portable — No Changes Needed)

File Purpose
internal/extproc/server.go ext_proc gRPC server
internal/extproc/processor_impl.go Router & upstream processor
internal/translator/translator.go Translator interface
internal/translator/openai_*.go Provider-specific translators
internal/filterapi/filterconfig.go Config model (explicitly gateway-agnostic)
internal/filterapi/watcher.go Filesystem config watcher
internal/backendauth/ Provider auth handlers
internal/apischema/ Provider data models
cmd/extproc/mainlib/main.go ext_proc binary startup

Control Plane — Needs Refactoring for Composability

File Current State Proposed Change
internal/controller/gateway.go Builds filterapi.Config + stores in K8s Secret Extract BuildFilterConfig() to shared package
internal/controller/ai_gateway_route.go Creates egv1a1.HTTPRouteFilter Keep as-is (EG adapter only)
internal/controller/backend_security_policy.go Creates egv1a1.SecurityPolicy Keep as-is (EG adapter only)
internal/extensionserver/ Implements EG extension hooks Keep as-is (EG adapter only)

New Code Needed for Istio Adapter

Component Purpose
Config controller Watch AI Gateway CRDs → generate filterapi.Config → store in ConfigMap
EnvoyFilter controller Generate EnvoyFilter CRDs for ext_proc injection
HTTPRoute controller Generate standard HTTPRoute for AI traffic routing
Helm chart ext_proc Deployment, Service, RBAC, EnvoyFilter templates

API Types (Shared — No Changes)

File Purpose
api/v1alpha1/ai_gateway_route.go AIGatewayRoute CRD
api/v1alpha1/ai_service_backend.go AIServiceBackend CRD
api/v1alpha1/backendsecurity_policy.go BackendSecurityPolicy CRD
api/v1alpha1/mcp_route.go MCPRoute CRD
api/v1alpha1/quota_policy.go QuotaPolicy CRD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment