nerdalert

Failed attempts at HTTPRoute Host Header Filter Removal

Background

The ExternalModel reconciler creates an HTTPRoute with a RequestHeaderModifier filter that sets Host: <provider-endpoint> (e.g., Host: api.openai.com). This filter was questioned during PR #709 review — could BBR handle this instead, since BBR already handles path rewriting and API key injection?

We investigated whether the filter could be moved into BBR's ext-proc pipeline so that all request mutations happen in a single place.

Attempts

nerdalert

ExternalModel Deployment & Validation Guide

Prerequisites

• OpenShift cluster with oc/kubectl access as cluster-admin
• MaaS repo cloned (e.g., ~/istio-gw/prs/4-rconciler-namespace-path/models-as-a-service)

Step 1: Deploy MaaS with ODH Operator

nerdalert

External Model Validation

#!/bin/bash
GW_HOST=$(kubectl get gateway maas-default-gateway -n openshift-ingress -o jsonpath='{.spec.listeners[0].hostname}')
TOKEN=$(oc whoami -t)

echo "Gateway: $GW_HOST"
echo ""

nerdalert

Validation of PR ns prefix to ExternalModel HTTPRoute path for llmisvc parity

PR #709 - re-validation and updated as of 4/11 12:06AM EST

$ HOST="https://maas.$(kubectl get ingress.config.openshift.io/cluster -o jsonpath='{.spec.domain}')"
  TOKEN=$(oc whoami -t)

$ API_KEY=$(curl -sSk -X POST "$HOST/maas-api/v1/api-keys" \
    -H "Authorization: Bearer $TOKEN" \

nerdalert

Multi-Turn BBR+MaaS Validation Output

• Anthropic multi-turn validation (the errors at turn 8 are TRLP and expected as part of the test)

HOST="https://maas.$(kubectl get ingress.config.openshift.io/cluster -o jsonpath='{.spec.domain}')"
TOKEN=$(oc whoami -t)

nerdalert

MaaS PR662 Install Log

$ ./scripts/deploy.sh --operator-type odh
[INFO] ===================================================
[INFO]   Models-as-a-Service Deployment
[INFO] ===================================================
[INFO] Validating configuration...
[INFO] Configuration validated successfully

nerdalert

MaaS API RBAC Fix

Root Cause

Breakage when deploying MaaS with: ./scripts/deploy.sh --operator-type odh

maas-api pods crash with CrashLoopBackOff because the opendatahub:maas-api service account lacks:

1. Permission to read the maas-db-config secret in opendatahub namespace
2. Permission to list maasmodelrefs and maassubscriptions CRDs

nerdalert

TLS Skip

tlsInsecureSkipVerify to ExternalModel spec validation with PR: opendatahub-io/models-as-a-service#646

• Commands to validate:

# Discover gateway
  HOST=$(kubectl get maasmodelref facebook-opt-125m-simulated -n llm \
    -o jsonpath='{.status.endpoint}' | sed -E 's#(https://[^/]+).*#\1#')

nerdalert

MaaS Istio External Mode Routing Validation

All three models (local + OpenAI + Anthropic) work through the MaaS gateway using the same sk-oai-* API key minted via the MaaS API.

Demo: External Model Routing with Istio ServiceEntry & DestinationRule

I didn't add the model listing to this validation but you can see an example modifications to MaaS required in egress-ai-gateway-poc/patches/maas-api-external-model-listing.patch. This patch adds ConfigMap-based external model listing to the MaaS API — it reads from an external-model-registry ConfigMap in the MaaS namespace and merges those models into the GET /v1/models response. I have tested that a couple of weeks ago with ghcr.io/nerdalert/maas-api:external-models.

Environment

nerdalert

$ ./scripts/validate.sh  all
Discovering gateway address...
  Found LoadBalancer hostname: http://a38603e70f1d34daa841061646a16427-402819449.us-east-1.elb.amazonaws.com

==========================================
  Iteration 1: httpbin.org (no auth)
==========================================

Resources: