Daniele Zonca danielezonca

Design Proposal: Serving-Runtime Catalog for OpenShift AI

RHAISTRAT-1047 -- Enable Unsupported Red Hat Builds of vLLM as Custom Runtimes

1. Executive Summary

This proposal extends the existing model catalog infrastructure to support a new serving-runtime catalog -- a browsable, periodically-refreshed list of Red Hat-built vLLM container images that platform administrators can import as ServingRuntimes in OpenShift AI.

NeMo Guardrails + MCP Gateway Integration

Preliminary design document for securing MCP Gateway traffic with NeMo Guardrails, orchestrated by the TrustyAI operator.

Problem Statement

MCP Gateway processes Model Context Protocol requests (tool calls, prompt fetches) through Envoy. Today, NeMo Guardrails deploys as a standalone service with no awareness of the gateway. Wiring them together requires manual deployment of the

Envoy AI Gateway: Composability Feasibility Analysis

Analysis of making the Envoy AI Gateway composable across different gateway implementations (Envoy Gateway, Istio Gateway, and others).

Repository: envoyproxy/ai-gateway

Envoy AI Gateway: Chat Completion Translation Architecture

An analysis of how the Envoy AI Gateway converts between native third-party AI provider formats and a unified OpenAI-compatible API for chat completion endpoints.

Repository: envoyproxy/ai-gateway Documentation: aigateway.envoyproxy.io

	- kind: bridge
	metadata:
	name: my-bridge

	- kind: source
	metadata:
	name: source-name
	bridge_id: c64kfqla5bkhm18frpdg
	connector_type_id: slack_source_0.1
	connector_spec:

	title Counterfactual (websocket)


	AuditUI->TrustyService:CounterfactualRequest
	activate TrustyService
	note over TrustyService:Add distribution data\n(if available)
	database Kafka
	TrustyService->Kafka:CounterfactualRequest(id)
	deactivate TrustyService

	title Counterfactual (REST)


	AuditUI->TrustyService:CounterfactualRequest
	activate AuditUI
	activate TrustyService
	note over TrustyService:Add distribution data\n(if available)

	TrustyService->ExplainableService:CounterfactualRequest(id)
	activate ExplainableService

	title Counterfactual (message)


	AuditUI->TrustyService:CounterfactualRequest
	activate AuditUI
	activate TrustyService
	note over TrustyService:Add distribution data\n(if available)
	database Kafka
	TrustyService->Kafka:CounterfactualRequest(id)
	AuditUI<-TrustyService:CounterfactualRequest(id)

	2020-12-10T15:10:38.0964400Z [ERROR] testPerturbCurrencyFeature Time elapsed: 0.008 s <<< ERROR!
	2020-12-10T15:10:38.0965451Z java.lang.IllegalArgumentException
	2020-12-10T15:10:38.0975367Z at java.base/java.util.Currency.getInstance(Currency.java:393)
	2020-12-10T15:10:38.0986989Z at org.kie.kogito.explainability.model.TypeTest.testPerturbCurrencyFeature(TypeTest.java:234)
	2020-12-10T15:10:38.0989425Z at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	2020-12-10T15:10:38.0991564Z at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	2020-12-10T15:10:38.1009678Z at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	2020-12-10T15:10:38.1011820Z at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	2020-12-10T15:10:38.1013513Z at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
	2020-12-10T15:10:38.1031735Z at org.junit.jupiter.e