Christian Posta christian-posta

Agentgateway Objects context

CEL name	Rust field	When / where it’s set
`request`	`request`	HTTP request (method, uri, host, scheme, path, version, headers, body, startTime, endTime). Set from live request or from `RequestSnapshot`.
`response`	`response`	HTTP response (code, headers, body). Only set when the executor is built with a response (e.g. `new_request_and_response`, `new_response`, or logger with response).
`jwt`	`jwt`	JWT claims. Present when JWT auth is enabled and a token was verified.
`apiKey`	`api_key`	API key claims. Present when API key auth is enabled.

source .env.sh

Create cluster

Based on this documentation: https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/google-gke.html

Create a cluster with CPU nodes and we'll add GPU nodes later

Here's a step-by-step guide to create a cost-conscious Kubernetes cluster in Google Kubernetes Engine (GKE), configure nodes with GPUs, and set up NVIDIA NGC Infrastructure Manager (NIM) along with deploying an LLM that uses the OpenAI API.

Step 1: Prerequisites

Google Cloud Account: Ensure you have an active Google Cloud account.
gcloud CLI: Install the Google Cloud SDK.
kubectl: Install kubectl if it's not already installed.
NVIDIA GPU Driver Support: Ensure you have access to NVIDIA resources and APIs.

Review of service mesh 2019:

In 2019 the common themes for service mesh were:

more service-mesh distributions! everyone in the API/software networking space is coming up with their own distributions of service mesh. I think this is natrually a good thing for the market as it shows there is some value to be provided here and that different approaches should be explored. this will also lead us to a point of convergence soon in the future.
more organizations are POCing service mesh (up from just having architectural discussions from previous year)
usability is key! mesh technology like linkerd has shown how a mesh can be simpler to use and operate, with other mesh technologies taking note and improving their usability

Future of microservices:

Service mesh is happening in large organizations
There are challenges with any new, complicated technology
Determine whether you need a service mesh and what challenges to expect up front

Challenges of adopting a service mesh in an enterprise

I have been fortunate to work closely with enterprises adopting service mesh over the past two years through both my work at Red Hat and now at a startup, Solo.io, that focuses entirely on successful service-mesh adoption. I have seen the progression from "I've never heard of it" to "wow that's cool" to now "yah we're [going to be] doing that". Within the past year, as folks at major enterprises being putting "rubber to the road", I've been at the forefront of the challenges, some expected, some not, that have cropped up as well as how those organizations have chosen to approach their solutions. Adopting service mesh has been coincident with adopting and operating microservices, so there are multiple inter-related challenges.

	apiVersion: gateway.networking.k8s.io/v1
	kind: HTTPRoute
	metadata:
	name: public-oauth-mcp
	namespace: enterprise-agentgateway
	spec:
	parentRefs:
	- name: agentgateway
	rules:
	- matches:

	apiVersion: gateway.networking.k8s.io/v1
	kind: Gateway
	metadata:
	name: agentgateway
	namespace: gloo-system
	spec:
	gatewayClassName: enterprise-agentgateway
	listeners:
	- name: http
	port: 8080

	SOURCE_NAME=$(kubectl get po -A -o wide \| grep $1 \| head -n 1 \| awk '{print $2}')
	TARGET_NAME=$(kubectl get po -A -o wide \| grep $2 \| head -n 1\| awk '{print $2}')

	SOURCE_IP=$(kubectl get po -A -o wide \| grep $1 \| head -n 1 \| awk '{print $7}')
	TARGET_IP=$(kubectl get po -A -o wide \| grep $2 \| head -n 1 \| awk '{print $7}')

	echo "Source: $SOURCE_NAME, Target: $TARGET_NAME"
	echo "Source: $SOURCE_IP, Target: $TARGET_IP"
	echo "Running command: sh -c \"kubectl sniff -i eth0 -o ./local.pcap $SOURCE_NAME -f '((tcp) and (net $TARGET_IP))'\""
	sh -c "kubectl sniff -i eth0 -o ./local.pcap $SOURCE_NAME -f '((tcp) and (net $TARGET_IP))'"

	GLOO=${1:-gloo}

	POD=$(kubectl get po -n gloo-system \| grep $GLOO \| awk '{ print $1 }' \| head -n 1)
	echo "gloo pod to debug '$POD'"

	PF_CMD=$(squashctl --debugger dlv --namespace gloo-system --machine --pod $POD)

	echo "PF CMD: $PF_CMD"

	K_CMD=$(echo "$PF_CMD" \| jq .PortForwardCmd \| sed s/:/2345:/)

	apiVersion: gateway.solo.io/v1
	kind: VirtualService
	metadata:
	name: banking-vs
	namespace: gloo-system
	spec:
	virtualHost:
	domains:
	- 'banking.api.solo.io'
	routes: