Status: architecture direction, grounded in the current burst-routing-v1 branches
Scope: Grid + Praxis AI + Praxis core + llm-d/EPP + self-hosted and cloud inference providers + shared token state
Primary design rule: Grid decides asynchronously; Praxis executes locally from an immutable snapshot
LoRA / multi-LoRA: deferred
| Demo step | What to do | Where to show it in the UI |
|---|---|---|
| 1. Open the demo | Log in with the demo credentials. | Login page |
| 2. Show the applications | Introduce App 1, App 2, and App 3. | Application budgets |
| 3. Show independent quotas | Point out that each app has its own allocation and usage. | Token allocation governance |
| 4. Send normal traffic | Send a few requests through both consumers. | Live topology and request history |
| 5. Show shared routing | Explain that East and West consumers use the same provider overlay. | Live topology |
| 6. Show token accounting | Explain that each request reserves tokens, then reconciles actual usage. | Application budgets and quota activity |
| 7. Show soft-cap behavior | Exceed an app’s soft allocation. The request still succeeds but is marked over allocation. | Token allocation governance |
| 8. Create GPU pressure | Click Create GPU pressure. | GPU/provider queue pressure |
Based on osc-grid-gap-analysis.md.
“Parity” here means that Praxis Grid has a comparable capability for the inference-routing portion of the OSAC/OSC vision. It does not mean that the implementations or ownership models are identical.
| 00:00:00,000 --> 00:00:05,783 | |
| This demonstration shows one authenticated token budget shared across distributed consumer gateways. | |
| 2 | |
| 00:00:05,783 --> 00:00:11,798 | |
| Alice can enter through either gateway, but both processes enforce the same model quota through Valkey. | |
| 3 | |
| 00:00:11,798 --> 00:00:16,887 |
| Demo: grid-glb-demo | |
| Subcommand: run-grid-glb-demo | |
| Forge config: /home/fedora/praxxis/ai-grid/prs/44-grid-demo-image-source/demos/demos/grid-glb-demo/forge.yaml | |
| Demo root: /home/fedora/praxxis/ai-grid/prs/44-grid-demo-image-source/demos/demos/grid-glb-demo | |
| Grid repo: /home/fedora/praxxis/ai-grid/prs/44-grid-demo-image-source/grid | |
| Compiling xtask v0.1.3 (/home/fedora/praxxis/ai-grid/prs/44-grid-demo-image-source/grid/xtask) | |
| Finished `dev` profile [unoptimized + debuginfo] target(s) in 7.95s | |
| Compiling libc v0.2.186 | |
| Compiling serde_core v1.0.228 |
This primer is for an operator who already has a working Kubernetes cluster on Amazon EC2 and wants to install Praxis Grid using the existing-cluster Helm workflow.
It does not create EC2 instances, build a Kubernetes cluster, configure an
AWS VPC, or install a CNI. It begins with an existing cluster that is reachable
through kubectl.
AI Gateway now has repeatable Kind validation for the core routing flow, including the llm-d-compatible provider-gateway path and native credential-injection path.
/v1/responses, llm-d-compatible routing, operator install/RBAC, and provider access-policy enforcement all have repeatable validation coverage.
The RHOAI 3.6 transition target is not for Praxis to call IPP. It is for Praxis
to replace IPP as Envoy's ExternalProcessor service while preserving the
MaaS body-based routing (BBR) behavior used by RHOAI 3.5. The gaps and
implementation phases in this document are the work required for RHOAI 3.6.
The planned release paths are:
A side-by-side analysis of the llm-d Inference Payload Processor (IPP) and Praxis, covering their architectures, design philosophies, integration models, and how they fit together in the broader llm-d and AI Grid ecosystems.