Epic: INFERENG-9476 From: Doug Smith Date: 2026-07-29
An epic to build a streaming midstream release pipeline for vLLM. Instead of only producing midstream images during zero-day release events, we want continuously up-to-date midstream builds — nightlies, release branch builds, and release tag builds — all the time.
Think OpenShift Origin: a continuously marching open-source distribution built the Red Hat way.
The #1 pain point: every time someone asks "do you have a Red Hat build of vLLM 0.X.X?" we're scrambling. A streaming release means the answer is always "yes." It also catches midstream integration issues early instead of during release crunch.
| # | Story | Key | What |
|---|---|---|---|
| 1 | Green nightlies | INFERENG-9477 | Fix nightly-build-whl-image.yml so it's reliably green. This is the foundation. |
| 2 | Release branch/tag triggers | INFERENG-9478 | Extend beyond nightly cron — trigger on upstream branch cuts and release tags. |
| 3 | Agentic nightly builds | INFERENG-9479 | Pick up Doug's Phase 1 PoC (INFERENG-6206). Autonomous merge + build + validate + report. |
| 4 | Agentic release branch management | INFERENG-9480 | Automate midstream release branch creation when upstream cuts a branch. |
| # | Story | Key | What |
|---|---|---|---|
| 5 | Public midstream fork | INFERENG-9481 | OpenShift Origin model — public repo for Red Hat's opinionated vLLM build. Needs stakeholder input. |
| 6 | Streaming downstream release | INFERENG-9482 | Continuous builds on the official Red Hat container registry. Needs AIPCC pipeline integration. |
- Nightly workflow (starting point):
nightly-build-whl-image.yml— merge upstream + build 4 configs (CUDA/ROCm × regular/merged) - Midstream build skill (agentic instructions):
midstream-build-from-upstream— this is the playbook for doing a midstream build - Agentic nightly PoC branches:
dougbtv/agentic-nightlyon nm-cicd and rhaiis-midstream-snippets - Phase 1 demo gist: https://gist.github.com/dougbtv/0bd47904c6bbe5a48ed1bad2ef35ba97
| Key | What | Why it matters |
|---|---|---|
| INFERENG-6206 | Agentic nightly build PoC | Doug's Phase 1 — autonomous merge proven, build validation next |
| INFERENG-3581 | Original nightly build concept | History and context |
| INFERENG-8134 | Tahmid: autofix bot on nm-vllm-ent | Agentic CI patterns to align with |
| INFERENG-8832 | Tahmid/Matt: autofix bot on nm-cicd | You're already on this — good context |
| INFERENG-9086 | Tahmid: cross-repo PR build dispatch | Complementary plumbing (nm-vllm-ent PR → nm-cicd build) |
| INFERENG-8980 | Tahmid: AIPCC repo prep for 3.5-GA | The manual release process to eventually automate |
| INFERENG-8929 | AI Agent Initiative: vLLM-Omni | Related epic — streaming releases are foundational for this |
- Tahmid Muttaki — agentic CI patterns (autofix bot), cross-repo dispatch, and deep knowledge of the AIPCC release branching process. You're already working with him on INFERENG-8832. His work on INFERENG-9086 (cross-repo build dispatch) is directly relevant plumbing.
- Doug Smith — PoC author, context on the vision and prior art. Ping me anytime.
- Start with Story 1 (green nightlies). Dispatch the nightly workflow manually, read the logs, figure out what's broken, fix it. This gives you deep context on the merge + build pipeline.
- Story 2 (release triggers) can overlap. While nightlies are stabilizing, you can research how to detect upstream branch cuts/tags.
- Story 3 (agentic nightlies) is where it gets fun. Read through the Phase 1 PoC comments on INFERENG-6206 and the midstream build skill. Coordinate with Tahmid on whether to reuse autofix patterns or build something parallel.
- Stories 5-6 (stretch) are opinion-forming. Read, think, talk to stakeholders, write a proposal. No code needed initially.
Form your own opinions here. The stories have acceptance criteria, but the how is yours to figure out. If you think a story should be split, reordered, or scrapped — say so. This is your epic now.
Generated 2026-07-29 by Doug + Claude