How to sync the midstream fork (neuralmagic/nm-vllm-omni-ent) with upstream (vllm-project/vllm-omni) when main is branch-protected and force-push is not allowed.
This follows the same merge-based pattern used by neuralmagic/nm-vllm-ent (the vLLM midstream fork).
# Ensure upstream remote exists
flowchart TD
subgraph upstream["Upstream"]
A["vllm-project/vllm-omni<br/><i>main branch</i>"]
end
subgraph midstream["Midstream Fork — neuralmagic/nm-vllm-omni-ent"]
B["Rebase onto upstream main"]
C["Apply carries"]
D["Tag release<br/><b>v0.21.0+rhaiv.3</b>"]
Main is the recommended release branch. It contains 11 critical-to-high severity bugfixes for target models (Qwen3-Omni, Flux 2, Qwen-TTS, Voxtral-TTS) that are absent from v0.21.0, including a Qwen3-TTS prefix cache corruption bug and a cross-request state leak that corrupts concurrent voice clone output. While main carries higher structural risk from two upstream vLLM rebases (0.22.0 and 0.23.0) and two major refactors (TTS adapter framework, Output Processor Phase 2), the data-correctness bugs on v0.21.0 are more dangerous to production deployments than the integration risk on main. Both branches share the same set of unresolved upstream bugs.
| # vLLM on DGX Spark (GB10) — Compatibility Issues Report | |
| **Date:** 2026-04-27 | |
| **Host:** `octo-et-spark-2.khw.eng.rdu2.dc.redhat.com` | |
| **Image:** `quay.io/aipcc/rhaiis/cuda-ubi9:3.4` | |
| **Model tested:** `RedHatAI/Qwen3-Next-80B-A3B-Instruct-quantized.w4a16` | |
| --- | |
| ## Hardware Summary |
| Spin up the AWS EC2 instance: | |
| ```# 1. Create a secure Key Pair and save the private key | |
| aws ec2 create-key-pair --key-name rnoriega --query 'KeyMaterial' --output text > rnoriega.pem | |
| chmod 400 rnoriega.pem | |
| # 2. Find a Subnet in an Availability Zone that supports G5 hardware | |
| SUBNET_ID=$(aws ec2 describe-subnets --filters "Name=availability-zone,Values=$(aws ec2 describe-instance-type-offerings --location-type availability-zone --filters Name=instance-type,Values=g5.xlarge --query 'InstanceTypeOfferings[0].Location' --output text)" --query 'Subnets[0].SubnetId' --output text) | |
| # 3. Get the 'default' Security Group for that specific Subnet's VPC |
| podman run --rm -it --device nvidia.com/gpu=all -p 8000:8000 --ipc=host --env "HF_TOKEN=xxxx" | |
| gdCcZlrQoK" --env "HF_HUB_OFFLINE=0" -v ~/.cache/vllm:/home/vllm/.cache --name=vllm quay.io/redhat-user-workloads/octo-edge-tenant/vllm-validated-for- | |
| nvidia-jetsons:vllm-validated-for-nvidia-jetsons-on-push-jf6fd-build-images --gpu-memory-utilization 0.7 --enforce-eager --model RedHatAI/gemma-3-1b-i | |
| t-quantized.w8a8 | |
| (APIServer pid=1) INFO 11-04 14:23:48 [api_server.py:1839] vLLM API server version 0.11.0+rhai1 | |
| (APIServer pid=1) INFO 11-04 14:23:48 [utils.py:233] non-default args: {'model': 'RedHatAI/gemma-3-1b-it-quantized.w8a8', 'enforce_eager': True, 'gpu_ | |
| memory_utilization': 0.7} | |
| config.json: 1.95kB [00:00, 5.72MB/s] | |
| (APIServer pid=1) INFO 11-04 14:24:07 [model.py:547] Resolved architecture: Gemma3ForCausalLM |
| FROM docker.io/library/ubuntu:22.04 AS build | |
| RUN bash <<EOF | |
| set -euxo pipefail | |
| apt-get update | |
| apt-get -y install curl | |
| curl -O https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/arm64/cuda-keyring_1.1-1_all.deb | |
| dpkg -i cuda-keyring_1.1-1_all.deb | |
| apt-get update | |
| apt-get -y install \ |
To get started with either the Mini PCIe or M.2 Accelerator, all you need to do is connect the card to your system, and then install our PCIe driver, Edge TPU runtime, and the TensorFlow Lite runtime. This page walks you through the setup and shows you how to run an example model.
The setup and operation is the same for both M.2 form-factors, including the M.2 Accelerator with Dual Edge TPU.
Raspberry Pi OS (64-bit) based on Debian 10 or newer
To get started with either the Mini PCIe or M.2 Accelerator, all you need to do is connect the card to your system, and then install our PCIe driver, Edge TPU runtime, and the TensorFlow Lite runtime. This page walks you through the setup and shows you how to run an example model.
The setup and operation is the same for both M.2 form-factors, including the M.2 Accelerator with Dual Edge TPU.
Raspberry Pi OS (64-bit) based on Debian 10 or newer
| FROM quay.io/centos-bootc/fedora-bootc:eln | |
| COPY rpmbuild/RPMS/x86_64/flightctl-agent-0.0.1-1.el9.x86_64.rpm /tmp/ | |
| COPY flightctl_rsa.pub /usr/etc-system/root.keys | |
| RUN touch /etc/ssh/sshd_config.d/30-auth-system.conf; \ | |
| mkdir -p /usr/etc-system/; \ | |
| echo 'AuthorizedKeysFile /usr/etc-system/%u.keys' >> /etc/ssh/sshd_config.d/30-auth-system.conf; \ | |
| chmod 0600 /usr/etc-system/root.keys | |
| VOLUME /var/roothome |