Skip to content

Instantly share code, notes, and snippets.

View rafay-ceoai's full-sized avatar

rafay-ceoai

View GitHub Profile
@thavoc
thavoc / weaver-nvfp4-blackwell-sm120.md
Created July 22, 2026 08:38
Running Weaver (DFlash-TfM) speculative decoding on Blackwell (sm_120) with NVFP4

Running Weaver (DFlash-TfM) speculative decoding on Blackwell (sm_120) with NVFP4

A reproducible recipe for standing up Weaver ("Trees from Marginals", arXiv 2607.06763) speculative decoding on Qwen3.6-27B in NVFP4, using the trymirai/sglang fork, on an NVIDIA Blackwell GPU (compute capability sm_120, e.g. RTX PRO 6000). The paper targets B200; this writeup documents the four changes needed to make it load and run on sm_120, plus the measured tree-budget curve on that hardware.

Status and versions (read before reproducing)