timinar

Running GLM-5.2 (753B DSA MoE) on 8× A100 80GB with vLLM

TL;DR. GLM-5.2 (glm_moe_dsa — DeepSeek Sparse Attention) does not run on Ampere (A100, sm_80) with stock vLLM: the sparse-MLA attention backend (FLASHMLA_SPARSE) and the lightning-indexer's fp8_mqa_logits (DeepGEMM) are Hopper/Blackwell-only. vLLM PR #38476 (issue #38006) adds a Triton sparse-MLA backend (TRITON_MLA_SPARSE) + a bf16 Triton indexer fallback that run on Ampere. Cherry-picking it onto current main is a Python-only change — no CUDA recompile. Result: GLM-5.2 AWQ-INT4 serves on 8× A100 at ~56 tok/s single-stream and ~625 tok/s aggregate decode (32-way), with coherent output.

This is an independent 8× A100 confirmation of PR #38476 (the author validated on 32× A100), plus a no-recompile install note. Credit to @haosdent for the PR.

---

Requirements

• 8× A100 80GB (sm_80). ~410 GiB VRAM used at TP=8, so all 8 GPUs.

the-spyke

Enable PipeWire on Ubuntu 22.04

This guide is only for original Ubuntu out-of-the-box packages. If you have added a custom PPA like pipewire-debian, you might get into conflicts.

Ubuntu 22.04 has PipeWire partially installed and enabled as it's used by browsers (WebRTC) for recoding the screeen under Wayland. We can enable remaining parts and use PipeWire for audio and Bluetooth instead of PulseAudio.

Starting from WirePlumber version 0.4.8 automatic Bluetooth profile switching (e.g. switching from A2DP to HSP/HFP when an application needs microphone access) is supported. Jammy (22.04) repos provide exactly version 0.4.8. So, we're good.

Based on Debian Wiki, but simplified for Ubuntu 22.04.