Date: 2026-06-14
Project: /home/devops/projects/tts
Runtime: omnivoice.cpp (GGML / llama.cpp stack)
Model: k2-fsa/OmniVoice → GGUF (Serveurperso/OmniVoice-GGUF)
| Item | Value |
|---|---|
| CPU | AMD Ryzen AI 9 HX 370 (24 logical threads, 12 physical used by GGML) |
| GPU | AMD Radeon 890M (gfx1150, Strix APU) |
| Memory | 26 GiB RAM, ~13.5 GiB UMA VRAM |
| OS | Linux 7.0.0-15-generic (Ubuntu) |
| Backend (GPU) | Vulkan0 (RADV STRIX1) |
| Backend (CPU) | GGML CPU (-march=native) |
| File | Size | Role |
|---|---|---|
omnivoice-base-Q8_0.gguf |
626 MB | Qwen3-0.6B backbone (text → audio tokens) |
omnivoice-tokenizer-F32.gguf |
701 MB | HuBERT + DAC codec (tokens → 24 kHz audio) |
cd omnivoice.cpp
cmake -B build -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build -j"$(nproc)"HIP/ROCm (-DGGML_HIP=ON -DGPU_TARGETS=gfx1150) was also tested: LM inference works, but codec decode fails with hipblasSgemm / CUBLAS_STATUS_INTERNAL_ERROR on gfx1150. Vulkan is the recommended backend on this hardware.
- Script:
benchmark-omnivoice.sh - Runs: 3 per case, fixed seed (
--seed 42) - Language: English (
--lang English) - GPU state: Warm (Vulkan shaders compiled; not a cold-start run)
- Metrics:
- RTF (real-time factor) = wall time ÷ estimated audio duration. Lower is better; < 1.0 = faster than realtime.
- Throughput = output audio seconds ÷ wall seconds.
| Case | Text |
|---|---|
| short | Hello world. |
| medium | Hello, this is OmniVoice running locally with GGML Vulkan on AMD Radeon eight ninety M. |
| long | The quick brown fox jumps over the lazy dog. This sentence is repeated to produce a longer utterance for benchmarking synthesis latency and real-time factor across different input lengths on integrated GPU hardware. |
Raw logs: omnivoice.cpp/bench-20260614-103144/
| Backend | Short (~1.6 s audio) | Medium (~5.5 s) | Long (~13.1 s) |
|---|---|---|---|
| Vulkan0 (GPU) | 0.81× RTF (~1.2 s) | 0.65× RTF (~3.4 s) | 0.56× RTF (~7.4 s) |
| CPU | 3.66× RTF (~5.3 s) | 3.49× RTF (~18.4 s) | 3.88× RTF (~51.7 s) |
| GPU speedup vs CPU | 4.5× | 5.3× | 7.0× |
On Vulkan, synthesis is 1.4–1.8× faster than realtime. CPU is ~3.5–3.9× slower than realtime.
| Backend | Case | Total (ms) | RTF | MaskGIT (ms) | Avg step (ms) | Codec (ms) | Output audio (s) |
|---|---|---|---|---|---|---|---|
| Vulkan0 | short | 1,160 | 0.806 | 1,035 | 32.2 | 25.2 | 1.64 |
| Vulkan0 | medium | 3,445 | 0.653 | 2,960 | 92.7 | 103.6 | 5.48 |
| Vulkan0 | long | 7,402 | 0.556 | 6,184 | 192.7 | 262.6 | 13.10 |
| CPU | short | 5,263 | 3.655 | 4,992 | 156.0 | 131.1 | 1.36 |
| CPU | medium | 18,403 | 3.485 | 17,440 | 543.1 | 468.8 | 5.48 |
| CPU | long | 51,666 | 3.879 | 49,309 | 1,541.6 | 1,212.4 | 13.52 |
Variance across the three runs was ±1–2% (Vulkan) and ±1% (CPU).
backend|label|run|status|maskgit_ms|avg_step_ms|generate_ms|codec_ms|total_ms|rtf|audio_s|frames|out_audio_s
Vulkan0|short|1|ok|1038.57|32.46|1145.8|25.7|1171.5|0.814|1.44|36|1.64
Vulkan0|short|2|ok|1036.71|32.40|1132.7|25.8|1158.5|0.805|1.44|36|1.64
Vulkan0|short|3|ok|1030.76|32.21|1127.4|24.1|1151.4|0.800|1.44|36|1.64
Vulkan0|medium|1|ok|2967.18|92.72|3344.3|99.6|3443.9|0.652|5.28|132|5.48
Vulkan0|medium|2|ok|2981.92|93.18|3357.5|99.5|3457.0|0.655|5.28|132|5.48
Vulkan0|medium|3|ok|2931.20|91.60|3323.8|111.7|3435.5|0.651|5.28|132|5.48
Vulkan0|long|1|ok|6158.73|192.46|7115.6|261.2|7376.8|0.554|13.32|333|13.10
Vulkan0|long|2|ok|6190.92|193.47|7144.4|265.9|7410.3|0.556|13.32|333|13.10
Vulkan0|long|3|ok|6201.33|193.79|7159.3|260.7|7420.0|0.557|13.32|333|13.10
CPU|short|1|ok|5009.01|156.53|5147.6|133.1|5280.7|3.667|1.44|36|1.36
CPU|short|2|ok|4935.07|154.22|5073.2|131.0|5204.2|3.614|1.44|36|1.36
CPU|short|3|ok|5032.71|157.27|5173.7|129.1|5302.9|3.683|1.44|36|1.36
CPU|medium|1|ok|17374.10|542.94|17853.1|453.3|18306.4|3.467|5.28|132|5.48
CPU|medium|2|ok|17514.10|547.32|18026.4|467.9|18494.4|3.503|5.28|132|5.48
CPU|medium|3|ok|17432.64|544.77|17922.0|485.2|18407.2|3.486|5.28|132|5.48
CPU|long|1|ok|49051.41|1532.86|50141.7|1193.2|51334.9|3.854|13.32|333|13.52
CPU|long|2|ok|49334.34|1541.70|50504.6|1240.8|51745.4|3.885|13.32|333|13.52
CPU|long|3|ok|49541.63|1548.18|50714.6|1203.2|51917.9|3.898|13.32|333|13.52
Time is dominated by MaskGIT (32 diffusion steps × Qwen3-0.6B forward):
| Case | Vulkan MaskGIT share | Vulkan Codec share |
|---|---|---|
| short | 89% | 2% |
| medium | 86% | 3% |
| long | 84% | 4% |
Codec decode (HuBERT + DAC) stays under 300 ms on GPU even for long text.
| Backend | short | medium | long |
|---|---|---|---|
| Vulkan0 | 1.41× realtime | 1.59× realtime | 1.77× realtime |
| CPU | 0.26× realtime | 0.30× realtime | 0.26× realtime |
| Run | Total | RTF | Notes |
|---|---|---|---|
| First GPU run (cold) | 170,218 ms | 32.2× | Vulkan shader compilation + pipeline warmup |
| Warm GPU (benchmark) | 1,151–7,420 ms | 0.56–0.81× | Normal operating performance |
First launch can take ~3 minutes on this iGPU; subsequent runs are near or faster than realtime.
| Backend | Status | Notes |
|---|---|---|
| Vulkan0 | ✅ Full pipeline | Recommended for Radeon 890M |
| HIP/ROCm (CUDA0) | LM works (~110 ms/step warm); codec decode crashes on gfx1150 | |
| CPU | ✅ Works | 4.5–7× slower than Vulkan; usable fallback |
- Production-ready on this GPU after warmup — RTF 0.56–0.81× (faster than realtime).
- MaskGIT is the bottleneck (~85% of time); codec is cheap on GPU.
- Vulkan beats CPU by 4.5–7×, with the gap widening on longer text.
- HIP is not viable yet on gfx1150 due to hipBLAS failures in the audio codec path.
- Compared to the official PyTorch claim (RTF ~0.025 on high-end NVIDIA), this GGML/Vulkan path on an integrated 890M is ~20–30× slower — expected for this hardware stack.
/home/devops/projects/tts/benchmark-omnivoice.shResults are written under omnivoice.cpp/bench-<timestamp>/.