AutoHub Offline Voice Stack — Snapshot (2026-06-15)
The fully-local realtime voice pipeline, with hybrid escalation to cloud for tool/agentic turns. Everything below runs on-device (Apple Silicon) except the cloud-escalation tier.
Pipeline (local-first)
Stage Component Notes VAD Silero local voice-activity detection STT Parakeet (:8178) local speech-to-text; Deepgram is the cloud alt LLM (local) Qwen3.6-35B-A3B 4-bit MoE, ~3B active params, ~20GB; served two ways (below) TTS Chatterbox (hub-managed) local; falls back to ElevenLabs on failure/60s cooldown Escalation Claude Sonnet tool / research / safety / agentic turns route to cloud
Local LLM — two runtimes of the same model
MLX — PRIMARY. mlx_lm.server on :8081, model mlx-community/Qwen3.6-35B-A3B-4bit.
- Faster: ~1.6× TTFT, ~1.4× generation vs Ollama (see benchmark).
- Has an active degeneration guard (cancels repeat-loops mid-stream).
- No tool calling (mlx_lm.server function-calling is unreliable → tool turns escalate to cloud).
- num_predict 96 (tight, voice-tuned). Ollama — FALLBACK. :11434, model huihui_ai/qwen3.6-abliterated:35b-a3b (abliterated/uncensored variant of the same Qwen).
- More robust daemon (stays resident, keep_alive=-1) — the right thing for a fallback.
- Has a native tool loop MLX lacks → unlocks experimental read-only local tools.
- Tuned 2026-06-15 for parity: NUM_PREDICT 256→128, NUM_CTX 8192→16384.
- Still missing the active degeneration guard (relies on repeat_penalty=1.3). Routing — src/voice/local-turn-router.js (hybrid mode):
- Casual chat + personal-memory recall → stay local.
- Tool / messaging / research / safety intent → escalate to cloud (Sonnet).
- Both VOICE_LOCAL_MLX_BYPASS and VOICE_LOCAL_OLLAMA_BYPASS are true; MLX takes precedence, Ollama runs only when MLX isn't ready.
Benchmark — MLX vs Ollama (2026-06-15, matched decoding)
Same model class, 4-bit, 128 max-tok / temp 0.7 / top-p 0.9 / rep 1.1, via the production stream clients: Metric (warm avg) MLX Ollama TTFT (first token) 186 ms 295 ms Generation ~79 tok/s ~58 tok/s Total (short reply) 389 ms 644 ms Cold load (pre-warm) ~4.0 s Quality @ matched decoding identical identical MLX leads on speed; answer quality is a wash (abliteration aside). Gotcha: the two clients' raw tps fields aren't comparable — MLX divides by total wall-clock (incl. prefill), Ollama by eval_duration (decode only). Normalize before comparing.
In progress
Spec: docs/superpowers/specs/2026-06-15-voice-local-cloud-escalation-design.md (branch feat/voice-local-cloud-escalation). Covers: widen tool-intent rules so messaging/contact turns escalate (the voice_communication group already holds the WhatsApp tools — just no rule enabled them); pre-load the group on escalation; confabulation guard in the per-path local prompts; brief spoken "one sec" transition; unify the local history window to ~20 turns; bring Ollama to MLX parity.
Recent reliability fixes (merged)
worktree MCP-config ENOENT (#702), stale tool_use 400s (#703), MCP SSE auto-reconnect (#704), voice audio-route fallback (#705), Slack messages_tab_disabled cascade (#706), agent_run reconciliation (#707), Chatterbox TTS retry-before-cooldown (#714), restart-churn EPIPE guard (#715), voice runtime hardening — crash guard / divergence / barge-in tool preservation (#716).