Smoke test: 120B endpoint on gx10-83fb (Qwen 122B)

SMOKE_TEST_120B_2026-03-13.md

Smoke test: 120B endpoint on gx10-83fb

Date: 2026-03-13
Endpoint: http://gx10-83fb.tail3dac72.ts.net:8002
Service: Qwen 122B A10B (llama.cpp) via systemd user unit qwen-122b

---

Deployment summary

• llama.cpp: Built with CUDA (GGML_CUDA=ON) at /home/lou/llama.cpp/build/ on gx10.
• Model: Unsloth Qwen3.5-122B-A10B-GGUF Q4_K_M (3-part), installed at /home/lou/models/qwen122b/Q4_K_M/.
• Systemd: User unit qwen-122b.service in /home/lou/.config/systemd/user/; start/swap via /home/lou/bin/swap-llm.sh qwen122b.
• Port: 8002 (bind 0.0.0.0).

---

Smoke test results (2026-03-13)

1. GET /v1/models

Request: curl -s http://gx10-83fb.tail3dac72.ts.net:8002/v1/models

Result: HTTP 200

Response (excerpt):

{"models":[{"name":"Qwen3.5-122B-A10B-Q4_K_M-00001-of-00003.gguf","model":"Qwen3.5-122B-A10B-Q4_K_M-00001-of-00003.gguf",...}],"object":"list",...}

Model id for clients: Qwen3.5-122B-A10B-Q4_K_M-00001-of-00003.gguf

---

2. POST /v1/chat/completions

Request:

curl -s -X POST http://gx10-83fb.tail3dac72.ts.net:8002/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen3.5-122B-A10B-Q4_K_M","messages":[{"role":"user","content":"Say exactly: smoke test ok"}],"max_tokens":32}'

Result: HTTP 200. Completion returned with choices[0].message (model uses reasoning_content for chain-of-thought; content may be empty for short replies).

Response (excerpt):

{"choices":[{"finish_reason":"length","index":0,"message":{"role":"assistant","content":"","reasoning_content":"Thinking Process:..."}}],"created":1773414610,"model":"Qwen3.5-122B-A10B-Q4_K_M-00001-of-00003.gguf",...}

---

3. Benchmark: output tokens per second (2026-03-17)

Script: scripts/benchmark_120b_tokens_per_second.py
Interpretation: docs/BENCHMARK_120B.md — good/better/best bands and bench.md Gist.

Command: python3 scripts/benchmark_120b_tokens_per_second.py
Extended: --runs 5, --ttft, --concurrent 2 (see --help).

Result (single run):

Metric	Value
Prompt tokens	33
Completion tokens	256
Wall-clock time	14.78 s
Output tokens/s	17.33

Re-run anytime to refresh; throughput may vary with load and thermal state.

---

Conclusion

• Deployment: Complete on gx10 (llama.cpp, model, user systemd unit, swap script).
• Smoke test: GET /v1/models and POST /v1/chat/completions both return 200 and valid JSON; the 120B endpoint is operational.
• Goose: Custom provider qwen_122b.json uses base_url http://gx10-83fb.tail3dac72.ts.net:8002 and model id Qwen3.5-122B-A10B-Q4_K_M-00001-of-00003.gguf; supports_streaming: false.