this script runs steam-run, and nixgl together to launch a llama.cpp server
tutorial:
- Get the latest llama.cpp version. 6/24/25, it is b5749. Replace the
<build>
in 1. with the correct number. wget https://github.com/ggml-org/llama.cpp/releases/download/<build>/llama-<build>-bin-ubuntu-vulkan-x64.zip --output l.zip && unzip l.zip
wget https://huggingface.co/TheBloke/airoboros-mistral2.2-7B-GGUF/blob/main/airoboros-mistral2.2-7b.Q4_K_S.gguf?download=true
(or use a different model)chmod +x run-llama-vulkan.sh
- if you are using vulkan, then just
./run-llama-vulkan.sh
, if you aren't follow further - in line 10, modify nixVulkanIntel to be nixVulkanNvidia.