How to build llama.cpp on Ubuntu Mantic

build-llama-cpp.sh

#!/bin/sh
# Build llama.cpp on Ubuntu 23.10
# Tested with `docker run -it --device=/dev/dri --device=/dev/kfd --security-opt seccomp=unconfined --volume $HOME:/mnt/home debian:sid`

apt -y update
apt -y upgrade
apt -y install git hipcc libhipblas-dev librocblas-dev cmake build-essential
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp/
git checkout b2110
CC=clang-15 CXX=clang++-15 cmake -H. -Bbuild -DLLAMA_HIPBLAS=ON -DAMDGPU_TARGETS="gfx803;gfx900;gfx906;gfx908;gfx90a;gfx1010;gfx1030" -DCMAKE_BUILD_TYPE=Release
make -j16 -C build
build/bin/main -ngl 32 --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -m ~/Downloads/dolphin-2.2.1-mistral-7b.Q5_K_M.gguf --prompt "Once upon a time"

@userbox020, you need to apt-get install -y hipcc libhipblas-dev librocblas-dev and use a newer OS and compiler. I suggest FROM ubuntu:mantic and CLANG_VERSION=15. The necessary packages are not available for Debian Bullseye or clang-11. Those are too old.

These instructions are now out of date for Debian 13, which has updated its ROCm stack to use clang-17. It should still work fine if you replace -15 with -17 for clang in the build command.

I also have Ubuntu 24.04 instructions for a newer version of llama.cpp. For the moment, the Debian 13 instructions are the same as Ubuntu 24.04, however, Debian 13 has not released yet so it's possible the instructions may change again (to use clang-18).