This assumes you already have AMD Adrenalin installed on the Windows side with up-to-date chipset drivers.
Even if you are not compiling GPU programs, you still need the shared libraries for binaries that dynamically link to them at runtime. For example, running ldd llama-server reports that it requires many libhip*.so and libroc*.so files. If these are not in your LD_LIBRARY_PATH, the program will crash.
Go to https://repo.radeon.com/amdgpu-install and find the latest release for your OS. I'm on Ubuntu 24 (Noble):
wget https://repo.radeon.com/amdgpu-install/7.2.4/ubuntu/noble/amdgpu-install_7.2.4.70204-1_all.deb