The following instructions refer to gemma-3-1b-it-qat-q4_0 and Ubuntu Linux 24.04.3 LTS but can be applied to other
Gemma 3 IT QAT checkpoints as well.
To quantize the original safetensors snapshot, we need llama.cpp and a few helper tools that can be obtained together
with llama.cpp. To make sure we are on the latest version, we first clone and build the newest version of llama.cpp.
sudo apt-get install -y --no-install-recommends \