RuntimeError: Unsloth: The file 'llama.cpp/llama-quantize' or 'llama.cpp/quantize' does not exist. But we expect this file to exist! Maybe the llama.cpp developers changed the name?

llama-quantize_fix.txt

First, clone the llama.cpp repository and follow the steps below:

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
git checkout b3345
git submodule update --init --recursive
make clean
make all -j
git log -1
Credits to Zhangy-ly for this fix.

If it still does not work, then hardcode the llama.cpp folder path which we cloned earlier in the save.py file found at /usr/local/lib/python3.10/dist-packages/unsloth.

Check out the repository for more information
https://github.com/siddhantmedar/FineLlama-3-2-1B-Instruct-Health-QA-Finetune