Skip to content

Instantly share code, notes, and snippets.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@savchenko
savchenko / my-clone
Created October 14, 2024 03:55 — forked from Artefact2/my-clone
My llama.cpp quantize scripts
#!/usr/bin/env sh
URI=$1
BASE=$(basename $1)
[ -f ../models/$BASE-f16.gguf ] && exit 0
(. ../huggingface-cli/bin/activate && HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli download $URI --local-dir ../models/$BASE --cache-dir ../models/$BASE/.hf-cache --exclude 'pytorch_model*' --exclude 'consolidated*' --resume-download) || exit 1
DTYPE=$(jq -r '.torch_dtype' < ../models/$BASE/config.json)