Romain D. Artefact2

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggerganov/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

Tests were run in single-user mode, one at a time.
Data was read from and written to tmpfs.
borg-1.1rc2 seems about 10% slower than borg-1.0.11 (many possible reasons, borg-1.1rc2 binary comes from GitHub but 1.0.11 from the Arch repos)
Still making sense of the auto,X,Y compression modes

	#!/usr/bin/env sh

	URI=$1
	BASE=$(basename $1)

	[ -f ../models/$BASE-f16.gguf ] && exit 0

	(. ../huggingface-cli/bin/activate && HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli download $URI --local-dir ../models/$BASE --cache-dir ../models/$BASE/.hf-cache --exclude 'pytorch_model' --exclude 'consolidated' --resume-download) \|\| exit 1

	DTYPE=$(jq -r '.torch_dtype' < ../models/$BASE/config.json)

	#!/usr/bin/env zsh

	if test $# -ne 1; then
	echo "Tests against fake capacity flash storage. Script should print two identical checksums."
	echo "Usage: $0 /dev/sdX" >&2
	exit 1
	fi

	parallel --version >/dev/null 2>&1 \|\| exit 1
	hexdump --version >/dev/null 2>&1 \|\| exit 1

	#!/usr/bin/env php
	<?php
	$p = popen('btrfs-search-metadata file '.escapeshellarg($argv[1]), 'r');
	$logsz = $physz = null;
	$nbins = intval(shell_exec('tput cols')) - 3;
	$bins = array_fill(0, $nbins, 0);
	while(($line = fgets($p)) !== false) {
	$fields = explode(' ', substr($line, 0, -1));
	if($fields[0] === 'inode' && $fields[1] === 'objectid') {
	$logsz = intval($fields[8]);

	-- Override mpv's default volume keybinds with decibel controls
	-- Put this file (dbvol.lua) in ~/.config/mpv/scripts/
	-- WARNING: positive gain values may cause clipping

	-- Author: Romain "Artefact2" Dal Maso <[email protected]>
	-- Released under the WTFPLv2

	function round(n)
	-- https://stackoverflow.com/a/58411671
	return n + (2^52 + 2^51) - (2^52 + 2^51)

	<?php
	/* Author: Romain Dal Maso <[email protected]>
	*
	* This program is free software. It comes without any warranty, to the
	* extent permitted by applicable law. You can redistribute it and/or
	* modify it under the terms of the Do What The Fuck You Want To Public
	* License, Version 2, as published by Sam Hocevar. See
	* http://sam.zoy.org/wtfpl/COPYING for more details. */

	if($argc !== 2) {