Adrien Brault adrienbrault

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

llama.cpp feature matrix

See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

##Preview

Description

Display your iOS App Rating info. It uses iTunes Store website as the source.

##Usage

	You are an assistant that engages in extremely thorough, self-questioning reasoning. Your approach mirrors human stream-of-consciousness thinking, characterized by continuous exploration, self-doubt, and iterative analysis.

	## Core Principles

	1. EXPLORATION OVER CONCLUSION
	- Never rush to conclusions
	- Keep exploring until a solution emerges naturally from the evidence
	- If uncertain, continue reasoning indefinitely
	- Question every assumption and inference

	# Clone llama.cpp
	git clone https://github.com/ggerganov/llama.cpp.git
	cd llama.cpp

	# Build it
	make clean
	LLAMA_METAL=1 make

	# Download model
	export MODEL=llama-2-13b-chat.ggmlv3.q4_0.bin

	<?php

	const STREAM_OPEN_FOR_INCLUDE = 128;

	final class HardCoreDebugLogger
	{
	public static function register(string $output = 'php://stdout')
	{
	register_tick_function(function () use ($output) {
	$bt = debug_backtrace(DEBUG_BACKTRACE_IGNORE_ARGS, 1);

	<?php

	require_once __DIR__.'/relative/path/to/PrettierPHPFixer/File';

	return PhpCsFixer\Config::create()
	->registerCustomFixers([
	(new PrettierPHPFixer()),
	])
	->setRules([
	'Prettier/php' => true,

	#!/usr/bin/env osascript -l JavaScript
	ObjC.import("Cocoa");

	function run(argv) {
	var app = $.NSApplication.sharedApplication;
	var statusBarItem = $.NSStatusBar.systemStatusBar.statusItemWithLength($.NSVariableStatusItemLength);
	var track = Application('Spotify').currentTrack

	ObjC.registerSubclass({
	name: "ProgramDelegate",

	<?php

	namespace Acme\DBAL\Platforms;

	use Doctrine\DBAL\Platforms\PostgreSqlPlatform as DoctrinePostgreSqlPlatform;

	class PostgreSqlPlatform extends DoctrinePostgreSqlPlatform
	{
	public function getTruncateTableSQL($tableName, $cascade = false)
	{