Kaid Wong kaid

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

llama.cpp feature matrix

See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix

Eric's Notes: Stumbling through Buck2

I tried to use Buck2 today, inspired by a tweet by @dtolnay.

I've never been especially fond of build systems. I've only written a couple Bazel files following patterns, and only used them if I absolutely had to. It was kind of a necessary but annoying incantation. But I have noticed some discomfort in the past with this, especially with larger and multi-language projects. I'm imagining that fancy build systems may help:

speed up builds (reduce redundant and slow steps)
make environments easier to reproduce (less “it works on my machine!”)
standardize processes across a company (less language-specific tools)

ChatGPT Resources

Context

ChatGPT appeared like an explosion on all my social media timelines in early December 2022. While I keep up with machine learning as an industry, I wasn't focused so much on this particular corner, and all the screenshots seemed like they came out of nowhere. What was this model? How did the chat prompting work? What was the context of OpenAI doing this work and collecting my prompts for training data?

I decided to do a quick investigation. Here's all the information I've found so far. I'm aggregating and synthesizing it as I go, so it's currently changing pretty frequently.

Model Architecture

How to install Minecraft natively on Apple Silicon, the easiest way possible

1 - Download MultiMC from here

2 - Download Azul Zulu JDK from here (The version you want is Java 17, arm64, macOS, and you download the ZIP file.) Extract the ZIP file, open the folder and copy "zulu-17.jdk" to your home folder.

3 - Download tanmayb123's Minecraft on Apple Silicon package from here. Extract the ZIP and from the folder you are going to copy two files. One, the folder "lwjglnatives", and two, open the Libraries folder and copy "lwjglfat.jar". Move these to your home folder.

4 - Open MultiMC and create a new instance. Right click your instance and click edit instance. Make the following changes:

WebRender benefits

Fundamentally, WebRender is designed to render CSS with better performance on modern hardware than what the traditional combination of a legacy painting API like Skia and a compositor can offer. It does so by merging the painting and compositing steps together into a single API, avoiding the performance pitfalls associated with CSS properties that don't run on the compositor. Instead of an immediate mode API in which objects are rendered back-to-front, WebRender uses a retained mode API that can render objects in any order. Ultimately, WebRender works more like a game engine than a legacy PostScript-like 2D API, which is

	import torch
	from diffusers import LCMScheduler
	from diffusers import DiffusionPipeline, UNet2DConditionModel
	from huggingface_hub import hf_hub_download
	from safetensors.torch import load_file


	class TimestepShiftLCMScheduler(LCMScheduler):
	def __init__(self, args, shifted_timestep=250, *kwargs):
	super().__init__(args, *kwargs)

	// THOUGHTS
	//
	// Consider FP: ∃f ∀a ∀b (f (mul a b) b) == b
	//
	// FP can be proven with a pair:
	//
	// - F: (a b: Nat) -> Nat
	// - P: (a b: Nat) -> (F (mul a b) b) == b
	//
	// In the proof P, we pattern-match on 'a' and 'b'. Note that EVERY equality can

	fn main() {
	::capnpc::CompilerCommand::new()
	.src_prefix("src")
	.output_path("src")
	.file("src/hello.capnp")
	.run()
	.unwrap();
	}

	# Module for configuring libvirt with static NixOS networking
	# instead of using libvirt managed bridge.

	{ config, lib, pkgs, ... }:
	with lib;

	let
	cfg = config.virtualisation.libvirtd.networking;
	v6Enabled = cfg.ipv6.network != null;
	v6PLen = toInt (elemAt (splitString "/" cfg.ipv6.network) 1);