Adeel Ahmad adeelahmad

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

After pulling everything together, I thought it would be a good idea to document what ended up working for me with the following setup:

*darr apps
Some 3d printers
MobilRaker
NZB360

This stack requires the following

Protecting the sites from unauthorized access

macOS Live Text has a very good quality/speed tradeoff.

Compared to Tesseract, it has much higher quality and is up to 3x as fast.

Finetune instruct that can enhance LLM capabilities and safety.

flowchart
	StartScriptExecution["Start Script Execution"]
	ReadTextFile["Read Text File"]
	ChunkTextIntoSegments["Chunk Text Into Segments"]
	ForEachTextChunk["For Each Text Chunk"]
	PerformNamedEntityRecognition["Perform Named Entity Recognition (NER)"]

Tokenizer Notes

Praxis Maldevide - Draft A

Introduction

This document is a collection of thoughts and observations about the tokenizers used in llama-rooted large language models.

The Tokenizer

Most language models use the LlamaTokenizer.

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

llama.cpp feature matrix

See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix

	import os
	import re
	from copy import deepcopy
	from functools import cache
	from typing import Any, AsyncIterator, Protocol, cast

	from mistralai import Mistral
	from openai import AsyncOpenAI, OpenAI

	from unmute.kyutai_constants import LLM_SERVER

	[TASK(s)]

	1.

	[INPUT]


	Design a chatbot interface where, instead of responding with plain text, the AI provides action buttons based on the user’s message. The chatbot should analyze the user input and dynamically generate relevant options as clickable buttons.

	Key Features:

	"""
	Conversation prompt templates.

	We kindly request that you import fastchat instead of copying this file if you wish to use it.
	If you have any changes in mind, please contribute back so the community can benefit collectively and continue to maintain these valuable templates.
	"""

	import base64
	import dataclasses
	from enum import auto, IntEnum