Nisim Hurst-Tarrab langheran

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

Visual Studio Code as Diff Tool

Comparing files using the command line Using Visual Studio Code

"%LOCALAPPDATA%\Programs\Microsoft VS Code\code.exe" --diff file1.cs file2.cs

Using Visual Studio Code Insiders

Log into your server with ssh, something like

ssh -i "my_secret.pem" ubuntu@12.123.12.123

If it's a new server, you'll need to install a few things.

Install conda with

	from typing import Union, Dict
	from io import BytesIO, StringIO
	import json
	import pandas as pd
	import requests
	from pydrive2.auth import GoogleAuth
	from pydrive2.drive import GoogleDrive

	def read_private_file_from_gdrive(
	file_url: str, file_format: str, google_auth: GoogleAuth, **kwargs

	<Configuration>
	<Add OfficeClientEdition="64" Channel="PerpetualVL2019" ForceUpgrade="TRUE">
	<Product ID="ProPlus2019Volume" PIDKEY="NMMKJ-6RK4F-KMJVX-8D9MJ-6MWKP">
	<Language ID="MatchOS" />
	<!-- If you need , comment it. -->
	<!-- <ExcludeApp ID="Word" /> -->
	<!-- <ExcludeApp ID="Excel" /> -->
	<!-- <ExcludeApp ID="PowerPoint" /> -->
	<ExcludeApp ID="Access" />
	<ExcludeApp ID="Groove" />

	(() => {
	let count = 0;

	function getAllButtons() {
	return document.querySelectorAll('button.is-following') \|\| [];
	}

	async function unfollowAll() {
	const buttons = getAllButtons();

	(() => {
	let count = 0;

	function getAllButtons() {
	return document.querySelectorAll('button.is-following') \|\| [];
	}

	async function unfollowAll() {
	const buttons = getAllButtons();

	import csv
	import numpy as np
	from collections import Counter
	from nltk.corpus import brown
	from mittens import GloVe, Mittens
	from sklearn.feature_extraction import stop_words
	from sklearn.feature_extraction.text import CountVectorizer


	def glove2dict(glove_filename):

	@startuml Kanban Board

	caption "Kanban board for 2019.12.19"


	rectangle stack_TODO as "To-do" {

	node Task3 [
	Task Name
	====

	# Use Pandas & SQLAlchemy.
	# https://stackoverflow.com/questions/23103962/how-to-write-dataframe-to-postgres-table

	# Note this will create a new table; see the 'faster option' at the above link for a method using 'copy' to an existing table.
	# However, 'copy' cannot do an upsert; that requires inserting to a temp table, then upserting form temp table to destination table.

	# This will lack PKs and FKs and indexes, of course, so if used naively you may see data duplication.

	# Also the df.to_sql command can do an append (but not upsert), using the if_exists param:
	# https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_sql.html