LM emptymalei

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

Setting Up Your Personal GPU Server: A Step-by-Step Guide

I've been using a GPU workstation with an RTX 4090 for almost a year now, and it's been one of the best decisions I've made. With a personal GPU server, you no longer need to rely on cloud-based GPU instances from services like RunPod or Vast.ai every time you want to run a job or try new models. The best part? No stress about recurring GPU instance costs! :-)

However, I rarely work directly on my workstation. Instead, I prefer the flexibility of accessing the GPU remotely using my MacBook, whether I'm working from different locations within my home, from a co-working space, or a cozy cafe in another part of town.

In this blog, I will walk you through the steps to configure a personal GPU Ubuntu server.

For this guide, I assume you already have a workstation running Ubuntu with a GPU and it is connected to your local network

How to install CUDA / cuDNN / TensorRT on Ubuntu

Install NVIDIA drivers

Update & upgrade

sudo apt update && sudo apt upgrade

	# This is the workround before https://github.com/pydantic/pydantic-ai/issues/224 is fixed
	from pydantic_ai.models.openai import (
	OpenAIModel,
	OpenAIModelSettings,
	ModelRequestParameters,
	)

	from itertools import chain
	from openai import NOT_GIVEN, AsyncOpenAI, AsyncStream
	from openai.types import ChatModel, chat

	import numpy as np
	import torch


	class MyModule(torch.nn.Module):

	def add_param(self, key, shape):
	val = torch.randn(*shape)/np.prod(shape)
	setattr(self, key, torch.nn.Parameter(val))

	# First, some imports:
	import numpy as np
	from darts.utils import timeseries_generation as tg

	np.random.seed(42)

	LENGTH = 3 * 365 # 3 years of daily data

	# Melting: a sine with yearly periodicity and additive white noise
	melting = (tg.sine_timeseries(length=LENGTH,

	import s3fs
	import pyarrow as pa
	import pyarrow.parquet as pq


	from itertools import chain
	from typing import Tuple, Any

	def iter_parquet(s3_uri: str, columns = None, batch_size=1_000) -> Tuple[Any]:

	#!/bin/bash

	set -e

	sudo -u ec2-user -i <<'EOF'

	ENVIRONMENT=python38
	VERSION=3.8

	conda create -y -n "$ENVIRONMENT" python="$VERSION" tensorflow-gpu numpy opencv pandas pyyaml

	"""
	WARNING: dont use loguru, use structlog
	https://gist.github.com/nkhitrov/38adbb314f0d35371eba4ffb8f27078f


	Configure handlers and formats for application loggers.
	"""
	import logging
	import sys
	from pprint import pformat

	#include <stdint.h>
	#include <stdio.h>
	#include <stdlib.h>

	// munged from https://github.com/simontime/Resead

	namespace sead
	{
	class Random
	{