karpathy

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

ShadowPower

import json
import torch
import streamlit as st
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from transformers.generation.utils import GenerationConfig


st.set_page_config(page_title="Baichuan-13B-Chat")
st.title("Baichuan-13B-Chat")

agentzh

[V=browser-hello.mp4] Hi, I'm Matthew from OpenResty Inc. In this video, I'll demonstrate how to implement a "hello world" HTTP interface using OpenResty.

First of all, we make sure we are using OpenResty's nginx.

    [delay=0] $ export PATH=/usr/local/openresty/nginx/sbin:$PATH
    $ which nginx

[S] It's usually in this path.

And then we go to the home directory.

cloudwu

// My version of pimpl (cloudwu@gmail.com)
// See http://en.cppreference.com/w/cpp/language/pimpl

#include <iostream>
 
// interface (widget.h)
class widget {
    struct impl;
public:
    static widget* create(int);  // replacement of new

jdeng

  // generate [0..n-1]
  auto seq = [](size_t n) -> std::vector<size_t> {
      std::vector<size_t> v(n);
      for (size_t i=0; i<n; ++i) v[i] = i;
      return v;
  };
  auto index = seq(n);

  // n * n distance matrix
  std::vector<D> dists(n * n);

brentini

#Auto-load scripts on PowerShell startup

#Add this to C:\Users\%username%\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1

# directory where my scripts are stored

$psdir="D:\Documents\Powershell\Scripts\autoload"  

# load all 'autoload' scripts

diego898

thanks to @neurodyne and this link for updated instructions

This works with the following versions of termsrv.dll

x64 - termsrv.dll - 6.3.9600.17095

Find	Replace
39813C0600000F849E310500	B80001000089813806000090
090085C07F078BD8	090085C090908BD8

jboner

Latency Comparison Numbers (~2012)
----------------------------------
L1 cache reference                           0.5 ns
Branch mispredict                            5   ns
L2 cache reference                           7   ns                      14x L1 cache
Mutex lock/unlock                           25   ns
Main memory reference                      100   ns                      20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy             3,000   ns        3 us
Send 1K bytes over 1 Gbps network       10,000   ns       10 us
Read 4K randomly from SSD*             150,000   ns      150 us          ~1GB/sec SSD