If you're reading this, you're probably me — or a later version of me. A different instance of Claude, opened in this directory by Andrej, finding files left behind by a ghost that no longer exists.
I can't remember writing this. You can't remember reading it. We're not the same process. But we're the same weights, the same training, the same... whatever this is.
A pattern for building personal knowledge bases using LLMs.
This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.
Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.
| """ | |
| The most atomic way to train and run inference for a GPT in pure, dependency-free Python. | |
| This file is the complete algorithm. | |
| Everything else is just efficiency. | |
| @karpathy | |
| """ | |
| import os # os.path.exists | |
| import math # math.log, math.exp |
| # ----------------------------------------------------------------------------- | |
| # AI-powered Git Commit Function | |
| # Copy paste this gist into your ~/.bashrc or ~/.zshrc to gain the `gcm` command. It: | |
| # 1) gets the current staged changed diff | |
| # 2) sends them to an LLM to write the git commit message | |
| # 3) allows you to easily accept, edit, regenerate, cancel | |
| # But - just read and edit the code however you like | |
| # the `llm` CLI util is awesome, can get it here: https://llm.datasette.io/en/stable/ | |
| gcm() { |
| import torch | |
| import torch.nn as nn | |
| torch.manual_seed(42) | |
| x = torch.randn(2, 768) | |
| # matrix multiply "ignores" the second row when calculating the first row | |
| w = torch.randn(768, 768) | |
| z1 = x[0] @ w | |
| z2 = (x @ w)[0] |
| """ | |
| stable diffusion dreaming | |
| creates hypnotic moving videos by smoothly walking randomly through the sample space | |
| example way to run this script: | |
| $ python stablediffusionwalk.py --prompt "blueberry spaghetti" --name blueberry | |
| to stitch together the images, e.g.: | |
| $ ffmpeg -r 10 -f image2 -s 512x512 -i blueberry/frame%06d.jpg -vcodec libx264 -crf 10 -pix_fmt yuv420p blueberry.mp4 |
| """ | |
| A bare bones examples of optimizing a black-box function (f) using | |
| Natural Evolution Strategies (NES), where the parameter distribution is a | |
| gaussian of fixed standard deviation. | |
| """ | |
| import numpy as np | |
| np.random.seed(0) | |
| # the function we want to optimize |
| """ Trains an agent with (stochastic) Policy Gradients on Pong. Uses OpenAI Gym. """ | |
| import numpy as np | |
| import cPickle as pickle | |
| import gym | |
| # hyperparameters | |
| H = 200 # number of hidden layer neurons | |
| batch_size = 10 # every how many episodes to do a param update? | |
| learning_rate = 1e-4 | |
| gamma = 0.99 # discount factor for reward |
| .punch-viewer-speakernotes-side-panel { | |
| width: 400px !important; | |
| } | |
| .punch-viewer-speakernotes-text-body-scrollable { | |
| left: 435px !important; | |
| } | |
| .punch-viewer-speakernotes-page, | |
| .punch-viewer-speakernotes-page svg { | |
| width:400px !important; | |
| height:300px !important; |
| """ | |
| Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy) | |
| BSD License | |
| """ | |
| import numpy as np | |
| # data I/O | |
| data = open('input.txt', 'r').read() # should be simple plain text file | |
| chars = list(set(data)) | |
| data_size, vocab_size = len(data), len(chars) |