A pattern for building personal knowledge bases using LLMs.
This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.
Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.
A pattern for building personal knowledge bases using LLMs.
This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.
Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.
| <!DOCTYPE NETSCAPE-Bookmark-file-1> | |
| <!-- This is an automatically generated file. | |
| It will be read and overwritten. | |
| DO NOT EDIT! --> | |
| <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8"> | |
| <TITLE>Bookmarks</TITLE> | |
| <H1>Bookmarks</H1> | |
| <DL><p> | |
| <DT><H3 ADD_DATE="1619134484" LAST_MODIFIED="1652674233" PERSONAL_TOOLBAR_FOLDER="true">Bookmarks Bar</H3> | |
| <DL><p> |
| #!/bin/bash | |
| function vault-aws () { | |
| VAULT_PATH=$1 | |
| if [ -z "$VAULT_PATH" ]; then | |
| echo "Missing VAULT_PATH argument.\nExample: `vault-aws documents-store`" | |
| exit 1 | |
| fi | |
| if [ -z "$VAULT_ADDR" ]; then | |
| echo "Missing VAULT_ADDR env variable" |
| #!/bin/sh | |
| # Launch a Pod ab-using a hostPath mount to land on a Kubernetes node cluster as root | |
| # without requiring `privileged: true`, in particular can abuse `DenyExecOnPrivileged` | |
| # admission controller. | |
| # Pod command in turn runs a privileged container using node's /var/run/docker.sock. | |
| # | |
| # Tweaked for PKS nodes, which run their docker stuff from different | |
| # /var/vcap/... paths | |
| node=${1} | |
| case "${node}" in |
| #! /bin/bash | |
| set -o errexit | |
| export APP="${1:-mutateme}" | |
| export NAMESPACE="${2:-default}" | |
| export CSR_NAME="${APP}.${NAMESPACE}.svc" | |
| echo "... creating ${app}.key" | |
| openssl genrsa -out ${APP}.key 2048 |
| require 'json' | |
| # Rename parameter store paths. Takes: | |
| # - parameter store json file path | |
| # - old path | |
| # - new path to use | |
| old_path, new_path, _rest = ARGV | |
| params = `aws ssm get-parameters-by-path --path #{old_path} --with-decryption` | |
| params = JSON.parse(params, object_class: OpenStruct) |
| #!/bin/bash | |
| # | |
| # NOTE: specify the absolutepath to the directory to use when | |
| # loading a plugin. '~' expansion is supported. | |
| # | |
| chunkc core::plugin_dir /usr/local/opt/chunkwm/share/chunkwm/plugins | |
| # |
| #!/usr/bin/env bash | |
| # Quinn Comendant <[email protected]> | |
| # https://gist.github.com/quinncomendant/3be731567e529415d5ee | |
| # Since 25 Jan 2015 | |
| # Version 1.2 | |
| CMD=$1; | |
| if [[ `id -u` = 0 ]]; then |