Jason Roberts jrob00

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

Make /robots.txt aware of the Rails environment

You probably don't want Google crawling your development staging app. Here's how to fix that.

$ mv public/robots.txt config/robots.production.txt
$ cp config/robots.production.txt config/robots.development.txt

Now edit config/routes.rb to add a route for /robots.txt, and add the controller code.

	The MIT License (MIT)

	Copyright (c) 2015 Textalk

	Permission is hereby granted, free of charge, to any person obtaining a copy
	of this software and associated documentation files (the "Software"), to deal
	in the Software without restriction, including without limitation the rights
	to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
	copies of the Software, and to permit persons to whom the Software is
	furnished to do so, subject to the following conditions:

	# Simple example of aes-256-cbc encryption for Ruby/Rails to encrypt URL params (or any text)

	require 'openssl'
	require 'base64'
	require 'uri'


	plaintext = 'id=example_id&user_id=the_user_id&username=voxtrot&user_first_name=john&user_last_name=doe&user_image=http://i1.nyt.com/images/misc/nytlogo379x64.gif'

	#

	source 'https://rubygems.org'

	# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
	gem 'rails', '4.0.0.rc1'

	# Use sqlite3 as the database for Active Record
	gem 'sqlite3'

	# Use SCSS for stylesheets
	gem 'sass-rails', '~> 4.0.0.rc1'

	SSH agent forwarding is great. It allows you to ssh from one server to
	another all the while using the ssh-agent running on your local
	workstation. The benefit is you don't need to generate ssh key pairs
	on the servers you are connecting to in order to hop around.

	When you ssh to a remote machine the remote machine talks to your
	local ssh-agent through the socket referenced by the SSH_AUTH_SOCK
	environment variable.

	So you the remote server you can do something like:

	# expand save dialog sheet
	defaults write NSGlobalDomain NSNavPanelExpandedStateForSaveMode -bool true

	# allow key repeat
	defaults write NSGlobalDomain ApplePressAndHoldEnabled -bool false

	# finder cmd-q
	defaults write com.apple.finder QuitMenuItem -bool true

	# text selection in quick look

	{
	"bold_folder_labels": true,
	"caret_style": "phase",
	"color_scheme": "Packages/User/Tomorrow-Night (SL).tmTheme",
	"drag_text": false,
	"find_selected_text": true,
	"flatland_square_tabs": true,
	"folder_exclude_patterns":
	[
	".svn",

	#!/bin/bash

	# php switch for homebrew
	# $ brew tap josegonzalez/php && brew install php53 --with-mysql && brew install php54 --with-mysql

	# Might as well ask for password up-front, right?
	sudo -v

	VERSION_FILE="$( cd -P "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/php-switch.version"

	NameVirtualHost *:80
	NameVirtualHost *:443

	#DocumentRoot "/var/www/html"

	<Directory />
	Options -Indexes +FollowSymLinks
	Order allow,deny
	Allow from all
	</Directory>