bytearchive

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

bytearchive

TLDR: I now add the following snippet to all my Dockerfiles:

# If host is running squid-deb-proxy on port 8000, populate /etc/apt/apt.conf.d/30proxy
# By default, squid-deb-proxy 403s unknown sources, so apt shouldn't proxy ppa.launchpad.net
RUN route -n | awk '/^0.0.0.0/ {print $2}' > /tmp/host_ip.txt
RUN echo "HEAD /" | nc `cat /tmp/host_ip.txt` 8000 | grep squid-deb-proxy \
  && (echo "Acquire::http::Proxy \"http://$(cat /tmp/host_ip.txt):8000\";" > /etc/apt/apt.conf.d/30proxy) \
  && (echo "Acquire::http::Proxy::ppa.launchpad.net DIRECT;" >> /etc/apt/apt.conf.d/30proxy) \
  || echo "No squid-deb-proxy detected on docker host"

bytearchive

My Elasticsearch cheatsheet with example usage via rest api (still a work-in-progress)

Shortlinks:

• Cluster Health
  • Index Level
  • Shard Level
• Nodes Overview
• Indices Overview
• Cluster Maintenance

bytearchive

#-------------------------------------------------------------------------------
# lexer.py
#
# A generic regex-based Lexer/tokenizer tool.
# See the if __main__ section in the bottom for an example.
#
# Eli Bendersky ([email protected])
# This code is in the public domain
# Last modified: August 2010
#-------------------------------------------------------------------------------

bytearchive

Useful Pandas Snippets

A personal diary of DataFrame munging over the years.

Data Types and Conversion

Convert Series datatype to numeric (will error if column has non-numeric values)
(h/t @makmanalp)

bytearchive

# Set variables in .bashrc file

# don't forget to change your path correctly!

export GOPATH=$HOME/golang
export GOROOT=/usr/local/opt/go/libexec
export PATH=$PATH:$GOPATH/bin
export PATH=$PATH:$GOROOT/bin

bytearchive

# -*- coding: utf-8 -*-

"""A simple tool to document how to control AWS resources.

AWS AUTHENTICATION
-------------------

In order to run any of the code below, you need a profile with AWS credentials
set up on your computer. It's very easy to do this. Google how to configure
your profile with boto3, or visit the docs:

bytearchive

# -*- coding: utf-8 -*-

"""A simple tool to document how to control AWS resources.

AWS AUTHENTICATION
-------------------

In order to run any of the code below, you need a profile with AWS credentials
set up on your computer. It's very easy to do this. Google how to configure
your profile with boto3, or visit the docs:

bytearchive

class="[a-zA-Z0-9:;\.\s\(\)\-\,]*"

bytearchive

# -*- coding:utf8 -*-
import re

from flask import Flask

from flask_sqlalchemy import SQLAlchemy as BaseSQLAlchemy
from flask_sqlalchemy import _SignallingSession as BaseSignallingSession
from flask_sqlalchemy import orm, partial, get_state

from datetime import datetime