Karmak23 Karmak23

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

Long is ≥ 0.5 sec

With 100 concurrent connections

python scripts/test_API.py 200 100
INFO: Starting timings by ranges of 200 calls on http://popshake.codegastudio.com/api, @2014-09-29T14:07:57.354054)
INFO: Results: 140/200 (70.0%) long calls for /; average others: 0.199sec
INFO: Results: 64/200 (32.0%) long calls for /moods/; average others: 0.223sec
INFO: Results: 84/200 (42.0%) long calls for /moods/1/; average others: 0.273sec

Keybase proof

I hereby claim:

I am Karmak23 on github.
I am karmak23 (https://keybase.io/karmak23) on keybase.
I have a public key whose fingerprint is 64D9 3693 529E 7C96 4F2A ACA8 9530 77A9 A145 96A0

To claim this, I am signing this object:

	# coding=UTF-8
	import nltk
	from nltk.corpus import brown

	# This is a fast and simple noun phrase extractor (based on NLTK)
	# Feel free to use it, just keep a link back to this post
	# http://thetokenizer.com/2013/05/09/efficient-way-to-extract-the-main-topics-of-a-sentence/
	# Create by Shlomi Babluki
	# May, 2013

	# -- coding: utf-8 --

	"""
	Whoosh backend for haystack that implements character folding, as per
	http://packages.python.org/Whoosh/stemming.html#character-folding .

	Tested with Haystack 2.4.0 and Whooch 2.7.0

	To use, put this file on your path and add it to your haystack settings, eg.

	#!/bin/bash

	### USAGE
	###
	### ./ElasticSearch.sh 1.5.0 will install Elasticsearch 1.5.0
	### ./ElasticSearch.sh 1.4.4 will install Elasticsearch 1.4.4
	### ./ElasticSearch.sh will fail because no version was specified (exit code 1)
	###
	### CLI options Contributed by @janpieper
	### Check http://www.elasticsearch.org/download/ for latest version of ElasticSearch

	# -*- coding: utf-8
	"""
	PostgreSQL backup script for Windows.
	Olivier Cortès <contact@oliviercortes.com> 2014
	Licensed under the GNU GPL version 3.

	https://gist.github.com/Karmak23/62872892dd49f04f58d2

	## Dependancies

	class WithoutNoneFieldsSerializer(serializers.ModelSerializer):

	""" Exclude model fields which are ``None``.

	This eventually includes foreign keys and other special fields.

	Source: https://gist.github.com/Karmak23/5a40beb1e18da7a61cfc
	"""

	def to_native(self, obj):

	class DRFLoggerMixin(object):
	"""
	Allows us to log any incoming request and to know what's in it.

	Usage:

	class MyOwnViewSet(DRFLoggerMixin,
	mixins.ListModelMixin,
	…
	viewsets.GenericViewSet):

	import types
	from django import forms

	class DynamicAttrsForm(forms.ModelForm):

	class Meta:
	model = models.DynamicAttrs
	fields = ['template', 'mood', 'text' ]