llm-wiki

llm-wiki.md

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

The idea here is different. Instead of just retrieving from raw documents at query time, the LLM incrementally builds and maintains a persistent wiki — a structured, interlinked collection of markdown files that sits between you and the raw sources. When you add a new source, the LLM doesn't just index it for later retrieval. It reads it, extracts the key information, and integrates it into the existing wiki — updating entity pages, revising topic summaries, noting where new data contradicts old claims, strengthening or challenging the evolving synthesis. The knowledge is compiled once and then kept current, not re-derived on every query.

This is the key difference: the wiki is a persistent, compounding artifact. The cross-references are already there. The contradictions have already been flagged. The synthesis already reflects everything you've read. The wiki keeps getting richer with every source you add and every question you ask.

You never (or rarely) write the wiki yourself — the LLM writes and maintains all of it. You're in charge of sourcing, exploration, and asking the right questions. The LLM does all the grunt work — the summarizing, cross-referencing, filing, and bookkeeping that makes a knowledge base actually useful over time. In practice, I have the LLM agent open on one side and Obsidian open on the other. The LLM makes edits based on our conversation, and I browse the results in real time — following links, checking the graph view, reading the updated pages. Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase.

This can apply to a lot of different contexts. A few examples:

• Personal: tracking your own goals, health, psychology, self-improvement — filing journal entries, articles, podcast notes, and building up a structured picture of yourself over time.
• Research: going deep on a topic over weeks or months — reading papers, articles, reports, and incrementally building a comprehensive wiki with an evolving thesis.
• Reading a book: filing each chapter as you go, building out pages for characters, themes, plot threads, and how they connect. By the end you have a rich companion wiki. Think of fan wikis like Tolkien Gateway — thousands of interlinked pages covering characters, places, events, languages, built by a community of volunteers over years. You could build something like that personally as you read, with the LLM doing all the cross-referencing and maintenance.
• Business/team: an internal wiki maintained by LLMs, fed by Slack threads, meeting transcripts, project documents, customer calls. Possibly with humans in the loop reviewing updates. The wiki stays current because the LLM does the maintenance that no one on the team wants to do.
• Competitive analysis, due diligence, trip planning, course notes, hobby deep-dives — anything where you're accumulating knowledge over time and want it organized rather than scattered.

Architecture

There are three layers:

Raw sources — your curated collection of source documents. Articles, papers, images, data files. These are immutable — the LLM reads from them but never modifies them. This is your source of truth.

The wiki — a directory of LLM-generated markdown files. Summaries, entity pages, concept pages, comparisons, an overview, a synthesis. The LLM owns this layer entirely. It creates pages, updates them when new sources arrive, maintains cross-references, and keeps everything consistent. You read it; the LLM writes it.

The schema — a document (e.g. CLAUDE.md for Claude Code or AGENTS.md for Codex) that tells the LLM how the wiki is structured, what the conventions are, and what workflows to follow when ingesting sources, answering questions, or maintaining the wiki. This is the key configuration file — it's what makes the LLM a disciplined wiki maintainer rather than a generic chatbot. You and the LLM co-evolve this over time as you figure out what works for your domain.

Operations

Ingest. You drop a new source into the raw collection and tell the LLM to process it. An example flow: the LLM reads the source, discusses key takeaways with you, writes a summary page in the wiki, updates the index, updates relevant entity and concept pages across the wiki, and appends an entry to the log. A single source might touch 10-15 wiki pages. Personally I prefer to ingest sources one at a time and stay involved — I read the summaries, check the updates, and guide the LLM on what to emphasize. But you could also batch-ingest many sources at once with less supervision. It's up to you to develop the workflow that fits your style and document it in the schema for future sessions.

Query. You ask questions against the wiki. The LLM searches for relevant pages, reads them, and synthesizes an answer with citations. Answers can take different forms depending on the question — a markdown page, a comparison table, a slide deck (Marp), a chart (matplotlib), a canvas. The important insight: good answers can be filed back into the wiki as new pages. A comparison you asked for, an analysis, a connection you discovered — these are valuable and shouldn't disappear into chat history. This way your explorations compound in the knowledge base just like ingested sources do.

Lint. Periodically, ask the LLM to health-check the wiki. Look for: contradictions between pages, stale claims that newer sources have superseded, orphan pages with no inbound links, important concepts mentioned but lacking their own page, missing cross-references, data gaps that could be filled with a web search. The LLM is good at suggesting new questions to investigate and new sources to look for. This keeps the wiki healthy as it grows.

Indexing and logging

Two special files help the LLM (and you) navigate the wiki as it grows. They serve different purposes:

index.md is content-oriented. It's a catalog of everything in the wiki — each page listed with a link, a one-line summary, and optionally metadata like date or source count. Organized by category (entities, concepts, sources, etc.). The LLM updates it on every ingest. When answering a query, the LLM reads the index first to find relevant pages, then drills into them. This works surprisingly well at moderate scale (~100 sources, ~hundreds of pages) and avoids the need for embedding-based RAG infrastructure.

log.md is chronological. It's an append-only record of what happened and when — ingests, queries, lint passes. A useful tip: if each entry starts with a consistent prefix (e.g. ## [2026-04-02] ingest | Article Title), the log becomes parseable with simple unix tools — grep "^## \[" log.md | tail -5 gives you the last 5 entries. The log gives you a timeline of the wiki's evolution and helps the LLM understand what's been done recently.

Optional: CLI tools

At some point you may want to build small tools that help the LLM operate on the wiki more efficiently. A search engine over the wiki pages is the most obvious one — at small scale the index file is enough, but as the wiki grows you want proper search. qmd is a good option: it's a local search engine for markdown files with hybrid BM25/vector search and LLM re-ranking, all on-device. It has both a CLI (so the LLM can shell out to it) and an MCP server (so the LLM can use it as a native tool). You could also build something simpler yourself — the LLM can help you vibe-code a naive search script as the need arises.

Tips and tricks

• Obsidian Web Clipper is a browser extension that converts web articles to markdown. Very useful for quickly getting sources into your raw collection.
• Download images locally. In Obsidian Settings → Files and links, set "Attachment folder path" to a fixed directory (e.g. raw/assets/). Then in Settings → Hotkeys, search for "Download" to find "Download attachments for current file" and bind it to a hotkey (e.g. Ctrl+Shift+D). After clipping an article, hit the hotkey and all images get downloaded to local disk. This is optional but useful — it lets the LLM view and reference images directly instead of relying on URLs that may break. Note that LLMs can't natively read markdown with inline images in one pass — the workaround is to have the LLM read the text first, then view some or all of the referenced images separately to gain additional context. It's a bit clunky but works well enough.
• Obsidian's graph view is the best way to see the shape of your wiki — what's connected to what, which pages are hubs, which are orphans.
• Marp is a markdown-based slide deck format. Obsidian has a plugin for it. Useful for generating presentations directly from wiki content.
• Dataview is an Obsidian plugin that runs queries over page frontmatter. If your LLM adds YAML frontmatter to wiki pages (tags, dates, source counts), Dataview can generate dynamic tables and lists.
• The wiki is just a git repo of markdown files. You get version history, branching, and collaboration for free.

Why this works

The tedious part of maintaining a knowledge base is not the reading or the thinking — it's the bookkeeping. Updating cross-references, keeping summaries current, noting when new data contradicts old claims, maintaining consistency across dozens of pages. Humans abandon wikis because the maintenance burden grows faster than the value. LLMs don't get bored, don't forget to update a cross-reference, and can touch 15 files in one pass. The wiki stays maintained because the cost of maintenance is near zero.

The human's job is to curate sources, direct the analysis, ask good questions, and think about what it all means. The LLM's job is everything else.

The idea is related in spirit to Vannevar Bush's Memex (1945) — a personal, curated knowledge store with associative trails between documents. Bush's vision was closer to this than to what the web became: private, actively curated, with the connections between documents as valuable as the documents themselves. The part he couldn't solve was who does the maintenance. The LLM handles that.

Note

This document is intentionally abstract. It describes the idea, not a specific implementation. The exact directory structure, the schema conventions, the page formats, the tooling — all of that will depend on your domain, your preferences, and your LLM of choice. Everything mentioned above is optional and modular — pick what's useful, ignore what isn't. For example: your sources might be text-only, so you don't need image handling at all. Your wiki might be small enough that the index file is all you need, no search engine required. You might not care about slide decks and just want markdown pages. You might want a completely different set of output formats. The right way to use this is to share it with your LLM agent and work together to instantiate a version that fits your needs. The document's only job is to communicate the pattern. Your LLM can figure out the rest.

Thank you for sharing this idea — it changed how I work.

I'm not a developer — I'm an accountant in Korea, doing K-IFRS advisory and audits at an accounting firm. English isn't my strong suit either, so this comment was written with AI help (fitting, given the topic). I've been running my own instance for a while: immutable sources/, an agent-maintained wiki/, and a CLAUDE.md schema, covering accounting standards plus IT, physics, math, and philosophy as separate domains in one vault.

llm-wiki/
├── sources/                  # immutable originals — the LLM never edits
│   ├── inbox/                # unclassified intake (incl. inbox/ai/ for chatbot summaries)
│   └── domain/               # accounting, capital-market, it, network, physics, math, philosophy...
│       ├── ai-archive/       # processed AI summaries (still "unverified")
│       ├── reference/        # full-text library for verification lookups
│       └── ...               # classified excerpts (standards, rulings, notes...)
├── wiki/
│   ├── domain/topics/        # agent-maintained pages, each with a verification status
│   ├── index.md              # catalog, updated on every ingest
│   └── open-questions.md     # contradictions and unresolved issues
├── scratch/                  # human-only notepad, excluded from all workflows
└── CLAUDE.md                 # the schema

One thing I added, coming from audit work: a verification status on every page. AI-generated content enters through a quarantined inbox, is marked unverified, and cannot be cited as grounds for any other page until checked against primary sources (accounting standards, paragraph by paragraph). An unverified page citing another unverified page is exactly the circular evidence auditors are trained to reject. Each domain also has its own mandatory citation format — standards paragraph numbers for accounting, RFC numbers for networking, and so on.

Another extension: conversations with external chatbots are a first-class ingest source. I keep a standardized prompt (and an agent skill) that distills a ChatGPT/Claude conversation into atomic notes with fixed frontmatter, tags, and naming conventions — which then enter through the same quarantined inbox as any other unverified material. Reading the comments here, it seems several of us converged on this independently.

Watching scattered knowledge compound into something I can actually trust in client work has been a real joy. If this pattern works for a non-developer like me, it can work for anyone. Thanks again for giving this away so generously.

Built the Notion side of this as a small pull-only bridge: notionwiki

notionwiki treats Notion as one feeder into raw/ — a scheduled job polls, converts pages/database rows to flat markdown (hierarchy kept as frontmatter, not folders, so a Notion move never reshuffles the corpus), and archives-on-change rather than overwriting silently. No write-back, no sync/push — Notion stays where you author, and the actual wiki-building (ingest → synthesize → lint) is exactly the loop described here, run by an assistant against CLAUDE.md as the schema. The part I keep coming back to: without a real push mechanism, the corpus is only ever as fresh as the last poll — so "compounding" ends up bounded by pull cadence, not by how fast the model can actually synthesize.

Fully Working integration of this:

https://github.com/Encod3d-Sec/ClaudeBrain

First of all, thanks to @karpathy for introducing and describing the LLM-Wiki paradigm: a simple yet brilliant idea that, in my opinion, will revolutionize corporate document management (replacing heavy, complex knowledge bases), help students avoid superficial LLM usage, and serve as a great ally for methodical learning.

I have built a Personal LLM-Wiki prioritizing two fundamental aspects:

1. Privacy: All components run completely offline.
2. Efficiency: I focused on creating an executable suitable for modest hardware (with or without a GPU), making it accessible despite current market prices for graphics cards.

The core concept is straightforward: perform non-LLM-specific operations deterministically using standard Python libraries, which consume significantly fewer resources than a giant language model. It feels inefficient to waste LLM tokens on repetitive tasks that only require classic computational power.

While my initial prototype worked, the model's responses were often too generic or limited to brief definitions. To achieve the exhaustive, structured lessons I envisioned, I integrated a few key components to enrich the context:

Key Components

• Graphify: Maps relationships between Markdown files, SQL schemas, scripts, and PDFs. It improves navigation based on structural relationships rather than just keywords, creating a navigable "map" directly accessible to the AI or via Obsidian. This drastically reduced dependency on compute while improving coherence.
• ChromaDB: A lightweight vector database used as the pillar for semantic search and document retrieval, executing operations efficiently on the CPU.
• NetworkX: Manages a dual data structure in memory with a lazy cache: an Undirected Graph ($G$) for generic relationships and a Directed Graph ($DiG$) focused exclusively on formative dependencies. This helps identify correlations and cite diverse sources covering the requested topic.
• SQLite: An embedded, serverless relational database dedicated to managing the system's transactional state and audit logs.

Operational Flow

[Phase 1 (Optional): Pre-processing (Whisper)] ──> 
──> [Phase 2: Synthesis (Qwen3VL)]
──> [Phase 3: Graph Building (Graphify)]
──> [Phase 4: Indexing (ChromaDB)]
──> [Phase 5: Context Assembly]
──> [Phase 6: Inference (Qwen3.5 9B)]

Advanced features included: Semantic Query Cache, Cross-Encoder Re-Ranker, and Reciprocal Rank Fusion (RRF) for HyDE.

Additional Features

• Localization & UI: Built a lightweight Flask web page with "on-the-fly" language switching managed via simple .lng files.
• Secure Telegram Bot: Accessible without exposing ports or reverse proxies (restricted to authorized Telegram IDs). It includes an interactive poll after each response, allowing the user to save the text, export it as a Marp Markdown presentation, or generate an audio podcast.
• Quiz Mode: Generates multiple-choice questions based on the wiki content with customizable difficulty, evaluating errors and explaining why a specific answer was wrong.

A Critique of the Paradigm: No New Wiki Pages Without Raw Sources

The only critique I have regarding the baseline LLM-Wiki paradigm concerns creating new wiki pages derived from LLM responses.

I don't find it useful to generate additional pages beyond those processed during the Ingest phase. Reprocessing existing concepts in different forms consumes resources and slows down the system (more pages to scan per query) without introducing genuinely new information. Since the LLM should not invent anything (avoiding hallucinations), it doesn't enrich the source base.

In my implementation, the only generated pages allowed are the Markdown files exported for Presentations (Marp syntax)-treated strictly as "output documents" for the user, rather than being re-fed into the pipeline as wiki sources.

I haven't published the repository on GitHub yet, as I am still refining the integration to make it as user-friendly as possible. I hope this architecture can serve as inspiration for anyone looking to customize or optimize their local LLM-Wiki setup!

Below is an example of an answer to the question “Explain me what Kubernetes is.”

Hi Maurizio, I like the way you implemented this. Well done and inspiring for me.
BR, Mike

Hi @karpathy, I implemented a complete toolkit to facilitate the OKF adoption (LLM Wiki ideas), which is composed of three cores:

An Agent Skill (the 🧠 ), a CLI/Lib (the 💪 ), and a Server (the 📊 )

The agent knows how to author, curate, and, more importantly, how to consume (CLI tools + Server (for humans)).

It is ​Open source and runs 100% locally! (using Ruby or Docker).

Here is the GitHub repo: https://github.com/serradura/okf-gem (https://demo.okfgem.com)

The knowledge graph looks like this:​

And you can test it here: https://demo.okfgem.com/

Step by step to experiment with it:

1. Install the gem: gem install okf
2. Install the skill: okf skill .claude (or any other agent)
3. Start an agent session (claude)
4. Execute /okf produce based on <path-to-my-existent-docs>

Once you have your first bundle, you can do:

Run the command /okf maintain in your agent session to keep it updated. Or, install the Claude Plugin, which will run this for you (after triggering the Stop hook).

You can also run okf server <folder> to render your live graph locally. Or ask the agent to up it for you.

If you want to know more, please check out these other resources:

• Site: https://okfgem.com/
• Blog: https://okfgem.com/blog
• Docs: https://okfgem.com/docs
• LLMs.txt: https://okfgem.com/llms.txt (https://okfgem.com/llms-full.txt)
• Claude Plugin: https://claude.okfgem.com/

It is also possible to run it through Docker (no Ruby required):

• https://docker.okfgem.com/

Looking forward to your feedback and critiques.

Thanks!

This resonates strongly - and I'd push one layer deeper on why the bookkeeping gets abandoned. In our experience the thing that doesn't scale is unqualified links + free text. A lovingly hand-maintained personal wiki decays into prose and dead links within a couple of years; a team or corporate one gets there faster. What actually compounds is structure - typed entities you can query, not paragraphs you have to re-read.

We took a structured variant of exactly this pattern into production about five years ago, on Semantic MediaWiki. Using MediaWiki's content-slot feature, every article keeps its unstructured body and a structured JSON slot side by side. The JSON slot's shape is defined by the article's category/categories, and each category carries a JSON Schema in its own slot - so one schema drives both auto-generated edit forms for humans and machine editing for everything else.

Putting JSON-LD on top (we specced this as OO-LD: one document that is simultaneously a JSON Schema and a JSON-LD context) turns the whole wiki into an RDF knowledge graph for free - you can run SPARQL across every entry. Your lint step (contradictions, orphans, missing cross-refs) becomes graph queries; query becomes structured retrieval, not just text search.

When LLMs with JSON-Schema-driven structured output arrived, it was a drop-in fit: the exact schemas that render the forms also constrain LLM extraction.

Mapping to your three layers: raw sources (immutable) / the wiki - but as typed, linkable entities rather than markdown / the schema - a JSON Schema per category rather than one prose config.

Where we're putting research effort now is precisely your ingest loop at scale: take an unstructured knowledge chunk -> select the right schema(s)/categories to represent it -> fill them correctly and dedup against entities that already exist. That last "and" - entity resolution against a live graph - is the hard part; it's your lint moved to write-time.

Curious whether others here have pushed on write-time dedup - that's where the pattern either compounds or sprawls.

Reference implementation: OpenSemanticLab (live demo: https://demo.open-semantic-lab.org) · schema approach: OO-LD.

Thanks for sharing this, Andrej. The distinction between retrieving documents and building a persistent knowledge artifact that compounds over time really clicked for me.

I used your LLM Wiki pattern as the foundation for My Second Brain by YFW, a local-first application that turns documents, images, PDFs, and web pages into a connected, searchable, Obsidian-compatible Markdown wiki.

It implements the three layers you described—immutable raw sources, an LLM-maintained wiki, and an inspectable instruction/schema layer—and extends them with a browser interface, CLI workflows, scoped knowledge branches, source provenance, grounded answers with citations, structural review, and optional hybrid search.

One lesson from building it is that retrieval is still useful, but mainly as a navigation layer. The durable object is the maintained Markdown wiki: readable by humans, editable by agents, portable across tools, and improved with every ingest and question.

Thank you for publishing the idea in such an open-ended form. It gave me a clear conceptual foundation while leaving enough room to develop a concrete implementation around my own workflow.

Project: github.com/EricWcr7/My-Second-Brain-by-YFW

Built a desktop app around this idea — Eva. Instead of the CLI-agent + Obsidian workflow, it's a native app that drives Claude Code or Codex to do the ingest/query/health-check loop, with a git-backed review gate (deletions and new lint issues always hold for review before merging) and a live graph you actually navigate, not just Obsidian's graph view. Tested it end to end on a few dozen Berkshire shareholder letters and a full college course — both held up.
Local-first, no account, no cloud DB, MIT licensed. V1 is macOS-only and unsigned (documented why in the README, mostly "$99/year isn't worth it yet for a solo project").
github.com/jp-lorenc1o/Eva-brain — jp-lorenc1o.github.io/Eva-brain

I absolutely love this idea! I've actually created a skill that works along this exact same approach, based on my experience over the last year working with LLMs—context matters so much!

It's a developer-friendly AI skill called MDCP (MarkDown Context Protocol) that helps keep track of ideas directly in the repo when developing software. The key to the approach is keeping the markdown shards small. The result is human- and machine-readable information that stays durable to the repo, helping you focus on building the software one idea at a time.

There's also a companion CLI to help compile the shards into comprehensive guides and validate links as well.

Thanks for sharing this pattern, it's incredibly validating to see others converging on the same solution!

https://github.com/betsalel-williamson/mdcp

https://www.skills.sh/betsalel-williamson/mdcp/mdcp

For my taste many implementations are at the border of over engineering! To much MCP servers, knowledge graphs, vector storage and third party tools! The power of this idea lies in simplicity, open protocols, ease of use for agents and humans alike. The core is auto discoverability and open standards. My implementation has all that, for wiki and task management. It replaces enterprise Confluence (Wiki) and Jira (tasks) and automates what is possible AI does the heavy lifting while keeping human operators in the loop: https://github.com/theafh/ai-modules

Thank you for writing this up. It was early to where the whole field seems to be heading now: memory that gets built and maintained instead of stored and queried. HORMA, TencentDB's agent memory pyramid, Anthropic's new Dreams feature all point the same way. Some of that is probably independent convergence, but this gist gave a lot of us the words first, and that is most of why I started building instead of just nodding along. The knowledge tier of mine came straight from here.

The part I keep wondering about, and I would genuinely like your take: what happens when the thing being compiled is the agent itself?

Even active memory leaves one question open. Who decides what is important, and who anchors it? With nothing at the center, every extraction re-decides importance from nowhere and the judgments never add up into anyone, so you land on generic. The bet I am making is not an agent that has a memory, but a self made of memory, which then curates the memory it gets rebuilt from. Two arrows instead of one.

The self part is not hand-waving, it is a structure on disk. Six layers, each allowed to change at a different speed: the fixed things that never move, an identity layer that only changes through a gate, values that can only tighten, and a temporal skin rewritten daily. It also models itself before it answers, and that is the part that makes it a self rather than a profile. Theory of mind is that same self-model pointed outward, since you cannot simulate another mind without one to run it on.

Which is the order evolution used too. A boundary first, before anything was even alive. Then movement, then vision. Brainpower alone was not enough either, Neanderthals had bigger brains and still died out, and what carried us was the social layer, which came last. Most AI memory builds that backwards: a finished assistant, with a profile of you clipped onto the side.

Wrote it up, your gist credited throughout:
https://gist.github.com/syahiidkamil/c6a3836fcfd88843865d87319c81ce37

Running PoC, all plain markdown and git:
https://github.com/syahiidkamil/vibe-ai-partner-entity

Thanks again for giving it away.

Most of our current interactions with AI, like using NotebookLM or ChatGPT's file uploads, follow a Retrieval-Augmented Generation (RAG) model. You upload a collection of files, and when you ask a question, the AI searches through them, pulls out the most relevant chunks, and synthesises an answer. The problem? It's starting from scratch every time. The AI is a brilliant researcher who forgets everything as soon as the query is done. Karpathy's idea flips this on its head. Instead of [fridayroll](url just indexing raw documents for retrieval, the LLM incrementally builds a persistent, structured wiki—a collection of interlinked Markdown files. When you add a new source, the LLM doesn't just index it; it reads it, extracts the key information, and integrates it into the existing wiki. It updates entity pages, revises topic summaries, and flags contradictions. The knowledge is compiled once and kept current.

Maybe mARC can help it...

https://github.com/notfounnd/marc-server

🧠 RAG is Dead. Build a Continuous Local AI Wiki with LLMWikiNG.

Stop rebuilding your knowledge base from scratch on every single prompt. LLMWikiNG (inspired by Andrej Karpathy's LLM-Wiki pattern) is a local, privacy-first platform that compiles your documents once into a structured, cross-linked, OKF-compliant Markdown wiki that grows and refines continuously.

🚀 Deploy in Seconds, Ingest Seamlessly

• Instant Docker Deployment: Spin up the entire FastAPI backend, Tokyo-Night web UI, and local hybrid search in seconds using docker-compose. Zero-hassle setup, maximum performance.
• Direct Web Ingest: Push raw data, text snippets, or web URLs straight from your local machine to the server. Use the web interface or automate it via curl with secure API keys.
• 100% Local & Secure: Powered by Ollama. Your data never leaves your hardware, secured by Argon2 authentication and strict cryptographic controls.

🌐 Get Started

Turn your chaotic data into a self-evolving, strictly governed knowledge ecosystem.

👉 Deploy now on GitHub: https://github.com/ZeroDot1/LLMWikiNG

This maps closely onto something I built independently: Kherad, an open-source git-backed wiki. One of its bundle modes ("AI-compiled") is this exact ingest -> compile -> review loop: point it at a folder of raw source docs, an LLM pipeline compiles them into a structured, cross-linked wiki, and the compilation lands as a merge request you review before it goes live (screenshot attached). The rest of the tool wraps that in a Notion-style block editor plus a git commit/merge-request layer for the parts people edit by hand.

Repo if useful for comparing implementations: https://github.com/mohammadmaso/kherad

Link 1.7.0 is out — the local LLM-wiki memory layer I've been building on this pattern.

This release was about making the automatic loop trustworthy enough to leave running: agents propose memory from sessions, you approve, and now you can see exactly how every capture was made. Each one records a decision trail (what was kept, what was dropped as the agent's own echo, why) and mines only from your own turns, so the assistant's prose can never become your preference.

remember refuses credential-shaped text (a saved password would leak into every agent session by design. Dogfooding caught mine), forget-memory now scrubs the audit log too and re-anchors its hash chain, and one-click accept lands on the actual rule instead of the "I want to set some conventions…" preamble.

Also spent the cycle benchmarking it properly, since plain Markdown files, no LLM in the memory layer invites the recall-quality question: 84.8% on LoCoMo (all 1,540 questions) under mem0's own open harness, vs their platform's 83.2% re-judged by the same judge. The result held up under a second, unrelated judge too. LongMemEval end-to-end we lose (80.6 vs 91.0, answerer-bound; retrieval put the gold evidence in context 99.4% of the time), which is in RESULTS.md next to the ablations that failed.

Still: wiki as the storage layer, every memory a file you can open, nothing durable without review.

brew upgrade link  (or brew install gowtham0992/link/link)

Release notes: https://github.com/gowtham0992/link/releases/tag/v1.7.0

Repo: https://github.com/gowtham0992/link
Site: https://gowtham0992.github.io/link/
PyPI: https://pypi.org/project/link-mcp/
MCP: https://registry.modelcontextprotocol.io/?q=io.github.gowtham0992%2Flink

Love it. a little late to the party but was headsdown to built something similar.

Applied this pattern to a different domain: behavioral verification of code changes.

The overlap is direct. Same three layers:

Raw sources = PRs, specs, test cases, production incidents (immutable, never modified)
Wiki = a behavioral knowledge graph. Not how the code is structured, how the system is supposed to behave. Concern pages, flow maps, state transitions, cross-references between what the spec promises and what the tests cover.
Schema = a concern taxonomy (auth flows, data integrity, state transitions, error propagation, integration contracts, observability) that tells the system what to look for when a PR comes in.

Same three operations:

Ingest = parse a PR with tree-sitter, extract behavioral changes, update the graph. A single PR might touch 5-10 concern pages.
Query = "what concerns does this change touch? what can break from a user's perspective?"
Lint = verify the PR against the behavioral graph before merge. Flag drift between what the system promises and what the code actually does.

The compounding effect is the same. Every PR that flows through makes the graph richer. Every production incident that gets filed teaches it a new failure mode. The graph gets better at catching the next thing because it's seen the last hundred things.

The Lint section of your gist describes exactly what's missing from QA tooling. Most test generation tools produce more scripts without understanding what the system is supposed to do. This is the other direction: build the knowledge first, then verify against it.

Open sourced as an MCP server (Claude Code, Cursor, Codex): https://github.com/OrangeproAI/orangepro-mcp

No API key, no cloud, runs locally. Curious if you've thought about this pattern applied to code specifically.

This is an interesting approach to building a long-term knowledge base. Instead of retrieving the same information repeatedly, organizing knowledge over time can make research more efficient. I also think integrating https://nbiappointmentsonline.com resources where appropriate can further improve the usefulness and accessibility of such systems.

This maps almost 1:1 to a system we have been running in production for ~6 months to manage infrastructure/ops knowledge. A few notes from actually living with the pattern at ~4000+ interlinked concepts:

• The schema file is everything. Our CLAUDE.md is exactly the "disciplined maintainer vs. generic chatbot" config you describe — it encodes the ingest/query/lint workflows + naming conventions, and it co-evolved into the single most important file in the repo.
• index.md at scale: the flat index works great to a few hundred pages. Past that we added hybrid search (SQLite FTS5 + on-device embeddings, reciprocal-rank-fused) rather than standing up embedding-RAG infra — same spirit as qmd. We expose it as both a CLI (agent shells out) and an MCP server (native tool). ~1ms keyword, ~350ms hybrid.
• New page vs. edit (@alinawab): heuristic that works for us — new page when it is a distinct entity/concept you would link to from elsewhere; edit in place when it is an attribute/update of an existing one. The agent gets this right ~90% of the time once the schema enumerates the page types.
• Team sharing (@geetansharora): the wiki is just a private git repo, auto-synced. Teammates browse in Obsidian or hit the same MCP server. Git history doubles as the log.md audit trail for free.
• Biggest failure mode (@alinawab): drift — the agent under-updating cross-references on ingest, so pages silently go stale. The lint pass is not optional; we run it on a timer (orphan detection + contradiction flagging + stale-claim checks) and that is what keeps the graph healthy.

The "compounding artifact" framing is exactly right — after a few thousand concepts the wiki answers questions the raw sources never could, because the synthesis already happened. Thanks for writing it up so cleanly.

@distorx — of everyone here your setup maps closest to mine (team-scale, production, git-backed), so I'd value your take; anyone else who's hit this, welcome too.

Mine is packaged as an agent skill, not a personal wiki: the knowledge lives alongside a SKILL.md — the schema/entry file agents load — as a git repo of markdown. Same three layers you'd recognize: an immutable source-of-truth layer (~30 metadata snapshots auto-pulled from our data platform), an LLM/human-maintained wiki layer derived from it (per-dataset field specs, metric formulas, business logic, query cases; ~450 md pages + ~110 python config files), and the SKILL.md schema + lint rules on top. It's distributed to a whole team, and each person's agent (Claude Code / Cursor / etc.) both uses and edits it.

Two things I can't settle:

1. Size vs. keeping source-of-truth local. The immutable snapshots are the heaviest, churniest part. The gist stresses keeping an immutable raw layer as the foundation, but do you keep raw sources in-repo, or split them out (submodule / LFS / on-demand fetch) to keep the skill light? And does repo size actually hurt agents at query time, or is it purely a human clone/CI cost?
2. Concurrent team maintenance without rot. With multiple people's agents editing the same skill: do they push directly, or through a review gate (PR) before a page lands? How do you stop two agents fighting over the same page / spawning near-duplicates? Who owns the SKILL.md schema when the whole team co-evolves it? And the lint pass — human-triggered, or on a timer / CI hook? You called drift the #1 failure mode, so I'm most curious what actually enforces it at team scale.

Trying to avoid the failure mode where a shared wiki slowly rots because no single person owns the bookkeeping. Any pointers appreciated.

1. Size vs. keeping source-of-truth local

Four tiers, each with matching versioning discipline — not two.

Raw: never git. Object store / DB / DVC, pinned by content hash. No diff value in blobs (PDFs, scrapes, transcripts), so git's model is the wrong fit.

Staging (new tier, borrowed from dbt): per-source extraction, 1:1 with raw, no cross-page synthesis yet. Kept lightly versioned or just timestamped — this is where dbt's "don't bother versioning the output" logic actually applies cleanly, because it's close to a deterministic function of raw + a fixed extraction rule. Cheap to regenerate, low review bar.

Mart/wiki: full git, PR-reviewed. This is where LLM synthesis happens — non-deterministic, carries editorial decisions a human vetted. Unlike a dbt mart table, rerunning the agent on the same staging input won't reproduce the same page, so skipping version control here would silently discard real review work. Keep it git.

Schema: full git, higher scrutiny than page edits — governs the other three tiers, rare but wide-blast-radius changes.

On the actual question — does size hurt agents or just humans/CI: mostly humans/CI (clone, backup, checkout), but agent-side Glob/Grep latency scales with total repo size too, even for files the agent never targets. So keeping raw and staging out of the wiki's git repo isn't just clone-time hygiene — it's a real, if second-order, agent-latency lever.

2. Concurrent team maintenance without rot

PR gate always on mart/wiki, no direct push — except a lint bot auto-fixing pure mechanical breakage (dead links, orphan tags), never content claims.

Two agents colliding on a page, or spawning near-duplicates: page-level lease/lock claimed before edit (wait, or work a different page, if claimed), plus a mandatory index check — "does a page for this entity already exist?" — before creating new, keyed to a canonical naming convention. Schema-enforced rules, not agent judgment.

Borrowing dbt's ref() model: each mart/wiki page declares its dependencies explicitly in frontmatter (refs: [page-x, staging-hash-y]) instead of relying on prose links. This turns the cross-reference set into a computed DAG. Payoff: staleness detection becomes push-based — when a raw/staging source changes, you can directly query which downstream wiki pages depend on it and flag them immediately, instead of waiting for a periodic scan to stumble onto the drift.

Scheduled lint (nightly + on every wiki-branch merge, not human-triggered) graph, catching what the graph can't — stale prose claims, missingcross-refs the schema requires, orphan pages. Lint output becomes its own reviewed PR for anything content-level.

SKILL.md/schema has one owner (or small council), not open co-evolution. Schema changes go through PR at higher scrutiny than page edits, since every future agent run reads it.

What actually enforces this at team scale is four mechanisms stacked, none sufficient alone: PR gate (bad content caught pre-merge), page-lock (concurrent-edit collision avoided),
ref-graph (push-based staleness the moment a dependency changes), scheduledand gate both miss). Schema ownership is the meta-layer keeping those fourfrom drifting themselves.

MindBase — an open-source implementation of this post (MCP server, MIT)

Built this over the past few months: https://github.com/frankchu91/mindbase

Mapping to the three layers as specced:

• Raw sources → sources/ is append-only, enforced by tooling rather
  than convention: each operation runs as a sub-agent with a strict tool
  allowlist — the builder agent has no file-write tool at all, only an
  atomic-write MCP call that snapshots context.md before replacing it.
• The wiki → LLM-maintained markdown with [[wikilinks]], plus a typed
  link index (sqlite, derived state — markdown stays the source of truth):
  edges carry contradicts/supersedes/mentions, so lint can proactively
  surface "this new source contradicts what you wrote in March."
• The schema → a per-project README.md the LLM re-reads at every
  operation; user-editable, co-evolves with the wiki.

All three operations are in — a day looks like:

/mb:contribute paper.pdf → sub-agent reads, discusses 3 takeaways, waits
for approval, then updates 5-15 wiki pages
/mb:ask "my stance on X?" → cited answer from the already-synthesized wiki
/mb:lint → contradictions · orphans · stale claims · gaps

Works in any MCP client (Claude Code / Cursor / Windsurf / Cline):
npx -y mindbase-mcp. All local markdown — you can open the data dir
straight in Obsidian.

One finding for the drift discussion in this thread: the biggest drift
reducer for us wasn't better prompts — it was making the layer contract
physically unviolable via tool boundaries. A sloppy LLM turn can't
corrupt the sources layer even in principle, so drift stays confined to
the wiki layer, where lint can catch it.

Happy to compare notes with the other implementations here.

I looked at a lot of personal second brains before building this one, and bounced off most of them for the same two reasons: I could never tell what the system was actually doing, and getting started meant installing a stack. I wanted something simple enough that I always know exactly what happens when, and that asks you to install as little as possible.

That constraint produced the design:
https://github.com/sturlese/hippocampus

No vector database, no embeddings, no MCP server, no Obsidian plugins, no dependencies. Markdown, one stdlib Python linter, one shell hook, git.

Retrieval is three reads instead of a similarity search: a 500-word working-memory cache injected at session start, then the master index (one line per page), then the three to five pages it points to. Grep as fallback.

Every step is a file you can open and read.

That's the part I care about most. A missed lookup is a bad line in index.md — visible, editable. With embeddings you get a number, and the cause could be a chunk boundary, a stale index or an unlucky neighbour, none of which you can see.

It stops working somewhere in the low thousands of pages, when the index no longer fits in context.

I wrote up the whole pipeline step by step, in case the approach is useful even without the tool:
https://github.com/sturlese/hippocampus/blob/main/docs/what-happens-when-you-ingest.md

Built a variant of this pattern backed by Neo4j instead of markdown — same three layers (immutable Source nodes / a Concept+Claim graph / a schema-as-constraints), but cross-references become real typed edges you can run graph algorithms on.

Benchmarked the two things that actually matter instead of just shipping a demo:

Entity resolution: 45 labeled mention pairs (same-entity / related-but-distinct / unrelated), real embeddings. A single cosine threshold doesn't work — related-but-distinct pairs often score higher than true synonyms (best F1 = 0.667). A two-stage pipeline instead — cheap embedding filter for candidates, then one real LLM judgment call per candidate ("same entity or just related?") — got F1 to 1.000 on the same pairs.

Graph vs flat RAG on multi-hop questions: small real-facts corpus, 6 questions each needing 2-3 connected documents. Flat RAG (top-k) covered 4/6; graph traversal (N hops over typed edges) covered 6/6. Caveat: on this small dense corpus the traversal's recall win came with a precision cost (pulled 5-6 of 8 docs on 3 of 6 questions) — not tested at scale.

Full pattern, corpus, labeled pairs, and the chart: https://gist.github.com/lucianfialho/44034e0d02a2bfccca2ad6358bde1dff

I’ve been applying lmm_wiki as a heavy influence in creating a CMMS; Manage assets, resources, build metrics, feed them with real time signals or data gathered from procedures, build fully fleshed unscheduled/preventable maintenance/upkeep plans, and create issues against any of the entities; All while being fully greppable, levering markdown in multiple ways, requiring no database or license. Llm_wiki is a great influence for a cmms as the systems themselves are relatively low event.

Flattest. 1st principles. The assets don’t have logs, they are logs with fields, the resources are counts, etc.

I just wanted to share the index file I am using inside of my project. Perhaps it shows how the schema aspect of the concept can be heavily expanded and most concepts on display held through testing during an earlier second iteration. Third iteration maybe in testing next month.

https://gist.github.com/bprice1000/f986547dda0a82c5178faa6e237247fe

@geetansharora asked on day one how to share one of these with a team — the answers so far have been "serve the vault over MCP." We've been running a different, git-native answer for a few months, so here are some notes from actually operating it.

Context: we build on a niche SDK that models happily hallucinate APIs for, so the wiki layer (synced official docs + our own verified tips) is not optional for us. It's a lighter take on the pattern than the gist describes — the LLM acts more as searcher than librarian, and the compiled layer is human-verified tips and SKILL.md procedures rather than an LLM-maintained wiki. The part that turned out to be interesting is distribution: that compiled layer replicates to every teammate's agent on git pull.

The repo itself is the deployment. No MCP server, no RAG index, no sync service. One git repo with two layers: sources/ (the wiki: synced official docs, API references, tips) and skills/ (the agency side: SKILL.md packages, including the search skills that query sources/). A one-time setup command links each skill into ~/.claude/skills (symlinks; junctions on Windows), and a post-merge hook re-runs the sync — so daily operation for everyone on the team is literally git pull.

Notes from living with it:

• Working copy = live instance changes behavior. The linked skills point into each member's checkout, so editing a skill + git push is the entire publish flow. The unexpected outcome: our designers — not engineers — started pushing skills. The contribution barrier is "edit a markdown file," and that turned out to be low enough for non-engineers.
• Retrieval stayed a skill in the repo, not a service. The template ships a ranked-grep search skill (generate keyword variants → grep sources/ → rank the hits). In our team repo we also run a hybrid semantic skill — embedded Chroma + a local multilingual MiniLM, RRF-fused with the grep results, index rebuilt incrementally by the skill itself — and honestly that's the one I reach for most: for natural-language "how do I…" queries over our (mostly Japanese) docs it beats keywords. So my data point for the grep-vs-RAG debate isn't "grep wins" — it's that at hundreds of pages, both flavors of retrieval fit inside the repo as skills, with nothing to host.
• A failure worth writing down: we first tried a shared .claude/ in a parent folder above all projects. Claude Code doesn't pick it up for the git-managed projects underneath it. Per-skill links into ~/.claude/skills was the reliable path.

Template (MIT): https://github.com/umezy/team-ai-workflows

Reading down this thread, nearly every tool here — mine included — exists to keep the model's memory current. But the pattern puts the judgment on the human: "you're in charge of sourcing, exploration, and asking the right questions." Nothing in it keeps that side from decaying. Your wiki stays current; your memory of why you decided things doesn't.

So I built the other half of the loop. The wiki quizzes you back — day-granular spaced repetition (1·3·7·16·35·60) over the decision and insight pages only, since those are the ones whose why rots. It's the one part of the system that spends your time instead of saving it.

Two things I didn't expect while building it:

Citations don't survive leaving your machine. A footnote pointing at session-abc.jsonl is provenance for exactly one person. Pages now carry 1–2 verbatim lines of the evidence inline — secret-screened, because transcripts really do contain credentials — so a teammate gets the grounding and not just the conclusion.

Indexing that evidence made pages harder to find. BM25 length normalization: the excerpt lengthens the page's own chunk, which lowers its score for its own topic. Excluding excerpts from the index put recall@5 back to baseline.

Markdown as the source of truth, local-first, no MCP, no build step. Works with Claude Code / Codex / OpenCode.

https://github.com/suwonleee/llmwiki

("llmwiki" is a crowded name in this thread — this is the one with the quiz.)

Synto v0.7.0 is out.

https://github.com/kytmanov/synto

Main addition: concept relations. Opt-in on ingest - the model extracts links between known concepts (e.g. Raft depends_on Consensus) with a short evidence quote. Packs ship the graph as graph/graph.json. Queries follow one hop to pull in related articles.

Also new in v0.7:

• reverse lookup: synto find ranks concept names, then titles, then body text.
• provenance: synto trace shows where a term shows up in sources, or which articles used a source segment.
• fix a wrong alias without a merge: synto concept alias add|remove|move.
• silence known lint noise: ack advisories in synto.toml so they stop cluttering maintain.

Same shape as before:

• works with local LLMs
• great with Ollama and LM Studio
• plain Markdown
• Obsidian-friendly
• no vector DB
• no cloud required
• multi-language

Star it if you want to support local-first AI tools. Fork it if you want to build on it.

CItadel 0.3.0

This clicked hard — especially the contrast with RAG-as-rediscovery and the three-layer split (immutable sources → LLM-maintained wiki → schema). I've been running a team version of the same idea, and it maps cleanly onto your ingest/query/lint loop.

For a personal wiki, your Obsidian + git + index.md approach is hard to beat at moderate scale. For a team, we hit different walls: who owns writes, what gets shared vs stays draft, semantic drift when five agents touch the same "entity page," and capture that can't depend on someone remembering to ingest after every meeting.

We built Citadel as that team layer — an Organization Vault on Cognee (Apache-2.0). It's not "markdown in a folder" (though markdown/Obsidian still fit as sources and UI):

Node (seat:{slug}): private working memory; default target for autonomous capture (git pre-push + session hooks, fail-silent)
Central: shared org memory; read-only for seats, evolves via governed promotion + org sync (GitHub/Linear digests)
MCP + CLI: agents search before coding; citadel_search replaces "read index.md first" at scale
Lint-ish: open Knowledge Conflicts instead of silently merging contradictions
Re @geetansharora team question: we did end up at MCP, but over a maintained vault rather than a static RAG index — closer to your wiki compounding than chunk retrieval.

Repo + docs:

https://github.com/masumi-network/Citadel-Archive
Connect an agent (MCP setup): https://citadel-archive-production.up.railway.app/skills/connect
Install the agent skill: npx skills add masumi-network/Citadel-Archive
Would love to see a future where a Karpathy-style personal git wiki can promote curated pages into an org Central. Citadel is the org half today; the personal half still looks a lot like this gist.

I've been documenting everything LLM Wikis for the past 2+ months: https://github.com/gavischneider/awesome-llm-wiki

If you'd like your implementation/tool/guide/article to be added, PRs are welcome.