*Model changed to anthropic/claude-opus-4-6*
*Thinking level: high*
Search for existing work on using AI and coding agents for turning an existing software project into a literate programming text.
anthropic/claude-opus-4-6 · 25,484 tokens · $0.1642
Let me start by reading today's context, then research this topic. Now let me research the topic thoroughly with varied search angles. Good overview. Let me dig deeper into the most promising specific papers and projects. Now let me fetch the most relevant primary sources for deeper detail. Now let me get the HN discussion thread which looked very relevant: Now I have a thorough picture. Here's my synthesis:
The question is essentially about "reverse literate programming" – instead of Knuth's original flow (write prose → tangle out code), using AI to take an existing codebase and weave it into a readable, narrative document. This is an emerging but not yet mature area. No single tool does this end-to-end today, but there's significant converging work.
- Paper: arXiv 2408.04820
- The closest thing to what you're asking about. Google researchers propose LLM-generated "NL outlines" – concise prose statements that partition code into logical sections, summarizing each in literate programming style.
- Key results: 60% of outlines rated "excellent" by professional devs, 90% "acceptable or better." Gemini 1.5 Pro and Flash performed best.
- Bidirectional sync: They built a "Finish Changes" prototype where editing the prose updates the code and vice versa – the first practical bidirectional literate programming with LLMs.
- Applied to: code understanding, code review (Virtual CL Split), and malware detection (26/30 reverse engineers found outlines "very/extremely helpful").
- This is function-level, not whole-project narrative yet, but they discuss scaling to files and projects hierarchically.
- Paper: arXiv 2502.17441 (Interoperable Literate Programming – ILP)
- Introduces ILP: using LP principles to enhance LLM code generation in large-scale projects. Proposes prompt engineering methods for writing LP documents that LLMs can work with. Tests on RepoBench with Scheme and Python.
- More about using LP format for LLMs rather than converting existing code to LP, but establishes the framework.
- Paper: research.chalmers.se/publication/549267
- Tests LLMs on Rosetta Code (1,228 tasks, 926 languages) for literate programming capabilities. Finds trillion-parameter models (GPT-4 class) can recognize programming tasks from code snippets and generate aligned NL descriptions – essentially the core capability needed for reverse-LP.
- blog.apiad.net
- Argues LP is uniquely suited for AI-assisted coding because the document inherently contains all context an LLM needs. Built
illiterate– a Rust CLI tool for LP whose own source code is written as a literate program. - Key insight: AI can help with both the code and the prose – refining explanations, suggesting better analogies, improving narrative quality.
- Limitation he identifies: LP macro expansion (the killer feature over notebooks) isn't supported by any AI tooling yet.
- dev.to/sukima
- Observes that LLM code generation accidentally embodies literate programming ideals – models naturally produce essay-like responses with interspersed code, chain-of-thought explanations following top-down logic.
- news.ycombinator.com/item?id=47300747
- 100+ comments, very rich discussion. Key takeaways:
- Jeremy Howard (nbdev creator): "Nearly all my coding for the last decade or so has used literate programming. I built nbdev... Over the last couple of years we integrated LLMs with notebooks and nbdev to create Solveit."
- Practical LLM-comments approach: One dev instructs LLMs to write
<remarks>XML comments with chain-of-thought reasoning – creating "just-in-time long-term memory" that future agents pick up when they read the code. - "Ephemeral prose" counter-argument: If agents can read and explain code on demand, maybe LP is unnecessary – code is the stable interface, prose is generated ephemerally when needed.
- Mech language: A literate programming language co-designed with AI, where the document is queryable at runtime and AI has access to live type information (mech-lang.org).
- mattrickard.com
- Short but sharp: notes LLM prose serves dual purpose – (1) documentation and (2) chain-of-thought reasoning. LLMs solve the notebook-to-production gap that plagued Jupyter.
| Tool | What it does | How close to "code→LP text" |
|---|---|---|
| Google NL Outlines (research prototype) | Generates interleaved prose+code outlines for functions | Closest – bidirectional sync, developer-tested |
| nbdev / Solveit (fast.ai) | Notebook-based LP with LLM integration | Mature LP tool, LLM integration newer |
| illiterate (Piad) | LP CLI tool in Rust with macro expansion | Tool for writing LP, not converting to it |
| Mech/Mechdown | LP language with AI co-writing, live runtime context | Research/prototype stage |
| C3 Generative AI | Parses legacy codebases, auto-generates traceable docs | Enterprise, not quite LP format |
| Mintlify | AI-generated docs that sync with code changes via PRs | Documentation, not narrative LP |
| Lean 4 + Verso | Used for literate-style code review of AI-generated code | Niche (theorem proving) |
What doesn't exist yet (and what your question points at):
An agentic tool that takes an existing repository and produces a coherent, narrative literate programming document – where the entire project can be read as a book, with prose explaining the architecture, design decisions, and implementation details, woven with the actual code.
The pieces are clearly there:
- LLMs can generate high-quality code explanations (Google NL Outlines proves this at function level)
- LLMs can understand project architecture (Claude Code, Cursor, etc. already do this in chat)
- Bidirectional sync is demonstrated (Google's "Finish Changes")
- LP tooling exists (
illiterate, nbdev, org-mode, noweb)
What's missing is the agentic orchestration – an agent that:
- Traverses a codebase systematically
- Builds a hierarchical narrative (project → module → function)
- Chooses the right order for human understanding (not compiler order)
- Generates macro-style cross-references
- Maintains the document as code changes
This feels very buildable with current technology. The Google NL Outlines paper's hierarchy discussion (Appendix J) essentially sketches this: generate function outlines → use them as building blocks → LLM reasons about file/project level → produces layered narrative.
Want me to dig deeper into any of these threads, or sketch what a concrete tool/agent for this would look like?
Which different literate programming syntaxes and tools were mentioned in your sources?
anthropic/claude-opus-4-6 · 123,541 tokens · $0.1372
Good question – the sources mentioned quite a few. Let me comb through everything I fetched systematically.
| Tool | Language(s) | Notes |
|---|---|---|
| WEB | Pascal | Knuth's original (1984). Tangling + weaving with TeX. |
| CWEB | C/C++ | Knuth & Levy's C adaptation of WEB. |
| noweb | Language-independent | Simpler LP system with hyperlinked code chunks. Mentioned in HN discussion and Wikipedia references. |
Literate Haskell (.lhs) |
Haskell | Native LP support in the language – mentioned in search results as a format option. |
| TeX + C++ | C++ | Used by Physically Based Rendering (pbr-book.org) – cited as a major real-world LP success. |
| Tool | Notes |
|---|---|
| Jupyter Notebooks | Mentioned by nearly every source. Piad's blog calls them "a reimagining of LP for data science" but notes they lack recursive macro expansion – code cells must be complete and executable, which "goes head-to-head with LP's core idea." Rickard notes they suffer from out-of-order execution, hard to test, hard to version. |
| Mathematica Notebooks | Mentioned in HN by the Org-mode LP practitioner as the original inspiration for notebook interfaces to programs. |
| Quarto | Mentioned by Piad as sharing LP principles but lacking recursive macro expansion (ultimately Jupyter-based). |
| Emacs Org-mode with Babel | Mentioned in HN discussion and web search results. One experienced dev writes all their Lisp libraries in Org-mode, calling it "a notebook interface to a program." Tests are embedded as executable examples. No tangling needed for Common Lisp due to the language's flexibility. |
| Tool | Language(s) | Notes |
|---|---|---|
| nbdev (fast.ai) | Python | Jeremy Howard (in HN thread): "Nearly all my coding for the last decade or so has used literate programming. I built nbdev." Lets you write, document, and test software in notebooks. |
| Solveit (fast.ai) | Multi | Built on nbdev + LLM integration. Howard: "everyone at our company uses [it] for nearly all our work (even our lawyers, HR, etc)." |
| illiterate (Piad) | Language-independent | Rust CLI tool for LP with full macro expansion and tangling. Its own source code is written as a literate program. github.com/apiad/illiterate |
| Mech / Mechdown | Mech language | LP tool co-designed with the host language. Document is queryable at runtime; AI gets live type info and intermediate values. Plan: AI generates code using live context, executes safely, integrates with reference to the generating conversation. mech-lang.org |
| Mogan | Multi | Mentioned in the ILP paper (2502.17441) as a unified editor that simplifies ILP workflows for complex repositories. |
| Pandoc | Multi | Mentioned by Piad as sharing some LP principles (code + text conversion). |
| Format | Language(s) | Notes |
|---|---|---|
| Rakudoc / Podlite | Raku, Perl, JS/TS | Mentioned in HN with a full example. Embeds structured docs (headings, synopses, parameter descriptions, examples) directly in source. podlite.org for language-neutral variant. |
Rust doc comments (///, //!) |
Rust | Mentioned in HN as approaching LP – extensive docs with testable examples inline, hyperlinks, rendered to HTML. |
| Go doc comments | Go | Similarly mentioned – one dev calls their approach "lighter LP": module-level docs + public API docstrings + Google Go style guide. |
| Python doctests | Python | Mentioned in HN as "we actually have had literate programming for a while" – executable examples in docstrings. |
| JSDoc / C# XML comments | JS/TS, C# | One HN commenter uses <summary> (brief) + <remarks> (chain-of-thought, business context, design decisions) as a practical LLM-era LP approach. LLM writes comments first, reads them back in future sessions. |
| Doxygen | C/C++, Java, etc. | Mentioned in search results as a traditional doc generator that parses code comments for HTML/PDF output. |
| Lean 4 + Verso | Lean 4 | Mentioned in HN: used for literate-style code review of AI-generated code, with proper typechecking integrated. |
| Tool / Concept | Notes |
|---|---|
Star comments (#*, //*) |
Proposed by Google's NL Outlines paper. Regular comment syntax + asterisk to mark AI-generated outlines. #*! for verified/human-approved outlines. Rules: treat cautiously, expect AI tools to change/add/remove them. |
| NL Outlines (Google) | Not a syntax per se but a specific format: concise prose statements partitioning code into logical sections, shown interleaved or standalone. Generated via Interleaved Generation, Constrained Generation, or Line Number Infilling prompting techniques. |
| "Master prompt" approach | HN commenter: maintains a prose specification document alongside sparsely-documented code. Both are fed to LLMs for coherent development. When code deviates from spec, spec gets updated. |
LLM <remarks> pattern |
HN: instruct LLMs to dump reasoning into structured comments. Future agents read these as "just-in-time long-term memory." The LLM writes the comment before the code, acting as "a mini-spec right before output tokens." |
| Ephemeral prose | Counter-proposal from HN: don't persist prose at all. Code is the stable artifact; generate explanations on demand when humans need them. "I am already doing that. For performance, I am just caching the latest explanation alongside the code." |
| Tool | Notes |
|---|---|
| C3 Generative AI | Parses legacy codebases into structured representations, auto-generates traceable documentation. |
| Mintlify | AI-generates docs, syncs via PRs, API references. |
| CodeGPT | VS Code extension – highlights code → AI docs with params, returns, examples. |
| Docuwriter.ai | Generates code comments, docstrings, READMEs, API docs from codebase. |
| Kodesage | Legacy system documentation with visualizations. |
None of the sources mentioned these LP-related tools, which I'd have expected:
- Entangled (modern LP with live sync)
- Eve (Chris Granger's LP-inspired language)
- Pweave (Python LP tool)
- Codebraid (Pandoc-based LP)
Want me to research any of these tools in more depth, or look into specific syntaxes/formats that might work well for an AI agent doing code→LP conversion?
My vision is a coding agent skill/tool which would turn a complete software project into a literate programming book. There would be "milestones" within the book – specific dividers which would guarantee a working version of the software when tangled from beginning of the book until the divider. This would help the reader get intermediate source code trees to test and study. This feature requires the ability to replace parts of the code in later milestones, since sometimes only the ability adding code might prevent doing straightforward solutions.
anthropic/claude-opus-4-6 · 126,673 tokens · $0.8485
This is a compelling vision – essentially a pedagogical reconstruction of a project, where the agent reverse-engineers not just what the code does but a good order in which to teach it, with each milestone being a working checkpoint.
Let me think through the key design dimensions.
This is the central technical challenge, and it's worth understanding how existing LP systems handle it, because your milestones need something most of them don't offer cleanly.
What you need: A chunk defined in milestone 1 can be replaced (not just appended to) in milestone 3. Tangling up to milestone 3 uses the later definition.
How existing systems handle this:
| System | Append | Replace | Notes |
|---|---|---|---|
| WEB/CWEB | Yes (section refinement) | Partial – sections can be "continued" but the model is additive | Knuth's own TeX source uses progressive refinement, but it's designed for one final output |
| noweb | <<chunk>>+= |
No native replace | You'd have to use a different chunk name, breaking the narrative flow |
| Org-mode Babel | Multiple blocks with same name concatenate | No clean replace | You can use :tangle per-block, but no milestone concept |
| illiterate (Piad) | Macro expansion | Macros are defined once | No redefinition |
| Entangled | Append | No | Focus is on live sync with files, not versioned output |
| Literate Haskell | N/A (linear) | No | Code appears in-order, no chunk system |
None of these natively support your milestone model. The closest conceptual precedent is actually Knuth's own practice in TeX: The Program – he uses progressive refinement where later sections flesh out earlier placeholders – but that's additive (filling in ⟨do something 42⟩), not replacement.
You need a syntax that supports versioned chunks. Something like:
<<file:src/parser.py section:tokenize milestone:1>>=
def tokenize(source: str) -> list[Token]:
# Simple whitespace splitting
return [Token(word) for word in source.split()]
@
... (narrative explains why this is a starting point, what it lacks) ...
<<file:src/parser.py section:tokenize milestone:3 replaces:1>>=
def tokenize(source: str) -> list[Token]:
"""Proper tokenizer with regex-based lexical analysis."""
tokens = []
for match in TOKEN_PATTERN.finditer(source):
tokens.append(Token(match.group(), match.lastgroup))
return tokens
@
Tangling to milestone 2 gives you the simple splitter. Tangling to milestone 3 gives you the regex tokenizer. The narrative in between explains why the simple version was good enough to start with and what drove the need for proper lexing.
The agent would need to do several very different things, each hard in its own way:
This is the hardest part and the most creative. The agent needs to answer: "If I were teaching someone this project from scratch, what order would I introduce things in?"
This is NOT the order the code was written (git history is full of false starts and refactors). It's not alphabetical. It's not dependency order (that's bottom-up and boring). It's a narrative order – the order that builds understanding.
Heuristics an agent could use:
- Core loop first: What's the minimal thing this program does? Start there.
- Dependency analysis: What can stand alone? What requires other pieces?
- Conceptual layering: Types/data model → core logic → I/O → error handling → optimization → polish
- Test-driven milestones: Each milestone should be testable – that constrains the ordering.
Each milestone must produce a program that compiles/runs/passes some tests. This means the agent needs to:
- Identify which subset of files/functions constitutes a working subset
- Potentially write stub/placeholder code for early milestones (e.g.,
def handle_auth(request): return True # milestone 4 adds real auth) - Plan replacements ahead of time – knowing that the stub in milestone 1 will be replaced by real code in milestone 4
- Actually tangle and test each milestone during generation to verify it works
This is where LLMs are actually quite good. The agent would generate:
- Why this piece comes now (motivation, what problem it solves)
- What design decisions were made and why (tradeoffs, alternatives considered)
- How it connects to what came before and what's coming next
- What to notice – pointing the reader to interesting patterns, idioms, or gotchas
When a milestone replaces earlier code, the narrative needs to explain the evolution:
"Our simple whitespace tokenizer from Milestone 1 served us well for parsing basic expressions, but now that we're adding string literals and operators, we need proper lexical analysis. Here's the replacement..."
This is actually a strength of the format – it mirrors how real software evolves.
A few choices worth thinking about:
Chunk granularity: File-level replacement is simpler (replace entire src/parser.py) but loses the surgical precision of replacing just one function. Section-level replacement (named chunks within files) is more powerful but needs more complex syntax.
Output format: The "book" itself could be:
- Markdown with fenced code blocks + custom milestone/chunk markers (simple, portable, renders anywhere)
- Org-mode (powerful tangling built in, but niche audience)
- A custom format with its own tangler (most flexible, most work)
- Jupyter Book / mdBook / Quarto (nice rendering, but tangling needs custom tooling)
My instinct: Markdown + a small custom tangler is the sweet spot. The syntax should be minimal enough that the book is pleasant to read as plain text, and the tangler is a small standalone tool (~200 lines of Python).
Milestone semantics: Two models:
- Cumulative: Tangle collects all chunks from start to milestone N, later definitions shadow earlier ones. Simpler.
- Explicit: Each milestone declares which files/chunks it modifies. More verbose but clearer.
I'd lean toward cumulative with explicit replaces annotations for clarity.
A tutorial is written by a human who decided what to teach. Your vision is an agent that reverse-engineers the teaching order from finished code. That's a genuinely novel contribution – I didn't find anything in my research that does exactly this.
The closest things:
- Google's NL Outlines do function-level explanation but not project-level narrative ordering
- nbdev/Solveit supports writing new code in LP style but doesn't convert existing projects
- Git history replay tools show what happened but not in pedagogical order
- Architecture documentation generators (C3, Mintlify) produce reference docs, not narrative
Shall I sketch a concrete syntax proposal and/or a skill design for the agent workflow?
Which of all the tools mentioned so far (also the ones you wondered about not being mentioned in the sources) uses a syntax most suitable for our vision, and could be extended with replacement capability?
anthropic/claude-opus-4-6 · 129,203 tokens · $0.8542
Let me check the tools I'm less certain about before comparing them all. This is a clear winner. Let me lay it out.
LMT already has chunk replacement as its default behavior. This is the crucial differentiator. Every other LP tool treats redefinition as either an error or an append – LMT treats it as an overwrite, which is exactly what your milestone model needs.
LMT has two kinds of targets: files and macros.
Macros (quoted names) – last definition wins:
```python "tokenize"
def tokenize(source):
return source.split() # simple version
```... narrative explaining why this suffices for now ...
def tokenize(source):
return [Token(m.group(), m.lastgroup)
for m in TOKEN_RE.finditer(source)]The second definition replaces the first. All <<<tokenize>>> references throughout the document resolve to the latest definition at time of tangling.
Files (unquoted names) – append with +=:
```python src/parser.py +=
<<<imports>>>
<<<tokenize>>>
<<<parse>>>
```Macro references (<<<name>>>) expand recursively.
LMT processes files sequentially. If the tangler simply stops processing at a milestone marker, it uses whatever definitions exist up to that point:
- Tangle to milestone 1: sees the simple
tokenize, produces working code - Tangle to milestone 3: sees the regex
tokenize(which replaced the simple one), produces working code
The extension needed is minimal – just a milestone directive and a --milestone N flag on the tangler. The core replacement semantics already work correctly.
This is the clincher. LMT's own source code uses progressive enhancement across multiple markdown files:
Implementation.md– basic implementationWhitespacePreservation.md– adds whitespace handlingSubdirectoryFiles.md– adds subdirectory supportLineNumbers.md– adds line number pragmasIndentedBlocks.md– adds indented block support"Larger features can be included as a patch in a new file."
Each file redefines macros from earlier files. Process only the first file, you get a basic tangler. Process all five, you get the full tool. This is your milestone model, already working in practice.
| Feature | LMT | Entangled | noweb | illiterate | Codebraid |
|---|---|---|---|---|---|
| Macro replacement (last-wins) | ✅ Default | ❌ Appends | ❌ Appends | ❌ | ❌ |
Macro append (+=) |
✅ | ✅ (same #id) |
✅ (+=) |
✅ | ❌ |
| Markdown-native | ✅ Fenced blocks | ✅ | ❌ (own syntax) | ✅ | ✅ |
| Language-agnostic | ✅ | ✅ | ✅ | ✅ | ✅ |
| Sequential processing (order matters) | ✅ | ❌ (all at once) | ✅ | ? | ✅ |
| Multi-file source | ✅ (CLI args) | ✅ (glob) | ✅ | ❌ | ✅ |
| Hidden content (tangle but don't render) | ✅ (HTML comments) | ❌ | ❌ | ❌ | ❌ |
| Self-hosted (written in itself) | ✅ | ❌ | ❌ | ✅ | ❌ |
| Line number pragmas | ✅ | ❌ | ✅ | ❌ | ❌ |
| Bidirectional sync | ❌ | ✅ | ❌ | ❌ | ❌ |
The extensions for your vision are modest:
- Milestone markers – e.g.,
<!-- milestone: 3 "Adding proper tokenization" -->or a fenced directive --milestone Nflag – tangler stops processing at milestone N- Milestone validation – after tangling to each milestone, run a test command to verify the output works
- Per-milestone file list – track which files exist at each milestone (a file introduced in milestone 2 shouldn't appear in milestone 1's output)
Strengths for your use case:
- Written in Go – single binary, fast, easy to fork
- ~600 lines of Go (tangled from the markdown source)
- Clean, minimal design – no framework dependencies
- The replacement semantics are battle-tested (LMT uses them on itself)
- Hidden content via HTML comments is useful for scaffolding code the reader shouldn't see yet
Limitations to consider:
- Small project, single maintainer (driusan)
- No weave step (relies on standard Markdown renderers) – which is actually fine for your case
- Global replacement scope: a macro redefinition affects ALL references, not just forward ones. This means milestone 1's tangled output would retroactively use milestone 3's definition if you processed the whole file. The
--milestone Nstopping point solves this, but it means the book must be structured so that each milestone's code is self-consistent when processed up to that point.
Entangled has the better ecosystem (Python, MkDocs/Quarto integration, pip-installable, active development) but lacks replacement semantics – named blocks with the same #id concatenate rather than overwrite. You'd need to add a replaces attribute:
```{.python #tokenize replaces}
...
```This is doable but changes Entangled's core semantics, which feels like swimming upstream. With LMT, you're extending something that already flows in the right direction.
Want me to look more closely at LMT's implementation to assess how hard the milestone extension would be, or sketch the agent skill that would generate these books?
Exported from Pi session a4848b6b-a43e-488c-9e76-24d4a30ab65c