Skip to content

Instantly share code, notes, and snippets.

@dfinke
Last active August 11, 2026 12:48
Show Gist options
  • Select an option

  • Save dfinke/9251aae7265eef0bb11160243eb423e1 to your computer and use it in GitHub Desktop.

Select an option

Save dfinke/9251aae7265eef0bb11160243eb423e1 to your computer and use it in GitHub Desktop.
# Install-Module PSAISuite -Force
$prompt = @"
read and understand llm-wiki.md
show a plan only for excuting and implementing the steps in llm-wiki.md
"@
$iccParams = @{
model = "openai:gpt-5.6-luna"
effortLevel = "high"
tools = "Get-ChildItem", "Get-Content", "Set-Content", "New-Item", "Get-Location"
messages = $prompt
}
Invoke-ChatCompletion @iccParams

Implementation Plan for llm-wiki.md

Phase 1: Assess the Existing Workspace

  1. Inventory the current repository and existing 2025 content.
  2. Determine whether the existing files are:
    • Raw source documents
    • Existing wiki pages
    • Journals or personal notes
    • Attachments or images
  3. Decide whether to:
    • Preserve the current structure and add the wiki around it, or
    • Migrate content into a new wiki structure.
  4. Identify the chosen LLM agent and its instruction-file convention, such as:
    • AGENTS.md
    • CLAUDE.md
    • Another project-specific schema file.
  5. Establish privacy, backup, and version-control requirements before processing personal or sensitive material.

Phase 2: Define the Wiki Architecture

Create and document the three required layers:

raw/       Immutable source documents
wiki/      LLM-maintained markdown knowledge base
AGENTS.md  Wiki schema, rules, and workflows

Proposed wiki structure:

wiki/
├── index.md
├── log.md
├── overview.md
├── synthesis.md
├── sources/
├── entities/
├── concepts/
├── topics/
└── analyses/

The exact categories should be adjusted after reviewing the existing content and intended domain.

Phase 3: Establish Page and Metadata Conventions

Define the schema for each page type.

Source pages

Specify conventions for:

  • Source title
  • Author or creator
  • Publication date
  • Ingestion date
  • Original location or URL
  • Source type
  • Source file link
  • Summary
  • Key claims
  • Important quotations
  • Related entities and concepts
  • Limitations or reliability notes

Entity and concept pages

Specify conventions for:

  • Definition or identity
  • Current understanding
  • Key claims
  • Evidence and source citations
  • Related wiki pages
  • Contradictions
  • Open questions
  • Last-reviewed date

Analysis pages

Specify conventions for:

  • Question or objective
  • Relevant sources
  • Method or reasoning
  • Findings
  • Uncertainty
  • Conclusions
  • Links to affected wiki pages

Global rules

Document rules for:

  • Markdown formatting
  • File naming
  • Internal links
  • YAML frontmatter, if used
  • Citation format
  • Claim confidence
  • Handling conflicting sources
  • Distinguishing source facts from LLM synthesis
  • Never modifying files under raw/
  • Avoiding unsupported claims
  • Updating backlinks and indexes

Phase 4: Write the LLM Schema Instructions

Create the agent instruction file containing:

  1. The purpose of the wiki.
  2. The directory structure.
  3. Page templates and metadata requirements.
  4. Source immutability rules.
  5. Required behavior for ingestion.
  6. Required behavior for answering questions.
  7. Required behavior for linting and maintenance.
  8. Citation and uncertainty requirements.
  9. Naming and linking conventions.
  10. Rules for updating index.md and log.md.
  11. Rules for asking the human for clarification.
  12. Rules for proposing, but not silently inventing, new sources or facts.

The schema should explicitly state that the LLM owns maintenance of the wiki layer while the human remains responsible for source curation and final judgment.

Phase 5: Build the Initial Wiki Foundation

Create the initial wiki files:

  • wiki/index.md
  • wiki/log.md
  • wiki/overview.md
  • wiki/synthesis.md

Populate the index with existing pages if any are already present.

Add a short initial log entry documenting:

  • Wiki creation
  • Initial structure
  • Existing material reviewed
  • Any migration or interpretation decisions

Do not generate extensive summaries until the page conventions have been agreed upon.

Phase 6: Perform a Controlled Initial Ingest

Use a single source as a pilot.

For the pilot source:

  1. Verify that the source is present in raw/.
  2. Confirm that the raw file will remain unchanged.
  3. Read the source completely, including relevant local images or attachments.
  4. Produce a source summary and identify key claims.
  5. Identify entities, concepts, topics, and relationships.
  6. Create or update the relevant wiki pages.
  7. Record conflicting or uncertain information explicitly.
  8. Update wiki/index.md.
  9. Append an ingest entry to wiki/log.md.
  10. Review the generated changes manually.
  11. Refine the schema based on problems found during the pilot.

Repeat the pilot with a second source that overlaps with the first to test:

  • Cross-referencing
  • Page reuse
  • Contradiction handling
  • Updates to existing summaries
  • Source attribution

Phase 7: Implement the Standard Ingest Workflow

Document and use the following repeatable workflow:

  1. Add a curated source to raw/.
  2. Ask the LLM to ingest only that source or an explicitly selected batch.
  3. Have the LLM inspect the index before creating duplicate pages.
  4. Extract and summarize the source.
  5. Discuss important takeaways and emphasis with the human when appropriate.
  6. Create or update source, entity, concept, and topic pages.
  7. Revisit affected summaries and synthesis pages.
  8. Add cross-links in both directions where appropriate.
  9. Record contradictions, outdated claims, and unresolved questions.
  10. Update the index.
  11. Append a structured log entry.
  12. Review the final diff and generated claims.

Use a consistent log format such as:

## [YYYY-MM-DD] ingest | Source title

Phase 8: Implement the Query Workflow

Define a query workflow in which the LLM:

  1. Reads wiki/index.md first.
  2. Selects relevant pages.
  3. Follows internal links to related pages.
  4. Uses raw sources when the wiki is incomplete or a claim needs verification.
  5. Produces an answer with citations to wiki pages and, where relevant, raw sources.
  6. Clearly separates:
    • Established information
    • Synthesis
    • Interpretation
    • Uncertainty
    • Open questions
  7. Asks whether the result should be filed into the wiki.
  8. If filed, creates an appropriate analysis or topic page.
  9. Updates the index and appends a query entry to the log.

Possible query outputs should include:

  • Direct answer
  • Comparison table
  • Timeline
  • Research summary
  • Decision analysis
  • Chart or visualization
  • Slide deck
  • New wiki page

Phase 9: Implement Wiki Linting and Maintenance

Create a recurring lint workflow covering:

Structural checks

  • Broken internal links
  • Missing referenced files
  • Duplicate pages
  • Incorrect file names
  • Missing required frontmatter
  • Missing index entries
  • Missing log entries

Knowledge checks

  • Orphan pages
  • Important concepts without dedicated pages
  • Missing backlinks
  • Contradictory claims
  • Stale summaries
  • Claims lacking citations
  • Sources that have not been integrated
  • Topics with insufficient evidence
  • Pages whose synthesis no longer reflects newer sources

Maintenance output

Each lint pass should produce:

  1. Issues found
  2. Pages affected
  3. Recommended corrections
  4. Suggested research questions
  5. Possible new sources to investigate
  6. Changes requiring human approval

The lint workflow should distinguish safe mechanical fixes from substantive knowledge changes.

Phase 10: Add Validation and Safety Controls

Before considering the system operational, define checks for:

  • Raw-source immutability
  • No accidental deletion of wiki pages
  • No unsupported citations
  • No fabricated source metadata
  • No silent overwriting of human-authored material
  • Consistent links and filenames
  • Proper logging of changes
  • Reviewable Git diffs
  • Recovery from incorrect updates

Use Git commits as checkpoints for:

  • Initial setup
  • Each source ingest
  • Major synthesis updates
  • Lint corrections
  • Schema changes

Phase 11: Add Optional Tooling Only When Needed

Defer custom tools until the wiki becomes difficult to navigate manually.

Potential additions:

  1. A broken-link checker.
  2. An index-generation or validation script.
  3. A frontmatter validator.
  4. A search utility for wiki pages.
  5. qmd or another local hybrid search tool.
  6. Obsidian Dataview views.
  7. Graph-view conventions.
  8. Image and attachment handling.
  9. Marp templates for presentations.
  10. Automated Git checks before commits.

Introduce tools incrementally and retain the index-based workflow until the wiki’s size justifies additional search infrastructure.

Phase 12: Establish the Ongoing Operating Cycle

Adopt the following recurring cycle:

When adding knowledge

  • Curate source
  • Store it immutably
  • Ingest it
  • Review updates
  • Commit changes

When asking questions

  • Query the wiki
  • Cite supporting pages
  • File valuable conclusions
  • Log the interaction

Periodically

  • Run linting
  • Review contradictions
  • Refresh stale synthesis
  • Identify orphan pages
  • Investigate recommended research gaps
  • Refine the schema

Completion Criteria

The implementation should be considered complete when:

  • Raw sources and wiki-generated content are clearly separated.
  • The LLM schema file describes the structure and operating rules.
  • The wiki has an index, chronological log, overview, and synthesis.
  • At least two overlapping sources have been ingested successfully.
  • Source pages and related entity/concept pages are cross-linked.
  • Queries can produce cited answers from the wiki.
  • Valuable query results can be filed back into the wiki.
  • A lint workflow can identify structural and knowledge-maintenance issues.
  • Changes are reviewable and recoverable through Git.
  • The workflow has been refined based on the pilot ingests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment