Skip to content

Instantly share code, notes, and snippets.

@and1truong

and1truong/000 - PRD.md Secret

Last active October 10, 2025 22:10
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save and1truong/e0c490632cbb28df7b70a70a2987a0e2 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save and1truong/e0c490632cbb28df7b70a70a2987a0e2 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
000 - PRD.md

Of course. Here is the complete, clean PRD (Version 1.2) with all the performance and scalability revisions fully integrated. You can copy and paste the entire text.

Product Requirements Document (PRD): Obsidian Vault Server

Document History

Version Date Author Changes
1.0 2023-10-15 [Your Name] Initial draft based on development iteration.
1.1 2023-10-25 [Your Name] Added MCP server integration with tool definitions.
1.2 2023-10-26 [Your Name] Revised API endpoints for performance (/vault-tree, /vault-tag) to handle large vaults. Updated core architecture to serve from an in-memory archive, eliminating slow startup file extraction.

1. Overview

1.1 Product Summary

Obsidian Vault Server is a lightweight, self-contained CLI application that transforms an Obsidian Markdown-based knowledge vault into a web-accessible static server. It embeds the entire vault (compressed for minimal binary size) and provides a simple HTTP API for browsing and querying content. On startup, it scans the embedded vault into an in-memory index for high-performance access, enabling front-end applications to render dynamic views without requiring Obsidian or a full file system. Additionally, it supports integration as an MCP (Model Control Plane) server, exposing the core APIs as structured tools for AI agents or tool-calling systems (e.g., LangChain, OpenAI function calling).

The app is built in Go for cross-platform compatibility (Linux, macOS, Windows) and zero external dependencies at runtime. During build, it compresses the vault using tar + gzip (max level 9) and generates a tag index for fast querying.

1.2 Problem Statement

Obsidian vaults are powerful for personal knowledge management but are file-system bound and not easily shareable or integrable with web apps or AI workflows. Users need a portable, secure way to serve vault content over HTTP without exposing raw files or running complex servers (e.g., no Node.js, Docker). Furthermore, AI agents require standardized tool interfaces to query and retrieve vault data programmatically.

1.3 Goals

  • Primary: Provide a single binary (~20-50MB including vault) that serves an embedded Obsidian vault via HTTP.
  • Secondary: Enable API-driven exploration (tree structure, tag-based search) for front-end integration; expose MCP-compatible tools for AI agent use.
  • Success Metrics:
    • Binary size < 100MB for typical vaults.
    • Startup time < 500ms; API response < 50ms for tree and tag lookups.
    • 100% uptime during runtime; graceful shutdown.
    • Tool schema validation passes for standard MCP/tool-calling frameworks.

1.4 Scope

  • In Scope: Vault embedding/compression, static file serving, tree API, tag API, MCP tool endpoints/schema.
  • Out of Scope: User authentication, real-time editing, Markdown rendering (raw text only), multi-vault support, advanced AI orchestration (e.g., agent chaining).

2. Target Audience

  • Primary Users: Obsidian power users (developers, researchers, writers) who want to share vaults via web links or integrate with custom front-ends (e.g., React/Vue apps).
  • Secondary Users: Teams collaborating on knowledge bases; front-end developers building vault UIs; AI engineers integrating vaults into agentic workflows.
  • User Personas:
    • Alex the Developer: Builds a personal wiki; needs quick API for tree navigation and tag filtering; uses MCP tools for AI-powered querying.
    • Jordan the Writer: Shares notes publicly; requires simple static serving without hosting costs.
    • Sam the AI Engineer: Chains vault tools in an LLM agent to answer questions like "Summarize notes tagged #project".

3. Features

3.1 Core Features

  1. Vault Embedding and Compression (Build-Time):

    • Input: ./vault directory (Obsidian-compatible Markdown files, attachments).
    • Process: Walk directory, create tarball, compress with gzip level 9.
    • Output: Embedded embedded_vault.tar.gz in binary (reduces size by 70-90% for text-heavy vaults).
    • Tag Indexing: Parse .md files for tags: [foo, bar] frontmatter; generate tag_index.json (map of tag → file paths).

  2. Static File Serving (Runtime):

    • Endpoint: GET /<path> (e.g., /notes/foo.md).
    • Behavior: Scans the embedded tar.gz archive into an in-memory index on startup. Files are served directly from memory, eliminating disk I/O and temporary directories. Serves files with appropriate MIME types (text/plain for .md, octet-stream for others).
    • Security: Path sanitization to prevent traversal.

  3. Vault Tree API:

    • Endpoint: GET /vault-tree?path=<path> (e.g., ?path=notes or empty for root).
    • Response: JSON object containing the immediate children of the requested path. Directories include a flag to indicate if they contain children, allowing UIs to render an expand icon. 
      // Request: GET /vault-tree?path=notes
{
  "path": "notes",
  "children": [
    {
      "name": "sub",
      "path": "notes/sub",
      "type": "directory",
      "is_expandable": true
    },
    {
      "name": "foo.md",
      "path": "notes/foo.md",
      "type": "file"
    }
  ]
}
    • Errors: 400 (invalid path), 404 (not found), 500 (internal).

  4. Tag Search API:

    • Endpoint: GET /vault-tag/<name>?limit=50&offset=0 (case-insensitive).
      • limit (int, optional): The maximum number of results to return. Defaults to 50.
      • offset (int, optional): The starting index for the results. Defaults to 0.
    • Response: JSON object containing pagination metadata and a list of files for the current page. 
      {
  "tag": "foo",
  "pagination": {
    "offset": 0,
    "limit": 50,
    "total": 123
  },
  "files": [
    "/notes/foo.md",
    "/other/bar.md"
    // ... up to 50 files
  ]
}
    • Errors: 400 (invalid tag), 404 (no matches).

3.2 MCP Server Integration (New)

The server exposes the core APIs as MCP-compatible tools, allowing AI agents to call them via standardized function-calling interfaces (e.g., JSON schema for OpenAI tools or LangChain tools). This enables programmatic access for tasks like knowledge retrieval in LLM workflows.

  • Activation: Run with --mcp flag to enable tool endpoints and schema exposure.
  • Tool Schema Endpoint: GET /mcp/tools – Returns JSON array of tool definitions (OpenAI-compatible format). 
    [
  {
    "type": "function",
    "function": {
      "name": "get_vault_tree",
      "description": "Retrieve the children (files and subdirectories) of a specific directory path in the vault.",
      "parameters": {
        "type": "object",
        "properties": {
          "path": {
            "type": "string",
            "description": "Optional directory path to list children from (e.g., 'notes'). Empty for full vault root."
          }
        },
        "required": []
      }
    }
  },
  {
    "type": "function",
    "function": {
      "name": "get_vault_file",
      "description": "Fetch the raw content of a specific file from the vault.",
      "parameters": {
        "type": "object",
        "properties": {
          "path": {
            "type": "string",
            "description": "Full path to the file (e.g., 'notes/foo.md')."
          }
        },
        "required": ["path"]
      }
    }
  },
  {
    "type": "function",
    "function": {
      "name": "get_vault_files_by_tags",
      "description": "Retrieve a paginated list of file paths in the vault that have a specific tag (case-insensitive).",
      "parameters": {
        "type": "object",
        "properties": {
          "tag": {
            "type": "string",
            "description": "The tag name to search for (e.g., 'project')."
          },
          "limit": {
            "type": "integer",
            "description": "Optional number of results to return per page. Defaults to 50."
          },
          "offset": {
            "type": "integer",
            "description": "Optional starting index for pagination. Defaults to 0."
          }
        },
        "required": ["tag"]
      }
    }
  }
]
  • Tool Execution Endpoints:
    • POST /mcp/execute – Body: JSON with tool_name and arguments (e.g., {"tool_name": "get_vault_tree", "arguments": {"path": "notes"}}).
    • Response: JSON with result (tool output).
    • Errors: 400 (invalid args), 404 (tool not found), 500 (execution error).
  • Tool Outputs:
    • get_vault_tree(path: str) -> dict: Returns the JSON object with children for the given path, as defined in 3.1.3.
    • get_vault_file(path: str) -> str: Returns the raw file content as a string.
    • get_vault_files_by_tags(tag: str, limit: int, offset: int) -> dict: Returns a structured dictionary with pagination info and a list of matching file paths, as defined in 3.1.4.

3.3 CLI Interface

  • Command: ./obsidian-server [--port=8080] [--mcp].
  • Output: Startup message with endpoints (including MCP tools if enabled); logs errors.
  • Shutdown: Graceful on Ctrl+C (SIGINT/SIGTERM).

3.4 Build Process

  • Script: go run build.go (compresses vault, builds index).
  • Then: go build -o obsidian-server main.go (embeds files).
  • Requirements: Go 1.16+; ./vault dir.

4. User Stories

4.1 As a User...

  • I want to build the app so that my vault is embedded securely.
  • I want to start the server so that I can access my vault via browser/HTTP.
  • I want to browse the vault structure so that my front-end can render a file tree.
  • I want to search by tags so that I can find related notes quickly.
  • I want secure access so that paths can't escape the vault.
  • As an AI Engineer, I want MCP tools so that I can integrate vault querying into LLM agents.

5. Technical Requirements

5.1 Tech Stack

  • Language: Go 1.16+ (embed package, standard libs only).
  • Libraries:
    • archive/tar, compress/gzip: Compression/in-memory reading.
    • net/http: Server.
    • encoding/json: API responses.
    • regexp: Tag parsing.
  • Embedding: //go:embed embedded_vault.tar.gz and tag_index.json.
  • Build Tool: Custom build.go for compression/indexing.
  • MCP Support: Hardcoded JSON schema in code; http.Handler for /mcp/* routes.

5.2 Architecture

  • Build Phase: Walk ./vault; tar + gzip; parse tags → JSON.
  • Runtime Phase:
    • Startup: Extract to /tmp/obsidian-vault-<pid>; load tag index.
    • Server: HTTP mux with handlers for /, /vault-tree, /vault-tag/*, and /mcp/* (if enabled).
    • MCP Execution: Parse POST body, route to internal functions (e.g., tree walker, file reader, tag lookup), serialize output.
    • Shutdown: context timeout; os.RemoveAll temp dir.
  • Data Flow:
    • API/Tool Call → Temp FS → JSON/string serialization.
    • Tags: In-memory map (loaded once).

5.3 Constraints

  • Vault size: < 500MB uncompressed (for reasonable binary size and memory footprint).
  • No persistent storage; all data served from memory per run.
  • Cross-platform: Handle Windows paths during build.
  • MCP: Schema fixed; no dynamic tool registration.

6. Non-Functional Requirements

6.1 Performance

  • Startup: < 2s extraction for 100MB vault.
  • API Latency: < 100ms for tree (depth-limited if huge); < 10ms for tags/files.
  • MCP Execution: < 200ms per tool call.
  • Throughput: Handles 100 concurrent requests (static files buffered).

6.2 Security

  • Path traversal prevention: filepath.Clean + prefix checks.
  • No auth (local-only by default).
  • MCP: Validate args (e.g., path length < 1024); rate-limit POSTs.

6.3 Reliability

  • Error Handling: Log + HTTP status codes; fail-fast on startup error. MCP errors include descriptive messages.
  • Logging: Stdout for startup/errors; no files.
  • Testing: Unit tests for in-memory indexing, APIs, MCP execution (80% coverage).

6.4 Usability

  • CLI: Simple flags (incl. --mcp); clear console output.
  • APIs/Tools: RESTful, JSON-only.

7. Assumptions and Dependencies

  • Assumptions:
    • Vault follows Obsidian structure (.md files, optional attachments).
    • Tags in format tags: [foo, bar] (single line, top of file).
    • User has Go installed for build; binary is standalone.
    • MCP clients handle standard OpenAI tool schema.
  • Dependencies:
    • None at runtime.
    • Build: Go toolchain.
  • Risks:
    • Large vaults: Slow extraction (mitigate: async extract?).
    • Tag parsing: Misses complex frontmatter (mitigate: Extend regex).
    • MCP Compatibility: Schema drift (mitigate: Versioned endpoint /mcp/tools/v1).

8. Future Enhancements

  • Phase 2: Markdown-to-HTML rendering (/render/<path>); tool for rendered content.
  • Phase 3: Search API (/search?q=query); authentication; dynamic tool registration.
  • Phase 4: Multi-vault support; Docker image; advanced MCP (e.g., streaming responses).

Approval

  • Product Owner: [TBD]
  • Engineering Lead: [TBD]
  • Status: Updated Draft – Ready for Review

Next step

Below is a well-structured prompt you can copy-paste or adapt for your Project Manager (PM) and Scrum Master (SM). It's designed to guide them in collaboratively breaking down the Obsidian Vault Server PRD (version 1.1) into actionable, workable user stories. The focus is exclusively on the server (backend) features, excluding the front-end viewer. I've emphasized Scrum best practices: using INVEST criteria for stories, grouping into epics/themes, including acceptance criteria, and prioritizing for an initial sprint or backlog.

Prompt for Project Manager and Scrum Master: Breaking Down Obsidian Vault Server PRD into Workable User Stories

Context:
We have the updated PRD for the Obsidian Vault Server (v1.1), which includes core features like vault embedding/compression, static file serving, Vault Tree API, Tag Search API, MCP server integration, CLI interface, and build process. Our immediate focus is on implementing the server backend only—no front-end work yet. The goal is to create a prioritized backlog of user stories that can be pulled into sprints, ensuring the server is buildable, runnable, and testable end-to-end.

Your Task:
As PM and SM, collaborate to decompose the PRD (sections 3-5, plus existing user stories in section 4) into 15-25 user stories. Group them into 3-5 high-level Epics/Themes (e.g., "Build-Time Preparation", "Runtime Serving", "API Endpoints", "MCP Integration").

Guidelines for Stories:

  • Format: Use the standard user story template: As a [user/persona], I want [feature] so that [benefit].
    • Personas: Draw from PRD section 2 (e.g., Alex the Developer, Jordan the Writer, Sam the AI Engineer).
  • INVEST Criteria: Ensure stories are:
    • Independent (can stand alone).
    • Negotiable (details flexible).
    • Valuable (delivers user/business value).
    • Estimable (sized for 1-3 days of effort).
    • Small (fits in one sprint; split large ones).
    • Testable (clear acceptance criteria).
  • Acceptance Criteria: For each story, include 3-6 bullet points with testable outcomes (e.g., "Run command X; verify output Y"). Reference PRD specs (e.g., endpoints, JSON schemas, error codes).
  • Prioritization: Use MoSCoW (Must-have, Should-have, Could-have, Won't-have) or business value/effort matrix. Flag 5-8 stories as "Sprint 1 Ready" (e.g., core build and serving).
  • Technical Notes: Stories should align with Go tech stack (section 5.1); include any dependencies (e.g., build script before runtime). Assume unit/integration tests for 80% coverage (section 6.3).
  • Output Structure:
    1. Epics/Themes: List with brief description and contained stories.
    2. User Stories: Numbered list, each with:
      • Story title.
      • User story sentence.
      • Acceptance criteria (bullets).
      • Priority (Must/Should/etc.).
      • Estimated effort (e.g., 3 story points).
    3. Sprint 1 Backlog: Top 5-8 stories ready to groom.
    4. Risks/Dependencies: Bullet list of any cross-cutting concerns (e.g., temp dir cleanup).

Deliverable:
Share a Google Doc, Notion page, or Markdown file with the full backlog by [insert deadline, e.g., EOD Friday]. Schedule a 30-min refinement session next week to validate. This will feed directly into Jira/Trello for sprint planning.

Key PRD Sections to Decompose:

  • 3.1: Vault Embedding/Compression, Static Serving, Tree API, Tag API.
  • 3.2: MCP Integration (tools schema, execution endpoint).
  • 3.3-3.4: CLI and Build Process.
  • 4.1: Existing user stories (enhance with AC).
  • 5.2-5.3: Architecture/Constraints (e.g., path security, performance).

Let's make this server rock-solid for Alex, Jordan, and Sam—focus on quick wins like embedding and basic APIs first!

Raw
001 - PRD - viewer.md

Product Requirements Document (PRD): Obsidian Vault Viewer

Document History

Version Date Author Changes
1.0 2023-10-20 [Your Name] Initial draft based on backend spec integration.

1. Overview

1.1 Product Summary

Obsidian Vault Viewer is a lightweight, single-page web application (SPA) that connects to the Obsidian Vault Server backend to provide an interactive, browsable interface for reading vault content. It renders the vault's file tree in a sidebar, displays Markdown files as formatted HTML in a main viewer, and supports tag-based filtering. The viewer is built with vanilla JavaScript (no build tools or frameworks) for minimal footprint and easy deployment, assuming the backend serves static assets (e.g., via a simple HTTP handler addition to the server).

The app fetches data via the backend's HTTP APIs and handles client-side rendering, enabling a responsive, Obsidian-like experience without requiring server-side rendering.

1.2 Problem Statement

While the Obsidian Vault Server exposes raw APIs and files, users need an intuitive web UI to navigate, read, and explore vault content without manually crafting API calls or parsing Markdown in a browser. This front-end bridges the gap, making the vault accessible like a web-based wiki.

1.3 Goals

  • Primary: Deliver a read-only web interface for vault browsing and content viewing.
  • Secondary: Integrate seamlessly with the backend APIs; support offline-capable assets (e.g., CSS/JS bundled).
  • Success Metrics:
    • Page load < 2s; API fetches < 500ms (aligned with backend).
    • 100% mobile-responsive; renders 95% of standard Markdown (tables, links, etc.).
    • Zero runtime dependencies; single HTML file + assets (< 500KB total).

1.4 Scope

  • In Scope: Tree sidebar, Markdown viewer, tag search pane, basic navigation (back/forward).
  • Out of Scope: Editing, user auth, full-text search (use backend Phase 3), wiki-style linking (render internal links as clickable).

2. Target Audience

  • Primary Users: Same as backend (Obsidian users sharing/reading vaults via web).
  • Secondary Users: Non-technical readers accessing shared vaults; front-end devs customizing the UI.
  • User Personas:
    • Alex the Developer: Uses the viewer to prototype vault UIs; needs fast tree navigation and tag filters.
    • Jordan the Writer: Reads shared notes in a clean, formatted view; appreciates mobile support.

3. Features

3.1 Core Features

  1. Vault Tree Sidebar:

    • Fetches: GET /vault-tree?root= (root) on load; recursive sub-paths on expand.
    • UI: Collapsible tree (dirs first, alphabetical); click file to load content.
    • Behavior: Highlights current file; supports keyboard navigation (arrow keys).

  2. Content Viewer:

    • Fetches: GET /<path> for raw Markdown.
    • Rendering: Client-side Markdown-to-HTML using a lightweight library (e.g., marked.js).
    • UI: Main pane with scrollable, styled content; supports images/attachments (inline if is non-.md).
    • Enhancements: Internal links (e.g., [[other.md]]) rendered as clickable paths; table of contents auto-generated.

  3. Tag Search Pane:

    • UI: Search input; dropdown of popular tags (fetched from backend index if exposed, or derived from tree).
    • Fetches: GET /vault-tag/<name> on search; displays results as clickable file list.
    • Behavior: Filters sidebar tree to show only matching files; clears on empty search.

  4. Navigation and Controls:

    • Breadcrumbs: Path-based (e.g., Home > notes > foo.md).
    • Back/Forward: Browser history integration (pushState).
    • Theme Toggle: Light/dark mode (CSS classes).

3.2 Configuration

  • Server URL: Configurable via data-server-url attribute on <body> (default: http://localhost:8080).
  • Deployment: Single index.html + app.js + styles.css; serve from backend root (/) or external host.

3.3 Error Handling

  • Network errors: Retry fetch with exponential backoff; show user-friendly messages (e.g., "Server offline").
  • Invalid paths: Redirect to root tree.
  • Loading states: Spinners for tree/content fetches.

4. User Stories

4.1 As a User...

  • I want to load the viewer so that I can immediately see the vault structure.
    • Acceptance: Page loads; fetches root tree; renders sidebar without errors.
  • I want to browse and expand folders so that I can navigate to files intuitively.
    • Acceptance: Click dir → fetch sub-tree → expand/collapse; files clickable to load content.
  • I want to read a note so that Markdown is rendered nicely (headings, lists, code blocks).
    • Acceptance: Click file → fetch raw MD → render HTML; images load from backend; internal links navigate.
  • I want to search by tags so that I can filter relevant notes.
    • Acceptance: Type tag → fetch API → highlight matching files in sidebar; click to view.
  • I want responsive design so that I can use it on mobile/desktop.
    • Acceptance: Sidebar collapses to hamburger menu on small screens; content scales.

4.2 Example Workflow

  1. User opens http://localhost:8080/ (assuming backend serves viewer assets).
  2. Sidebar loads root tree; user expands "notes" → clicks "foo.md".
  3. Main pane renders formatted Markdown; internal link to "bar.md" navigates on click.
  4. User types "project" in tag search → sidebar filters to tagged files.
  5. User toggles dark mode; navigates back via browser button.

5. Technical Requirements

5.1 Tech Stack

  • Frontend: Vanilla JavaScript (ES6+); no frameworks for simplicity.
  • Libraries (bundled/minified):
    • marked.js: Markdown parsing/rendering (with GitHub Flavored Markdown support).
    • No external deps at runtime; all via <script> tags.
  • Styling: Vanilla CSS (Flexbox/Grid); responsive breakpoints (@media).
  • Build Tool: None (hand-written); optional: Use a bundler like Parcel for dev.
  • Backend Integration: Fetch API for all endpoints; CORS headers added to backend if cross-origin.

5.2 Architecture

  • Client-Side:
    • index.html: Entry point with <div id="sidebar"> and <div id="viewer">.
    • app.js: Handles fetches, DOM updates, event listeners (e.g., tree clicks).
    • State: Simple object for current path, expanded nodes, search filter.
  • Data Flow:
    • Load → Fetch tree → Render sidebar.
    • Click file → Fetch content → Parse MD → Update viewer.
    • Search → Fetch tags → Filter/render tree subset.
  • Asset Serving: Add to backend: http.FileServer for /static/ (place viewer files there).

5.3 Constraints

  • Browser Support: Modern (Chrome 80+, Firefox 75+); no IE.
  • Vault Size: Assumes backend handles large trees (client paginates deep levels if needed).
  • Security: Sanitize Markdown output (marked.js defaults); no user input beyond search.

6. Non-Functional Requirements

6.1 Performance

  • Initial Load: < 2s (tree fetch + render); lazy-load sub-trees.
  • Rendering: < 100ms for 10KB Markdown file.
  • Memory: < 50MB for large trees (virtualize deep nodes if >1000 files).

6.2 Accessibility

  • ARIA labels for tree nodes; keyboard-navigable sidebar.
  • High contrast mode; screen reader friendly (semantic HTML).

6.3 Usability

  • Intuitive layout: Sidebar (20% width) + Viewer (80%); mobile: Stacked.
  • Offline Fallback: Cache tree/content via Service Worker (future Phase 2).

6.4 Reliability

  • Error Resilience: Graceful degradation (e.g., show raw text if MD parse fails).
  • Testing: Manual browser tests; unit tests for MD rendering (80% coverage).

7. Assumptions and Dependencies

  • Assumptions:
    • Backend runs on configurable URL; exposes APIs as spec'd.
    • Vault Markdown is standard (no plugins requiring custom rendering).
    • Viewer assets served from backend (add /static handler to main.go).
  • Dependencies:
    • Runtime: None (self-contained).
    • Dev: marked.js CDN or download for bundling.
  • Risks:
    • Large Trees: Slow rendering (mitigate: Virtual scrolling for sidebar).
    • MD Edge Cases: Complex embeds (mitigate: Extend marked.js config).

8. Future Enhancements

  • Phase 2: Full-text search integration (backend /search); wiki-link resolution.
  • Phase 3: PWA support (offline reading); custom themes.
  • Phase 4: React/Vue rewrite for advanced features (e.g., graph view).

Approval

  • Product Owner: [TBD]
  • Engineering Lead: [TBD]
  • Status: Draft – Ready for Review

This PRD aligns directly with the Obsidian Vault Server spec. For implementation, start with a basic index.html skeleton and app.js for API fetches. Sample code snippets available on request!

Next step

Below is a well-structured prompt you can copy-paste or adapt for your Project Manager (PM) and Scrum Master (SM). It's tailored to guide them in breaking down the Obsidian Vault Viewer PRD into actionable, workable user stories, with a strict focus on the front-end viewer (no server/backend work). It mirrors the structure of the previous server-focused prompt for consistency, emphasizing Scrum practices like INVEST, epics/themes, and sprint readiness.

Prompt for Project Manager and Scrum Master: Breaking Down Obsidian Vault Viewer PRD into Workable User Stories

Context:
We have the PRD for the Obsidian Vault Viewer (v1.0), a lightweight single-page web app that integrates with the Obsidian Vault Server backend to provide an interactive UI for browsing and reading vault content. Features include a tree sidebar, Markdown content viewer, tag search pane, and navigation controls. Our immediate focus is on implementing the viewer front-end only—assume the server APIs (e.g., /vault-tree, /) are available and stable. The goal is to create a prioritized backlog of user stories that can be pulled into sprints, ensuring the viewer is responsive, testable, and deployable as a single HTML/JS/CSS bundle.

Your Task:
As PM and SM, collaborate to decompose the PRD (sections 3-5, plus existing user stories in section 4) into 15-25 user stories. Group them into 3-5 high-level Epics/Themes (e.g., "Core Layout and Navigation", "Content Rendering", "Search and Filtering", "Polish and Responsiveness").

Guidelines for Stories:

  • Format: Use the standard user story template: As a [user/persona], I want [feature] so that [benefit].
    • Personas: Draw from PRD section 2 (e.g., Alex the Developer, Jordan the Writer).
  • INVEST Criteria: Ensure stories are:
    • Independent (can stand alone).
    • Negotiable (details flexible).
    • Valuable (delivers user/business value).
    • Estimable (sized for 1-3 days of effort).
    • Small (fits in one sprint; split large ones).
    • Testable (clear acceptance criteria).
  • Acceptance Criteria: For each story, include 3-6 bullet points with testable outcomes (e.g., "Click X → Verify Y renders in <2s"). Reference PRD specs (e.g., fetch endpoints, Markdown rendering, responsive breakpoints).
  • Prioritization: Use MoSCoW (Must-have, Should-have, Could-have, Won't-have) or business value/effort matrix. Flag 5-8 stories as "Sprint 1 Ready" (e.g., basic layout and tree sidebar).
  • Technical Notes: Stories should align with vanilla JavaScript tech stack (section 5.1: ES6+, marked.js for Markdown); include DOM manipulation, fetch API calls, and CSS for responsiveness. Assume no build tools (hand-written files); unit tests for rendering/search (80% coverage, section 6.4). Handle errors gracefully per section 3.3.
  • Output Structure:
    1. Epics/Themes: List with brief description and contained stories.
    2. User Stories: Numbered list, each with:
      • Story title.
      • User story sentence.
      • Acceptance criteria (bullets).
      • Priority (Must/Should/etc.).
      • Estimated effort (e.g., 3 story points).
    3. Sprint 1 Backlog: Top 5-8 stories ready to groom.
    4. Risks/Dependencies: Bullet list of any cross-cutting concerns (e.g., browser compatibility, backend API mocks for testing).

Deliverable:
Share a Google Doc, Notion page, or Markdown file with the full backlog by [insert deadline, e.g., EOD Friday]. Schedule a 30-min refinement session next week to validate. This will feed directly into Jira/Trello for sprint planning.

Key PRD Sections to Decompose:

  • 3.1: Core Features (Vault Tree Sidebar, Content Viewer, Tag Search Pane, Navigation/Controls).
  • 3.2: Configuration (e.g., server URL).
  • 3.3: Error Handling (e.g., loading states, network retries).
  • 4.1: Existing user stories (enhance with AC).
  • 5.2: Architecture (e.g., state management, data flow).
  • 6.1-6.3: Performance/Accessibility/Usability (e.g., <2s load, ARIA labels).

Let's make this viewer intuitive and snappy for Alex and Jordan—focus on quick wins like the sidebar and basic rendering first!

This prompt is concise, actionable, and aligned with the previous one for easy handover. If you need sample stories, adjustments (e.g., for specific tools), or a combined server+viewer prompt, just let me know!

Raw
002 - epics.md

Obsidian Vault Server: Backend User Story Backlog

This document breaks down the PRD for the Obsidian Vault Server (v1.2) into a prioritized backlog. The focus is exclusively on the backend implementation, providing a clear path for development sprints.

1. Epics / Themes

Epic/Theme Description Contained Stories
1. Build-Time Preparation Features related to the pre-compilation process: creating the build script, compressing the vault, and generating the tag index. #1, #2, #3, #4
2. Core Server & Static Serving Foundational runtime features: starting the server via CLI, loading the vault into memory, and serving raw files over HTTP. #5, #6, #7, #8, #9
3. Dynamic API Endpoints API endpoints that provide structured data about the vault, such as the file tree and tag-based search results. #10, #11, #12, #13
4. MCP Integration for AI Agents All features required to expose the vault as a set of tools for AI agents, including the --mcp flag, tool schema, and execution endpoints. #14, #15, #16, #17, #18, #19

2. User Stories

Epic 1: Build-Time Preparation

1. Build Script Orchestration

  • User Story: As Alex the Developer, I want a build.go script that orchestrates all pre-build steps so that I can reliably and repeatably prepare the vault assets for embedding.
  • Acceptance Criteria:
    • A build.go file exists in the project root.
    • Running go run build.go executes the vault compression and tag indexing steps.
    • The script logs its progress (e.g., "Compressing vault...", "Generating tag index...").
    • The script exits with a non-zero status code if any step fails.
  • Priority: Must-have
  • Effort: 2 Story Points

2. Vault Directory Compression

  • User Story: As Alex the Developer, I want the build script to compress the ./vault directory into a embedded_vault.tar.gz file so that the entire vault can be efficiently embedded into the final binary.
  • Acceptance Criteria:
    • The build script looks for a ./vault directory.
    • It creates a single embedded_vault.tar.gz file in the project root.
    • The compression uses gzip level 9 for maximum size reduction.
    • The script correctly handles nested directories and various file types within the vault.
  • Priority: Must-have
  • Effort: 3 Story Points

3. Frontmatter Tag Indexing

  • User Story: As Alex the Developer, I want the build script to parse all .md files for frontmatter tags and generate a tag_index.json file so that tag-based searches at runtime are instantaneous.
  • Acceptance Criteria:
    • The script recursively scans the ./vault directory for .md files.
    • It correctly parses YAML frontmatter for a tags: [tag1, tag2] key.
    • It generates a tag_index.json file with the format { "tag_name": ["/path/to/file1.md", ...] }.
    • Files with no tags key or empty tags array are ignored.
    • Tag matching is case-insensitive during index creation (all tags are lowercased).
  • Priority: Must-have
  • Effort: 3 Story Points

4. Asset Embedding

  • User Story: As Alex the Developer, I want the main Go application to embed the embedded_vault.tar.gz and tag_index.json files at compile time so that the final binary is a single, self-contained executable.
  • Acceptance Criteria:
    • The main.go file uses the //go:embed directive to include the two generated files.
    • The application can access the byte content of both embedded files at runtime.
    • The build fails if the asset files are missing.
  • Priority: Must-have
  • Effort: 1 Story Point

Epic 2: Core Server & Static Serving

5. Basic CLI Server Startup

  • User Story: As Jordan the Writer, I want to start the server with a simple ./obsidian-server command so that my vault becomes accessible over HTTP on my local machine.
  • Acceptance Criteria:
    • The compiled binary can be executed from the command line.
    • On startup, the server binds to a default port (e.g., 8080).
    • A startup message is logged to the console, e.g., "Obsidian Vault Server started on http://localhost:8080".
    • The server shuts down gracefully upon receiving a SIGINT signal (Ctrl+C).
  • Priority: Must-have
  • Effort: 3 Story Points

6. CLI Port Configuration

  • User Story: As Alex the Developer, I want to specify a custom port using a --port flag so that I can avoid port conflicts with other running services.
  • Acceptance Criteria:
    • Running ./obsidian-server --port=9090 starts the server on port 9090.
    • The startup message reflects the custom port.
    • The application provides a helpful error if an invalid port number is given.
  • Priority: Should-have
  • Effort: 1 Story Point

7. In-Memory Vault Loading

  • User Story: As Alex the Developer, I want the server to load the embedded tar.gz archive into an in-memory map on startup so that file access is extremely fast and avoids slow disk I/O.
  • Acceptance Criteria:
    • On startup, the application reads the embedded embedded_vault.tar.gz.
    • It iterates through the tar archive and stores each file's path and content in an in-memory map (e.g., map[string][]byte).
    • The startup time remains under 500ms for a moderately sized test vault.
    • The application logs an error and exits if the archive is corrupt or cannot be read.
  • Priority: Must-have
  • Effort: 5 Story Points

8. Static File Serving Endpoint

  • User Story: As Jordan the Writer, I want to access any file in my vault via a GET /<path> request (e.g., /notes/project-alpha.md) so that I can easily view my raw notes or share links to attachments.
  • Acceptance Criteria:
    • A GET request to a valid file path returns a 200 OK status.
    • The response body contains the raw, unmodified content of the requested file from memory.
    • The Content-Type header is set appropriately (e.g., text/plain for .md, image/png for .png).
    • A request to a non-existent path returns a 404 Not Found error.
  • Priority: Must-have
  • Effort: 3 Story Points

9. Path Traversal Security

  • User Story: As a security-conscious user, I want the file server to sanitize all incoming paths so that malicious requests cannot access files outside the embedded vault.
  • Acceptance Criteria:
    • A request like GET /../../etc/hosts is cleaned and results in a 404 Not Found or 400 Bad Request.
    • URL-encoded traversal sequences (e.g., %2e%2e%2f) are also blocked.
    • Valid paths containing special characters (but not traversal) are handled correctly.
  • Priority: Must-have
  • Effort: 2 Story Points

Epic 3: Dynamic API Endpoints

10. Vault Tree API (Root & Sub-directories)

  • User Story: As Alex the Developer, I want to call the GET /vault-tree endpoint to list the contents of a directory so that my front-end application can render a file explorer.
  • Acceptance Criteria:
    • A GET /vault-tree request returns the immediate children of the vault's root.
    • A GET /vault-tree?path=notes/subfolder returns the children of that specific subdirectory.
    • The JSON response matches the structure defined in PRD section 3.1.3, including name, path, type, and is_expandable.
    • The is_expandable flag is true for directories that are not empty.
    • A request for a non-existent path returns a 404 Not Found error.
  • Priority: Must-have
  • Effort: 3 Story Points

11. Tag Search API (Basic)

  • User Story: As Sam the AI Engineer, I want to query GET /vault-tag/<name> to find all files associated with a specific tag so that I can programmatically retrieve relevant documents.
  • Acceptance Criteria:
    • The endpoint uses the pre-loaded tag_index.json for lookups.
    • A request to GET /vault-tag/project returns a 200 OK with a JSON object.
    • The response includes a files array containing the full paths to all matching notes.
    • Tag search is case-insensitive.
    • A request for a tag with no associated files returns a 404 Not Found.
  • Priority: Must-have
  • Effort: 3 Story Points

12. Tag Search API (Pagination)

  • User Story: As Alex the Developer, I want to use limit and offset query parameters on the /vault-tag endpoint so that my UI can handle tags with hundreds of files without overwhelming the user.
  • Acceptance Criteria:
    • GET /vault-tag/project?limit=10 returns a maximum of 10 file paths.
    • GET /vault-tag/project?limit=10&offset=10 returns the next 10 file paths.
    • The JSON response includes the pagination object (offset, limit, total) as specified in the PRD.
    • Default values are applied if parameters are omitted (limit: 50, offset: 0).
  • Priority: Should-have
  • Effort: 2 Story Points

13. API Error Handling Standardization

  • User Story: As Alex the Developer, I want API endpoints to return consistent and meaningful HTTP error codes so that I can build robust error handling in my client application.
  • Acceptance criteria:
    • Invalid request parameters (e.g., non-integer limit) result in a 400 Bad Request.
    • Requests for resources that do not exist (files, paths, tags) result in a 404 Not Found.
    • Unexpected server-side issues result in a 500 Internal Server Error with a generic message.
  • Priority: Should-have
  • Effort: 1 Story Point

Epic 4: MCP Integration for AI Agents

14. Enable MCP Mode via CLI Flag

  • User Story: As Sam the AI Engineer, I want to start the server with an --mcp flag so that the MCP tool endpoints are activated for my AI agent to use.
  • Acceptance Criteria:
    • Running ./obsidian-server --mcp enables the /mcp/* routes.
    • Without the flag, requests to /mcp/* return a 404 Not Found.
    • The server logs that "MCP Mode is enabled" on startup when the flag is present.
  • Priority: Must-have
  • Effort: 1 Story Point

15. MCP Tool Schema Endpoint

  • User Story: As Sam the AI Engineer, I want to call GET /mcp/tools to retrieve the JSON schema of available functions so that my agent framework (like LangChain or OpenAI) can register them as tools.
  • Acceptance Criteria:
    • The endpoint is only available when --mcp is active.
    • It returns a 200 OK with a JSON array of tool definitions.
    • The schema for get_vault_tree, get_vault_file, and get_vault_files_by_tags exactly matches the structure in PRD section 3.2.
  • Priority: Must-have
  • Effort: 2 Story Points

16. MCP Execution: get_vault_tree Tool

  • User Story: As Sam the AI Engineer, I want my agent to execute the get_vault_tree tool via POST /mcp/execute so it can programmatically explore the vault structure.
  • Acceptance Criteria:
    • A POST to /mcp/execute with {"tool_name": "get_vault_tree", "arguments": {"path": "notes"}} succeeds.
    • The response result contains the same JSON object that GET /vault-tree?path=notes would return.
    • The tool works correctly when the path argument is omitted (for root).
  • Priority: Must-have
  • Effort: 2 Story Points

17. MCP Execution: get_vault_file Tool

  • User Story: As Sam the AI Engineer, I want my agent to execute the get_vault_file tool so it can read the content of a specific note required for a task.
  • Acceptance Criteria:
    • A POST to /mcp/execute with {"tool_name": "get_vault_file", "arguments": {"path": "notes/foo.md"}} succeeds.
    • The response result is a string containing the raw content of the file.
    • The endpoint returns a proper tool execution error if the file path is not found.
  • Priority: Must-have
  • Effort: 2 Story Points

18. MCP Execution: get_vault_files_by_tags Tool

  • User Story: As Sam the AI Engineer, I want my agent to execute the get_vault_files_by_tags tool so it can find all notes related to a topic to synthesize an answer.
  • Acceptance Criteria:
    • A POST with {"tool_name": "get_vault_files_by_tags", "arguments": {"tag": "project"}} succeeds.
    • The response result contains the same JSON object as GET /vault-tag/project.
    • Pagination arguments (limit, offset) are correctly handled when passed in the arguments object.
  • Priority: Must-have
  • Effort: 3 Story Points

19. MCP Execution Error Handling

  • User Story: As Alex the Developer, I want the /mcp/execute endpoint to return clear, structured errors so that I can easily debug my agent's interactions with the server.
  • Acceptance Criteria:
    • A request with an unknown tool_name returns a 404 Not Found error.
    • A request with missing required arguments (e.g., no tag for get_vault_files_by_tags) returns a 400 Bad Request with a descriptive message.
    • An internal error during tool execution returns a 500 Internal Server Error.
  • Priority: Should-have
  • Effort: 2 Story Points

3. Sprint 1 Backlog (Recommended)

This selection focuses on creating a minimum viable product: a server that can be built, started, and can serve files. This provides a complete end-to-end vertical slice.

  1. #1: Build Script Orchestration (2 SP)
  2. #2: Vault Directory Compression (3 SP)
  3. #4: Asset Embedding (1 SP)
  4. #5: Basic CLI Server Startup (3 SP)
  5. #7: In-Memory Vault Loading (5 SP)
  6. #8: Static File Serving Endpoint (3 SP)
  7. #9: Path Traversal Security (2 SP)

Total Sprint 1 Effort: 19 Story Points

4. Risks & Dependencies

  • Dependency: All build-time stories (#1, #2, #3, #4) must be completed before any runtime stories can be fully tested, as the server depends on the embedded assets.
  • Risk: Performance of the in-memory loading (#7) could be slow for exceptionally large vaults (>500MB). The initial implementation should be benchmarked to validate the PRD's performance goals.
  • Cross-Cutting Concern: Graceful shutdown logic needs to be implemented early (#5) and considered in all subsequent features that might hold resources.
  • Technical Debt: The initial tag parsing regex (#3) is simple. A risk exists that it won't handle complex or malformed frontmatter, which can be addressed in a future story if required.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment