website-to-epub AI agent skill

gistfile1.txt

---
name: website-to-epub
description: Convert an mdBook/mkDocs documentation website to EPUB for e-reader reading. Downloads clean Markdown from GitHub source, assembles in chapter order, converts with pandoc.
tags: [epub, pandoc, mdbook, documentation, ebook, converter, website]
trigger: epub documentation website mdbook docs-to-epub convert-docs ebook
---

# Website Documentation → EPUB

Convert a full mdBook documentation site (or similar static-doc site) into a clean EPUB for offline reading on an e-reader (phone, Kindle, etc.).

## When to Use

Someone says "turn this docs site into an epub I can read on my phone" — use this workflow.

## Prerequisites

```bash
which pandoc calibre ebook-convert 2>/dev/null   # pandoc must be installed
```

If pandoc isn't installed:
```bash
sudo apt install pandoc    # Debian/Ubuntu
sudo pacman -S pandoc      # Arch/Manjaro
brew install pandoc        # macOS
```

## Workflow

### Step 1: Identify the Site

Check if it's an **mdBook** project (most Rust documentation sites). The GitHub repo typically has:
- `book/book.toml` — mdBook config
- `book/src/SUMMARY.md` — chapter structure
- `book/src/introduction.md`
- `book/src/assets/` — images

For mkDocs sites, look for `mkdocs.yml` instead of `book.toml`, but the approach is similar.

### Step 2: Get the Chapter Structure

```bash
curl -sL "https://raw.githubusercontent.com/<user>/<repo>/main/book/src/SUMMARY.md"
```

Parse this to extract section names (`# Section`) and chapter links (`[Title](path.md)`).

### Step 3: List & Download All Markdown Files

```bash
curl -sL "https://api.github.com/repos/<user>/<repo>/git/trees/main?recursive=1" | \
python3 -c "
import json, sys
data = json.load(sys.stdin)
for item in data['tree']:
    if item['path'].startswith('book/src/') and item['path'].endswith('.md'):
        print(item['path'])
" | while read path; do
  dest="${path#book/src/}"
  mkdir -p "src/$(dirname "$dest")"
  curl -sL "https://raw.githubusercontent.com/<user>/<repo>/main/$path" -o "src/$dest"
done
```

Also download images:
```bash
curl -sL "https://api.github.com/repos/<user>/<repo>/contents/book/src/assets" | \
python3 -c "import json, sys; [print(x['download_url']) for x in json.load(sys.stdin)]" | \
while read url; do
  curl -sL "$url" -o "src/assets/$(basename $url)"
done
```

### Step 4: Write the Build Script

Create a Python script (`build_epub.py`) that:

**A. Parses SUMMARY.md** to build ordered chapter list:
```python
chapters = []
current_section = None
for line in summary_lines:
    section_match = re.match(r'^#+\s+(.+)$', line)
    if section_match and 'Summary' not in line.strip():
        current_section = section_match.group(1).strip()
    link_match = re.match(r'^[-\s]*\[([^\]]+)\]\(([^)]+\.md)\)', line)
    if link_match:
        chapters.append({'title': ..., 'path': ..., 'section': current_section})
```

**B. Fixes image paths** — mdBook uses relative paths like `../assets/img.png`. Convert to absolute GitHub Raw URLs so pandoc embeds them:
```python
import re
content = re.sub(
    r'!\[([^\]]*)\]\(((?:\.\./)*assets/[^)]+)\)',
    lambda m: f'![{m.group(1)}](https://raw.githubusercontent.com/<user>/<repo>/main/book/src/{m.group(2)})',
    content
)
```

**C. Assembles one combined markdown** in chapter order:
- Section headers → `\n\n---\n\n# Section Name\n\n`
- Each chapter → `\n\n# Chapter Title\n\n` + content (with the file's own H1 stripped to avoid duplication)

**D. Converts to EPUB with pandoc:**
```bash
pandoc combined.md \
  --from markdown+smart \
  --to epub3 \
  --metadata "title=Doc Title" \
  --metadata "author=author" \
  --metadata "language=en-US" \
  --toc --toc-depth=3 \
  --split-level=2 \
  --metadata "cover-image=path/to/cover.png" \
  -o output.epub
```

**Critical flags:**
- **`--split-level=2`** (NOT `--epub-chapter-level` — that's deprecated)
- **`--toc`** for e-reader table of contents
- **cover-image** must be a local absolute path to a real file

### Step 5: Verify

```python
import zipfile
epub = zipfile.ZipFile('output.epub')
print(f'Files in EPUB: {len(epub.namelist())}')
for n in epub.namelist():
    if n.endswith(('.opf', '.ncx', 'nav.xhtml')):
        print(f'  {n}')
print(f'Size: {epub.getinfo("EPUB/content.opf").file_size} bytes')
epub.close()
```

## Full Reference Script

The complete `build_epub.py` script used in production (adjust URLs for each project):

```python
#!/usr/bin/env python3
import re, os, subprocess
from pathlib import Path

SRC = Path("src")
OUT = Path(".")

# Parse SUMMARY.md
summary = (SRC / "SUMMARY.md").read_text()
chapters = []
current_section = None

for line in summary.splitlines():
    section_match = re.match(r'^#+\s+(.+)$', line)
    if section_match and 'Summary' not in line.strip():
        current_section = section_match.group(1).strip()
    link_match = re.match(r'^[-\s]*\[([^\]]+)\]\(([^)]+\.md)\)', line)
    if link_match:
        chapters.append({
            'title': link_match.group(1).strip(),
            'path': link_match.group(2).strip(),
            'section': current_section
        })

# Build combined markdown
BASE_URL = "https://raw.githubusercontent.com/<user>/<repo>/main/book/src"
combined_parts = [f"# Title\n\n---\n\n"]

for ch in chapters:
    md_path = SRC / ch['path']
    if not md_path.exists():
        continue
    
    content = md_path.read_text()
    
    if ch['section'] and ch['section'] != combined_parts.get('_last_section'):
        combined_parts.append(f"\n\n---\n\n# {ch['section']}\n\n")
        combined_parts['_last_section'] = ch['section']
    
    combined_parts.append(f"\n\n# {ch['title']}\n\n")
    
    # Strip first heading from file content to avoid duplication
    lines = content.split('\n')
    if lines[0].startswith('# '):
        content = '\n'.join(lines[1:]).strip()
    
    # Fix relative image paths
    content = re.sub(
        r'!\[([^\]]*)\]\(((?:\.\./)*assets/[^)]+)\)',
        lambda m: f'![{m.group(1)}]({BASE_URL}/{m.group(2)})',
        content
    )
    
    combined_parts.append(content)

combined_md = ''.join(combined_parts)
combined_path = OUT / "combined.md"
combined_path.write_text(combined_md)

# Convert with pandoc
subprocess.run([
    "pandoc", str(combined_path),
    "--from", "markdown+smart",
    "--to", "epub3",
    "--metadata", "title=Documentation",
    "--metadata", "author=author",
    "--metadata", "language=en-US",
    "--toc", "--toc-depth=3",
    "--split-level=2",
    "--metadata", "cover-image=path/to/cover.png",
    "-o", "output.epub"
], check=True)
```

## Common Pitfalls

- **`--epub-chapter-level` is deprecated** → Use `--split-level=N` instead
- **Cover image not found** → Must be a local absolute path. Download it first.
- **Markdown files have duplicate H1** → Strip the first `# Heading` from each file's content
- **Image refs broken in EPUB** → Fix relative paths to absolute GitHub Raw URLs before pandoc conversion
- **Pandoc not installed** → Install it first (see Prerequisites)
- **Too many/few chapter splits** → Adjust `--split-level`: 1 = per-H1, 2 = per-H2, omit = one giant file

## Verification

```bash
ls -lh output.epub
file output.epub   # Should say "EPUB document"
```