Claude.ai data export → readable Markdown: split conversations.json (1 GB, one line) into per-chat files, expand projects and memories

README.md

Claude.ai data export → readable Markdown

Four small scripts that turn an Anthropic Claude.ai data export into a folder tree you can actually read, grep and keep in git.

The export arrives as a handful of opaque JSON files: every non-ASCII character escaped as \uXXXX, and conversations.json delivered as a single line that can run to a gigabyte. These scripts unpack it into one Markdown file per chat, one folder per project, and drop each project's memory next to its system prompt.

No dependencies — Python 3.8+ and the standard library. Every script supports --dry-run, and none of them ever modify the source export.

The numbers that motivated this

Measured on one real 1 GB export — roughly 1,300 chats over six months:

Content in conversations.json	Size	Share
tool_result — search and web output	253 MB	50.5%
thinking — assistant reasoning	117 MB	23.3%
tool_use — tool calls	90 MB	18.0%
text — the actual conversation	25 MB	5.1%
attachments (extracted text)	13 MB	2.6%

92% of that gigabyte is machine chatter. Drop it and half a year of conversations becomes ~41 MB of readable Markdown.

Pipeline

Unzip the export, then from inside it:

# 1. optional: make the raw JSON human-readable (\uXXXX -> real characters)
python3 decode_projects.py projects

# 2. the export ships the folder as `projects`; the scripts build into `projects`,
#    so rename the raw one out of the way first
mv projects projects-raw

# 3. projects -> one folder each, with system prompt and attached docs
python3 build_projects.py

# 4. conversations.json -> one Markdown file (or folder) per chat
python3 split_chats.py

# 5. memories.json -> global memory + one file inside each project folder
python3 split_memories.py

Steps 3–5 are independent except that split_memories.py needs build_projects.py to have run.

Result

projects/Beekeeping notes/
├── _meta.md                 # uuid, links back to source, dates, doc list
├── _project-memory.md       # this project's memory from memories.json
├── _project-prompt.md       # prompt_template, verbatim and paste-ready
├── hive-inspection-log.md   # …one file per attached document
└── varroa-treatment.md

chats/
├── _index.md                                  # every chat, sorted by date
├── 2026-02-24 Bike tire pressure.md           # no attachments -> flat file
└── 2026-02-24 Greenhouse layout/              # has attachments -> folder
    ├── chat.md
    └── attachments/soil-test.md

memory/
├── conversations-memory.md
└── _index.md

Every generated file is prefixed with _; everything without the prefix is your own content. That one rule makes the tree safe to re-upload: if you ever rebuild a project in Claude, drop in every file except the underscored ones. _project-prompt.md goes into the project's instructions field as text rather than as an attachment, and _project-memory.md is memory rather than knowledge — putting it into project knowledge would have the model retrieve its own summary instead of your sources.

Each chat.md opens with a metadata table and the conversation summary, then the dialogue. Tool calls collapse to one line — `🔧 project_knowledge_search` — “varroa treatment schedule” — and thinking / tool_result blocks are dropped. Every file links back to https://claude.ai/chat/<uuid> so you can jump to the original.

The scripts

Script	What it does
decode_projects.py	Rewrites \uXXXX escapes as real UTF-8 in any directory of JSON files. Atomic writes, verifies the data round-trips, backs the directory up first. Optional — json.loads reads escapes fine; this is for your eyes.
build_projects.py	Expands exported projects into folders: system prompt, attached docs, metadata. Skips projects with an empty name and lists them.
split_chats.py	Splits conversations.json into per-chat Markdown. Hybrid layout, name-collision handling, skips chats with no readable content and indexes them separately.
split_memories.py	Expands memories.json. project_memories is keyed by project uuid, so the match to project folders is exact rather than by name.

Notes and limitations

• Memory. split_chats.py reads the whole export into RAM — for a 1 GB file that is a ~1 GB string (the export is pure ASCII) plus decoded objects. Budget a few GB. The naive alternative, a hand-rolled character scanner, is quadratic and takes many minutes; walking the array with JSONDecoder.raw_decode() at successive offsets uses the C parser and finishes a gigabyte in about two seconds.
• --force on split_chats.py deletes the output directory before rebuilding. The other scripts overwrite in place.
• Chats cannot be grouped by project. The export gives conversations no project_uuid — only uuid, name, summary, created_at, updated_at, account, chat_messages. Project memories do carry the uuid, which is why step 5 works and a chats-by-project split does not.
• Not every attachment survives the export. Messages carry both attachments (with extracted_content) and files (a name and uuid, no bytes). In the sample export, hundreds of .md, .pdf and .png files appeared by name only — their content is simply absent upstream. Those are rendered as a 🖼 File … line so you can see what is missing.
• CRLF is preserved. Some attachments use \r\n; the scripts copy the bytes through untouched. Worth knowing if you diff the output — reading with Python's universal newlines will make identical files look different.
• Schema drift. Field names match Anthropic's export format as of August 2026.

License

Public domain / CC0 — do whatever you like with it.

build_projects.py

#!/usr/bin/env python3
"""Expand the exported project JSON files into a readable folder tree.

For every named project it writes:

    projects/<name>/
        _project-prompt.md   — the prompt_template verbatim (when non-empty)
        _meta.md            — uuid, links back to the source, dates, doc list
        <doc>.md            — one file per entry in docs[]

Projects with an empty name are skipped and listed at the end — in a real
export those tend to be abandoned drafts with no content at all.

The source directory is only ever read, never modified.

Usage:
    python3 build_projects.py --dry-run     # show the plan
    python3 build_projects.py               # build the tree
    python3 build_projects.py --force       # overwrite an existing projects/
"""

import argparse
import json
import os
import re
import sys
from pathlib import Path

RAW_DIR = "projects-raw"
OUT_DIR = "projects"
PROMPT_FILE = "_project-prompt.md"
META_FILE = "_meta.md"
MEMORY_FILE = "_project-memory.md"  # written by split_memories.py

# characters that are illegal or awkward in macOS/Windows file names
ILLEGAL = re.compile(r'[/\\:*?"<>|\x00-\x1f]')


def safe_name(raw: str, fallback: str) -> str:
    """A file/folder name safe for the filesystem."""
    name = ILLEGAL.sub("-", raw).strip().strip(".")
    # trim by bytes, not characters: the macOS limit is 255 bytes
    while len(name.encode("utf-8")) > 200:
        name = name[:-1]
    return name or fallback


def md_link(text: str, target: str) -> str:
    """Markdown link in angle brackets — file names contain spaces."""
    return f"[{text}](<{target}>)"


def human(n: int) -> str:
    return f"{n:,}"


def build_meta(project: dict, doc_files: list, prompt_written: bool,
               has_memory: bool = False) -> str:
    uuid = project["uuid"]
    lines = [
        f"# {project['name']}",
        "",
        "|  |  |",
        "| --- | --- |",
        f"| UUID | `{uuid}` |",
        f"| Project in Claude | https://claude.ai/project/{uuid} |",
        f"| Source JSON | {md_link(uuid + '.json', f'../../{RAW_DIR}/{uuid}.json')} |",
        f"| Created | {project.get('created_at', '')} |",
        f"| Updated | {project.get('updated_at', '')} |",
        f"| Private | {'yes' if project.get('is_private') else 'no'} |",
        f"| Documents | {len(project['docs'])} |",
        "",
    ]

    description = project.get("description", "").strip()
    if description:
        lines += ["## Description", "", description, ""]

    if has_memory:
        lines += ["## Project memory", "",
                  f"{md_link(MEMORY_FILE, MEMORY_FILE)} — extracted from `memories.json` "
                  f"by `split_memories.py`.", ""]

    lines += ["## System prompt", ""]
    if prompt_written:
        lines += [
            f"{md_link(PROMPT_FILE, PROMPT_FILE)} — "
            f"{human(len(project['prompt_template']))} characters",
            "",
        ]
    else:
        lines += ["Empty — `prompt_template` is not set on the source project.", ""]

    lines += ["## Attached files", ""]
    if doc_files:
        for fname, size in doc_files:
            lines.append(f"- {md_link(fname, fname)} — {human(size)} characters")
    else:
        lines.append("None.")
    lines.append("")

    return "\n".join(lines)


def write(path: Path, content: str, dry_run: bool) -> None:
    if dry_run:
        return
    path.write_text(content, encoding="utf-8")


def process(project: dict, out_root: Path, dry_run: bool) -> tuple:
    """Expand one project. Returns (folder, file_count, warnings)."""
    warnings = []
    folder = out_root / safe_name(project["name"], project["uuid"])
    if not dry_run:
        folder.mkdir(parents=True, exist_ok=True)

    prompt = project["prompt_template"]
    prompt_written = bool(prompt.strip())
    if prompt_written:
        text = prompt if prompt.endswith("\n") else prompt + "\n"
        write(folder / PROMPT_FILE, text, dry_run)

    doc_files = []
    used = {PROMPT_FILE.lower(), META_FILE.lower(), MEMORY_FILE.lower()}
    for i, doc in enumerate(project["docs"]):
        raw_name = doc["filename"]
        content = doc["content"]
        if not raw_name.strip() and not content.strip():
            continue  # phantom entry: no name, no content

        fname = safe_name(raw_name, f"document-{i + 1}")
        if not fname.lower().endswith(".md"):
            fname += ".md"
            warnings.append(f"added .md: {raw_name!r} -> {fname!r}")

        # name clash inside one project — disambiguate with a suffix
        if fname.lower() in used:
            stem, ext = os.path.splitext(fname)
            fname = f"{stem} ({i + 1}){ext}"
            warnings.append(f"name clash, renamed to {fname!r}")
        used.add(fname.lower())

        text = content if content.endswith("\n") else content + "\n"
        write(folder / fname, text, dry_run)
        doc_files.append((fname, len(content)))

    has_memory = (folder / MEMORY_FILE).exists()
    write(folder / META_FILE, build_meta(project, doc_files, prompt_written, has_memory), dry_run)

    count = len(doc_files) + 1 + (1 if prompt_written else 0)
    return folder, count, warnings


def main() -> int:
    parser = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("--dry-run", action="store_true", help="write nothing")
    parser.add_argument("--force", action="store_true",
                        help="write into an existing non-empty output directory")
    parser.add_argument("--raw", default=RAW_DIR,
                        help=f"directory with the exported project JSON (default: {RAW_DIR})")
    parser.add_argument("--out", default=OUT_DIR,
                        help=f"where to build the tree (default: {OUT_DIR})")
    args = parser.parse_args()

    raw_root = Path(args.raw).resolve()
    out_root = Path(args.out).resolve()

    if not raw_root.is_dir():
        print(f"No source directory: {raw_root}", file=sys.stderr)
        return 1

    if raw_root == out_root:
        print("--raw and --out point at the same directory; rename the exported "
              "'projects' folder to 'projects-raw' first.", file=sys.stderr)
        return 1

    if out_root.exists() and any(out_root.iterdir()) and not args.force and not args.dry_run:
        print(f"{out_root} already exists and is not empty. Use --force to overwrite.",
              file=sys.stderr)
        return 1

    files = sorted(raw_root.glob("*.json"))
    if not files:
        print(f"No .json files in {raw_root}", file=sys.stderr)
        return 1

    if not args.dry_run:
        out_root.mkdir(parents=True, exist_ok=True)

    named, skipped, all_warnings, total_files = [], [], [], 0

    for path in files:
        project = json.loads(path.read_text(encoding="utf-8"))
        if not project["name"].strip():
            skipped.append((path, project))
            continue
        folder, count, warnings = process(project, out_root, args.dry_run)
        total_files += count
        named.append((project["name"], folder, count))
        all_warnings += [f"{project['name']}: {w}" for w in warnings]

    prefix = "[dry-run] " if args.dry_run else ""
    print(f"{prefix}Built {len(named)} folders, {total_files} files\n")
    for name, folder, count in sorted(named, key=lambda x: -x[2]):
        print(f"  {count:>3} files  {folder.name}")

    if all_warnings:
        print("\nName fixes:")
        for w in all_warnings:
            print(f"  · {w}")

    print(f"\nSkipped unnamed projects: {len(skipped)}")
    for path, project in skipped:
        rel = f"{raw_root.name}/{path.name}"
        print(f"  {rel}  (created {project.get('created_at', '')[:10]}, "
              f"docs={len(project['docs'])})")

    return 0


if __name__ == "__main__":
    sys.exit(main())

decode_projects.py

#!/usr/bin/env python3
"""Rewrite JSON files so \\uXXXX escapes become real UTF-8 characters.

Anthropic's data export writes every non-ASCII character as a \\uXXXX escape,
which makes the raw files unreadable for anything but ASCII. This re-reads each
file and dumps it back with ensure_ascii=False. Structure and key order are
preserved; only the on-disk representation changes.

Purely cosmetic for the rest of the pipeline: json.loads() understands escapes
either way. Run it if you want to grep or eyeball the raw JSON.

Usage:
    python3 decode_projects.py                 # ./projects, in place, with backup
    python3 decode_projects.py --dry-run       # show what would change
    python3 decode_projects.py --indent 2      # also pretty-print
    python3 decode_projects.py --no-backup     # skip the backup copy
    python3 decode_projects.py path/to/dir     # some other directory
"""

import argparse
import json
import os
import shutil
import sys
import tempfile
from pathlib import Path


def convert_file(path: Path, indent, dry_run: bool) -> str:
    """Returns 'converted' | 'skipped' | 'unchanged'."""
    raw = path.read_text(encoding="utf-8")

    try:
        data = json.loads(raw)
    except json.JSONDecodeError as exc:
        print(f"  ! {path.name}: malformed JSON ({exc}) — skipping", file=sys.stderr)
        return "skipped"

    # indent=None keeps the export's own shape: one line, ", " and ": " separators
    new_text = json.dumps(data, ensure_ascii=False, indent=indent)
    new_text += "\n" if raw.endswith("\n") else ""

    if new_text == raw:
        return "unchanged"

    # the re-serialized file must carry exactly the same data
    if json.loads(new_text) != data:
        print(f"  ! {path.name}: data changed on re-serialization — skipping", file=sys.stderr)
        return "skipped"

    if dry_run:
        return "converted"

    # atomic write: temp file next to the target, then rename over it
    fd, tmp = tempfile.mkstemp(dir=str(path.parent), prefix=path.name + ".", suffix=".tmp")
    try:
        with os.fdopen(fd, "w", encoding="utf-8") as fh:
            fh.write(new_text)
        shutil.copymode(path, tmp)
        os.replace(tmp, path)
    except BaseException:
        if os.path.exists(tmp):
            os.unlink(tmp)
        raise

    return "converted"


def main() -> int:
    parser = argparse.ArgumentParser(description=__doc__,
                                     formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("directory", nargs="?", default="projects",
                        help="directory holding the JSON files (default: projects)")
    parser.add_argument("--dry-run", action="store_true",
                        help="write nothing, just report")
    parser.add_argument("--indent", type=int, default=None,
                        help="pretty-print with this indent (default: keep one-line shape)")
    parser.add_argument("--no-backup", action="store_true",
                        help="do not copy the directory before writing")
    args = parser.parse_args()

    src = Path(args.directory).resolve()
    if not src.is_dir():
        print(f"No such directory: {src}", file=sys.stderr)
        return 1

    files = sorted(p for p in src.glob("*.json") if p.is_file())
    if not files:
        print(f"No .json files in {src}")
        return 0

    print(f"Directory: {src}")
    print(f"Files: {len(files)}")

    if not args.dry_run and not args.no_backup:
        backup = src.with_name(src.name + ".bak")
        if backup.exists():
            print(f"Backup already exists, leaving it alone: {backup}")
        else:
            shutil.copytree(src, backup)
            print(f"Backup: {backup}")

    stats = {"converted": 0, "skipped": 0, "unchanged": 0}
    for path in files:
        result = convert_file(path, args.indent, args.dry_run)
        stats[result] += 1
        if result == "converted":
            print(f"  {'[dry-run] ' if args.dry_run else ''}✓ {path.name}")
        elif result == "unchanged":
            print(f"  = {path.name} (already decoded)")

    print(f"\nDone: {stats['converted']} converted, "
          f"{stats['unchanged']} unchanged, {stats['skipped']} skipped")
    return 1 if stats["skipped"] else 0


if __name__ == "__main__":
    sys.exit(main())

split_chats.py

#!/usr/bin/env python3
"""Split conversations.json into one readable Markdown file per chat.

The export ships as a single JSON array on one line — a gigabyte with no
newlines. Instead of a hand-rolled character scanner (quadratic and slow), this
walks the array with the stdlib C decoder via JSONDecoder.raw_decode() at
successive offsets: about two seconds for 1 GB.

Layout is hybrid — a chat with no attachments is a flat file, a chat with
attachments becomes a folder:

    chats/2026-02-24 Bike tire pressure.md      — flat
    chats/2026-02-24 Greenhouse layout/         — folder
        chat.md
        attachments/soil-test.md
    chats/_index.md                             — table of contents

Only the conversation itself is kept: human and assistant text. Tool calls
collapse to a one-line marker, while thinking blocks and tool_result payloads
are dropped — in a real export those account for ~92% of the bytes.

Usage:
    python3 split_chats.py --dry-run     # show the plan
    python3 split_chats.py               # split
    python3 split_chats.py --force       # DELETE and rebuild an existing chats/
"""

import argparse
import json
import os
import re
import shutil
import sys
from pathlib import Path

SRC = "conversations.json"
OUT_DIR = "chats"
CHAT_FILE = "chat.md"
ATT_DIR = "attachments"

ILLEGAL = re.compile(r'[/\\*?"<>|\x00-\x1f]')
SPACES = re.compile(r"\s+")


def safe_name(raw: str, fallback: str, limit: int = 100) -> str:
    """A file/folder name that is both filesystem-safe and still readable."""
    name = (raw or "").replace(":", " -")
    name = ILLEGAL.sub("-", name)
    name = SPACES.sub(" ", name).strip().strip(".")
    if len(name) > limit:
        name = name[:limit].rsplit(" ", 1)[0].rstrip(" -,")
    while len(name.encode("utf-8")) > 180:
        name = name[:-1]
    return name or fallback


def md_link(text: str, target: str) -> str:
    return f"[{text}](<{target}>)"


def human_size(n: int) -> str:
    return f"{n / 1024:.1f} KB" if n < 1024 * 1024 else f"{n / 1048576:.1f} MB"


def short_ts(ts: str) -> str:
    """2026-02-24T15:30:05.650472Z -> 2026-02-24 15:30"""
    if not ts or "T" not in ts:
        return ts or ""
    date, time = ts.split("T", 1)
    return f"{date} {time[:5]}"


def tool_marker(block: dict) -> str:
    """One-line stand-in for a tool call."""
    name = block.get("name") or block.get("tool_identifier") or "tool"
    payload = block.get("input")
    hint = ""
    if isinstance(payload, dict):
        for key in ("query", "prompt", "url", "command", "path", "file_path"):
            if payload.get(key):
                hint = str(payload[key])
                break
        else:
            if payload:
                hint = json.dumps(payload, ensure_ascii=False)
    elif payload:
        hint = str(payload)
    hint = SPACES.sub(" ", hint).strip()
    if len(hint) > 160:
        hint = hint[:160] + "…"
    return f"`🔧 {name}`" + (f" — “{hint}”" if hint else "")


def message_text(msg: dict) -> str:
    """All readable text of a message: text blocks, else the text field."""
    parts = [(b.get("text") or "").strip() for b in msg.get("content") or []
             if b.get("type") == "text"]
    joined = "\n\n".join(p for p in parts if p)
    return joined or (msg.get("text") or "").strip()


def render_message(msg: dict, att_paths: dict) -> list:
    """One message -> Markdown lines. An empty message yields an empty list."""
    body = []
    for block in msg.get("content") or []:
        btype = block.get("type")
        if btype == "text":
            text = (block.get("text") or "").strip()
            if text:
                body.append(text)
        elif btype == "tool_use":
            body.append(tool_marker(block))
        # thinking / tool_result / token_budget / flag are dropped

    if not body:
        fallback = (msg.get("text") or "").strip()
        if fallback:
            body.append(fallback)

    attachments = msg.get("attachments") or []
    files = msg.get("files") or []
    if not body and not attachments and not files:
        return []  # genuinely empty in the export — emit nothing at all

    who = "👤 **You**" if msg.get("sender") == "human" else "🤖 **Claude**"
    lines = [f"### {who} · {short_ts(msg.get('created_at'))}", ""]
    if body:
        lines += ["\n\n".join(body), ""]

    for att in attachments:
        fname = att.get("file_name") or "unnamed"
        size = human_size(att.get("file_size") or len(att.get("extracted_content") or ""))
        target = att_paths.get(id(att))
        link = md_link(fname, target) if target else f"**{fname}**"
        lines += [f"📎 Attachment: {link} ({size})", ""]

    # files[] carries a name but no content — the bytes are not in the export
    att_names = {a.get("file_name") for a in attachments}
    for fl in files:
        fname = fl.get("file_name")
        if fname and fname not in att_names:
            lines += [f"🖼 File `{fname}` — content not included in the export", ""]

    return lines


def render_chat(conv: dict, att_paths: dict) -> str:
    uuid = conv["uuid"]
    name = (conv.get("name") or "").strip() or "Untitled"
    msgs = conv["chat_messages"]

    lines = [
        f"# {name}",
        "",
        "|  |  |",
        "| --- | --- |",
        f"| Date | {short_ts(conv.get('created_at'))} |",
        f"| Updated | {short_ts(conv.get('updated_at'))} |",
        f"| Messages | {len(msgs)} |",
        f"| UUID | `{uuid}` |",
        f"| Chat in Claude | https://claude.ai/chat/{uuid} |",
        "",
    ]

    summary = (conv.get("summary") or "").strip()
    if summary:
        # a blank line would end the callout — every line needs its own ">"
        lines.append("> [!NOTE] Summary")
        lines += ["> " + ln if ln.strip() else ">" for ln in summary.splitlines()]
        lines.append("")

    lines += ["---", ""]
    for msg in sorted(msgs, key=lambda m: m.get("created_at") or ""):
        lines += render_message(msg, att_paths)

    return "\n".join(lines).rstrip() + "\n"


def iter_conversations(path):
    """Yield conversations from the one-line JSON array.

    Reads the file into memory: a 1 GB export is a ~1 GB str (the export is
    pure ASCII thanks to the \\uXXXX escaping) plus one decoded object at a
    time. Budget a few GB of RAM for a gigabyte-sized export.
    """
    text = Path(path).read_text(encoding="utf-8")
    dec = json.JSONDecoder()
    i = text.index("[") + 1
    while True:
        while i < len(text) and text[i] in ", \t\r\n":
            i += 1
        if i >= len(text) or text[i] == "]":
            return
        conv, i = dec.raw_decode(text, i)
        yield conv


def chat_title(conv: dict) -> str:
    """Chat name; for untitled chats, the start of the first human message."""
    name = (conv.get("name") or "").strip()
    if name:
        return name
    for msg in sorted(conv["chat_messages"], key=lambda m: m.get("created_at") or ""):
        if msg.get("sender") != "human":
            continue
        text = SPACES.sub(" ", message_text(msg))
        if text:
            return text[:60].rstrip() + ("…" if len(text) > 60 else "")
    return ""


def skip_reason(conv: dict):
    """Why this chat is not worth a file, or None if it is.

    A chat is kept when it has at least one line of readable text or one
    attachment with extracted content. Anything else would land on disk as a
    file made entirely of empty placeholders.
    """
    msgs = conv["chat_messages"]
    if not msgs:
        return "no messages"

    has_text = any(message_text(m) for m in msgs)
    has_att = any((a.get("extracted_content") or "").strip()
                  for m in msgs for a in m.get("attachments") or [])
    if has_text or has_att:
        return None

    if any(m.get("files") for m in msgs):
        return "file names only, content not included in the export"
    return "empty messages with no text and no attachments"


def main() -> int:
    parser = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("--dry-run", action="store_true", help="write nothing")
    parser.add_argument("--force", action="store_true",
                        help="delete an existing output directory and rebuild it")
    parser.add_argument("--keep-empty", action="store_true",
                        help="also emit files for chats with no readable text")
    parser.add_argument("--src", default=SRC, help=f"source file (default: {SRC})")
    parser.add_argument("--out", default=OUT_DIR, help=f"output directory (default: {OUT_DIR})")
    args = parser.parse_args()

    src = Path(args.src).resolve()
    out = Path(args.out).resolve()
    if not src.is_file():
        print(f"No such file: {src}", file=sys.stderr)
        return 1
    if out.exists() and any(out.iterdir()) and not args.force and not args.dry_run:
        print(f"{out} already exists and is not empty. Use --force to rebuild.", file=sys.stderr)
        return 1
    if out.exists() and args.force and not args.dry_run:
        shutil.rmtree(out)
    if not args.dry_run:
        out.mkdir(parents=True, exist_ok=True)

    print(f"Reading {src.name} ({src.stat().st_size / 1e9:.2f} GB)...")

    used_names = {}
    index_rows = []
    empty_chats = []
    stats = {"folders": 0, "flat": 0, "attachments": 0, "messages": 0, "bytes": 0}

    for conv in iter_conversations(src):
        msgs = conv["chat_messages"]
        title = chat_title(conv)

        reason = None if args.keep_empty else skip_reason(conv)
        if reason:
            empty_chats.append((conv["uuid"], title, (conv.get("created_at") or "")[:10],
                                len(msgs), reason))
            continue

        date = (conv.get("created_at") or "")[:10]
        base = f"{date} {safe_name(title, conv['uuid'][:8])}"

        # same name on the same day happens; disambiguate deterministically
        key = base.lower()
        used_names[key] = used_names.get(key, 0) + 1
        if used_names[key] > 1:
            base = f"{base} ({used_names[key]})"

        atts = [a for m in msgs for a in m.get("attachments") or []
                if (a.get("extracted_content") or "").strip()]

        att_paths = {}
        if atts:
            folder = out / base
            chat_path = folder / CHAT_FILE
            att_dir = folder / ATT_DIR
            if not args.dry_run:
                att_dir.mkdir(parents=True, exist_ok=True)
            seen = {}
            for idx, att in enumerate(atts):
                fname = safe_name(att.get("file_name") or "", f"attachment-{idx + 1}", limit=120)
                if not os.path.splitext(fname)[1]:
                    fname += ".md"
                low = fname.lower()
                seen[low] = seen.get(low, 0) + 1
                if seen[low] > 1:
                    stem, ext = os.path.splitext(fname)
                    fname = f"{stem} ({seen[low]}){ext}"
                att_paths[id(att)] = f"{ATT_DIR}/{fname}"
                content = att["extracted_content"]
                if not args.dry_run:
                    (att_dir / fname).write_text(
                        content if content.endswith("\n") else content + "\n", encoding="utf-8")
                stats["attachments"] += 1
            stats["folders"] += 1
            rel = f"{base}/{CHAT_FILE}"
        else:
            chat_path = out / f"{base}.md"
            stats["flat"] += 1
            rel = f"{base}.md"

        body = render_chat(conv, att_paths)
        if not args.dry_run:
            chat_path.parent.mkdir(parents=True, exist_ok=True)
            chat_path.write_text(body, encoding="utf-8")

        stats["messages"] += len(msgs)
        stats["bytes"] += len(body.encode("utf-8"))
        index_rows.append((date, title or "Untitled", rel, len(msgs), len(atts)))

    index_rows.sort()
    index = [
        "# All chats",
        "",
        f"{len(index_rows)} chats, {stats['messages']} messages. Source: `{src.name}`.",
        "",
        "| Date | Chat | Messages | Attachments |",
        "| --- | --- | --- | --- |",
    ]
    for date, title, rel, nmsg, natt in index_rows:
        safe_title = title.replace("|", "\\|").replace("[", "(").replace("]", ")")
        index.append(f"| {date} | {md_link(safe_title, rel)} | {nmsg} | {natt or ''} |")

    if empty_chats:
        index += ["", f"## Skipped empty chats: {len(empty_chats)}", "",
                  "No files were written for these — they contain no readable text. "
                  "The links open them in the web app so you can check and delete them.", ""]
        by_reason = {}
        for uuid, title, date, nmsg, reason in empty_chats:
            by_reason.setdefault(reason, []).append((date, title, nmsg, uuid))
        for reason, rows in sorted(by_reason.items(), key=lambda kv: -len(kv[1])):
            index += [f"### {reason} — {len(rows)}", ""]
            for date, title, nmsg, uuid in sorted(rows):
                index.append(f"- {date} · {title or 'untitled'} · {nmsg} msg · "
                             f"https://claude.ai/chat/{uuid}")
            index.append("")

    if not args.dry_run:
        (out / "_index.md").write_text("\n".join(index) + "\n", encoding="utf-8")

    prefix = "[dry-run] " if args.dry_run else ""
    print(f"\n{prefix}Done:")
    print(f"  folders with attachments : {stats['folders']}")
    print(f"  flat .md files           : {stats['flat']}")
    print(f"  attachment files         : {stats['attachments']}")
    print(f"  messages                 : {stats['messages']}")
    print(f"  text volume              : {stats['bytes'] / 1e6:.1f} MB")
    print(f"  skipped empty chats      : {len(empty_chats)}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

split_memories.py

#!/usr/bin/env python3
"""Expand memories.json into files, dropping each project's memory next to it.

memories.json is a list holding a single object:
    conversations_memory : str   — the global cross-chat memory
    project_memories     : dict  — project uuid -> memory text
    account_uuid         : str

Because the dict is keyed by project uuid, matching memories to projects is
exact — no guessing by name.

Output:
    memory/conversations-memory.md          — the global memory
    memory/_index.md                        — table of contents
    projects/<name>/_project-memory.md       — one project's memory

The uuid -> name mapping comes from the same raw export that build_projects.py
consumed, so run that first.

Usage:
    python3 split_memories.py --dry-run
    python3 split_memories.py
"""

import argparse
import json
import re
import sys
from pathlib import Path

SRC = "memories.json"
MEM_DIR = "memory"
PROJ_DIR = "projects"
RAW_DIR = "projects-raw"
MEMORY_FILE = "_project-memory.md"

ILLEGAL = re.compile(r'[/\\:*?"<>|\x00-\x1f]')


def safe_name(raw: str, fallback: str) -> str:
    name = ILLEGAL.sub("-", raw or "").strip().strip(".")
    while len(name.encode("utf-8")) > 200:
        name = name[:-1]
    return name or fallback


def md_link(text: str, target: str) -> str:
    return f"[{text}](<{target}>)"


def human(n: int) -> str:
    return f"{n:,}"


def project_names(raw_dir: Path) -> dict:
    """uuid -> project name, for named projects only."""
    names = {}
    for path in raw_dir.glob("*.json"):
        proj = json.loads(path.read_text(encoding="utf-8"))
        if (proj.get("name") or "").strip():
            names[proj["uuid"]] = proj["name"].strip()
    return names


def wrap(title: str, uuid: str, body: str, extra: list = None) -> str:
    lines = [f"# {title}", ""]
    lines += extra or []
    if uuid:
        lines += [f"Project UUID: `{uuid}` · https://claude.ai/project/{uuid}", ""]
    lines += ["---", "", body.strip(), ""]
    return "\n".join(lines)


def main() -> int:
    parser = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument("--dry-run", action="store_true", help="write nothing")
    parser.add_argument("--src", default=SRC, help=f"source file (default: {SRC})")
    parser.add_argument("--memory-dir", default=MEM_DIR,
                        help=f"where to put the global memory (default: {MEM_DIR})")
    parser.add_argument("--projects", default=PROJ_DIR,
                        help=f"tree built by build_projects.py (default: {PROJ_DIR})")
    parser.add_argument("--raw", default=RAW_DIR,
                        help=f"raw exported project JSON (default: {RAW_DIR})")
    args = parser.parse_args()

    src = Path(args.src).resolve()
    mem_dir = Path(args.memory_dir).resolve()
    proj_dir = Path(args.projects).resolve()
    raw_dir = Path(args.raw).resolve()

    for path, what in ((src, "file"), (raw_dir, "directory"), (proj_dir, "directory")):
        if not path.exists():
            print(f"Missing {what}: {path}", file=sys.stderr)
            return 1

    data = json.loads(src.read_text(encoding="utf-8"))
    if isinstance(data, list):
        if len(data) != 1:
            print(f"Expected a list with one object, got {len(data)}", file=sys.stderr)
            return 1
        data = data[0]

    conversations = data.get("conversations_memory") or ""
    memories = data.get("project_memories") or {}
    account = data.get("account_uuid", "")
    names = project_names(raw_dir)

    if not args.dry_run:
        mem_dir.mkdir(parents=True, exist_ok=True)

    if conversations.strip():
        body = wrap("Conversations memory", "", conversations,
                    [f"Account: `{account}`", "",
                     f"Source: {md_link(src.name, '../' + src.name)} · "
                     f"`conversations_memory` · {human(len(conversations))} characters", ""])
        if not args.dry_run:
            (mem_dir / "conversations-memory.md").write_text(body, encoding="utf-8")

    written, orphans, missing_folder = [], [], []

    for uuid, textblob in sorted(memories.items(), key=lambda kv: -len(kv[1])):
        name = names.get(uuid)
        if not name:
            orphans.append(uuid)
            continue
        folder = proj_dir / safe_name(name, uuid)
        if not folder.is_dir():
            missing_folder.append((uuid, name))
            continue
        body = wrap(f"Project memory: {name}", uuid, textblob,
                    [f"Source: {md_link(src.name, '../../' + src.name)} · "
                     f"`project_memories[\"{uuid}\"]` · {human(len(textblob))} characters", ""])
        if not args.dry_run:
            (folder / MEMORY_FILE).write_text(body, encoding="utf-8")
        written.append((name, folder, len(textblob)))

    index = [
        "# Memory",
        "",
        f"Extracted from {md_link(src.name, '../' + src.name)}. Account: `{account}`.",
        "",
        f"- {md_link('Conversations memory', 'conversations-memory.md')} — "
        f"{human(len(conversations))} characters",
        "",
        f"## Project memories — {len(written)}",
        "",
        "Each one lives inside its project folder, next to the system prompt.",
        "",
        "| Project | Memory | Characters |",
        "| --- | --- | --- |",
    ]
    for name, folder, size in sorted(written):
        rel = f"../{proj_dir.name}/{folder.name}/{MEMORY_FILE}"
        safe_title = name.replace("|", "\\|").replace("[", "(").replace("]", ")")
        index.append(f"| {safe_title} | {md_link(MEMORY_FILE, rel)} | {human(size)} |")
    index.append("")

    if orphans or missing_folder:
        index += ["## Unmatched", ""]
        for uuid in orphans:
            index.append(f"- `{uuid}` — memory present, no such project in the export")
        for uuid, name in missing_folder:
            index.append(f"- `{uuid}` — {name}: no folder in `{proj_dir.name}/`")
        index.append("")

    if not args.dry_run:
        (mem_dir / "_index.md").write_text("\n".join(index) + "\n", encoding="utf-8")

    prefix = "[dry-run] " if args.dry_run else ""
    print(f"{prefix}Conversations memory: {human(len(conversations))} characters")
    print(f"{prefix}Project memories written: {len(written)} of {len(memories)}")
    for name, _, size in sorted(written, key=lambda x: -x[2]):
        print(f"  {human(size):>7} chars  {name}")
    if orphans:
        print(f"\nMemory with no matching project: {orphans}")
    if missing_folder:
        print(f"\nNo project folder for: {missing_folder}")
    return 1 if (orphans or missing_folder) else 0


if __name__ == "__main__":
    sys.exit(main())