Plan for hedgedoc#4943 - Robots Meta Tag Support

gistfile1.txt

# Robots Meta Tag Support

**Issues:** [#4943](https://github.com/hedgedoc/hedgedoc/issues/4943), [#4951](https://github.com/hedgedoc/hedgedoc/issues/4951)

**Goal:** Allow admins to set a server-wide default `robots` meta tag (e.g., `noindex, nofollow`) and allow per-note override via frontmatter. Render the tag server-side so crawlers see it.

## Current State

- The `robots` frontmatter field is already parsed in `commons/src/note-frontmatter/` and defaults to `''`
- The parsed value is never persisted to the database, returned via API, or rendered as a meta tag
- The editor page (`frontend/src/app/(editor)/n/[noteId]/page.tsx`) has a TODO comment referencing issue #4766 about implementing `generateMetadata()` for SSR
- `frontend/public/robots.txt` allows all crawlers (unchanged by this work)

## Design

### 1. Data Model Changes

**Database schema** — add `robots` column to the `revision` table:

- File: `database/src/types/revision.ts`
- Add `robots` to `Revision` interface and `FieldNameRevision` enum
- New Knex migration: add non-nullable string column `robots` with default `''` to `revision` table (matches `title` and `description` column conventions)

**Frontmatter extraction** — include `robots` in extracted metadata:

- File: `backend/src/revisions/utils/extract-revision-metadata-from-content.ts`
- Add `robots: string` to `FrontmatterExtractionResult`
- Extract `frontmatter?.robots ?? ''` alongside existing title/description/tags

**Revision creation** — persist `robots` when inserting:

- File: `backend/src/revisions/revisions.service.ts`
- In `innerCreateRevision`, destructure `robots` from `extractRevisionMetadataFromContent()` and include `[FieldNameRevision.robots]: robots` in the insert

**Commons DTO** — add `robots` to the API schema:

- File: `commons/src/dtos/note/note-metadata.dto.ts`
- Add `robots: z.string().describe('The robots meta tag directive for this note')` to `NoteMetadataSchema`

**Backend service** — include `robots` in the metadata DTO:

- File: `backend/src/notes/note.service.ts`
- In `innerToNoteMetadataDto`, read `robots` from `latestRevision` and include it in the `NoteMetadataDto.create()` call

### 2. Server Default Configuration

**Config** — add `HD_NOTE_ROBOTS_DEFAULT` env var:

- File: `backend/src/config/note.config.ts`
- Add `robotsDefault: z.string().optional().default('').describe('HD_NOTE_ROBOTS_DEFAULT')` to the schema
- In the `registerAs` factory, read `process.env.HD_NOTE_ROBOTS_DEFAULT`

**Fallback logic** — per-note frontmatter takes precedence:

- File: `backend/src/notes/note.service.ts`
- In `innerToNoteMetadataDto`, resolve robots as: `latestRevision.robots || this.noteConfig.robotsDefault`
- This means: per-note frontmatter > server default > empty string (no meta tag)

**Documentation**:

- File: `.env.example`
- Add `HD_NOTE_ROBOTS_DEFAULT` with comment explaining the behavior

### 3. Frontend SSR Metadata

**Problem:** The editor page is `'use client'`, but `generateMetadata()` must be exported from a server component.

**Solution:** Split the page into a server component (exports `generateMetadata()` and renders the client component) and a client component (the existing editor content).

**Editor page** (`frontend/src/app/(editor)/n/[noteId]/page.tsx`):

- Remove `'use client'` directive from the page file
- Move the existing client component to an imported file (or inline it as a client component import)
- Export `generateMetadata()` that:
  1. Calls `getNote(noteId, internalApiUrl)` using the base URL extractor
  2. Returns `{ title, description, robots }` from the note metadata
  3. Returns empty metadata on error (note not found, etc.)

**Slide page** (`frontend/src/app/(editor)/s/[noteId]/page.tsx`):

- Same treatment as the editor page

### 4. Scope Boundaries

**In scope:**
- `robots` field in revision table, extraction, DTO, and API response
- `HD_NOTE_ROBOTS_DEFAULT` config option with fallback logic
- `generateMetadata()` for editor and slide pages (robots, title, description only)

**Out of scope:**
- OpenGraph metadata (separate concern, issue #4766)
- `robots.txt` changes
- `X-Robots-Tag` HTTP header
- `LicenseLinkHead` or `OpengraphHead` components

### 5. Testing

- **`extract-revision-metadata-from-content.spec.ts`**: Test that `robots` is extracted from frontmatter, defaults to `''` when absent
- **`note.service.spec.ts`**: Test fallback logic — per-note robots overrides server default; server default used when per-note is empty; empty string when neither is set
- **`NoteMetadataSchema` validation**: Test that `robots` field is accepted and validated as a string
- **No new frontend tests**: `generateMetadata` is a thin data-fetch-to-metadata mapping; Next.js handles the rendering