gistfile1.txt

# Joliet Extension Support for ISO 9660

## Problem

The ISO 9660 specification restricts filenames to 8.3 format (8 characters + 3 character extension) using only uppercase ASCII, digits, and underscores. This is extremely limiting for modern use cases where files commonly have long, mixed-case, or Unicode names. While Rock Ridge extensions solve this on Unix-like systems by embedding POSIX metadata in System Use entries, Rock Ridge is not universally supported ??? particularly on Windows, where Joliet is the de facto standard for long filename support on optical media.

Prior to this change, go-diskfs had no ability to create or read Joliet-extended ISOs. Users who needed long filenames had to rely on Rock Ridge alone, which meant ISOs created by go-diskfs would show truncated 8.3 names on any system that only understands Joliet (notably Windows).

## What is Joliet?

Joliet is a Microsoft extension to ISO 9660 that adds a **Supplementary Volume Descriptor (SVD)** alongside the standard Primary Volume Descriptor (PVD). The SVD contains its own independent directory tree and path table where filenames are encoded in **UCS-2** (a subset of UTF-16 for the Basic Multilingual Plane), allowing filenames up to 64 characters with full Unicode support.

Key characteristics:
- The SVD is identified by escape sequences in bytes 88-120: `%/@` (Level 1), `%/C` (Level 2), or `%/E` (Level 3)
- File **data extents are shared** between the PVD and SVD trees ??? only the directory metadata is duplicated
- Joliet directory entries never carry SUSP (System Use Sharing Protocol) extensions
- The SVD has its own path table (L-type and M-type) with UCS-2 encoded directory names

## Architecture of Changes

### 1. Directory Entry Layer (`directoryentry.go`)

Added a `joliet bool` field to `directoryEntry` to distinguish Joliet entries from PVD entries. This flag controls:

- **Filename encoding**: Joliet entries encode/decode filenames as UCS-2 via `ucs2StringToBytes()` / `bytesToUCS2String()` instead of ASCII
- **Name length calculation**: `countNamelenBytes()` computes byte length from UCS-2 encoding when `joliet` is true
- **SUSP suppression**: Joliet entries skip all System Use extension parsing and serialization (`!de.joliet` guards)
- **Parsing**: Added `dirEntryFromBytesWithJoliet()` that accepts a joliet flag, and `parseDirEntriesJoliet()` for reading an entire Joliet directory extent

### 2. Finalization / Write Path (`finalize.go`)

The `Finalize()` function gained a `Joliet bool` option in `FinalizeOptions`. When enabled:

1. **Sector allocation**: Reserves one additional sector for the SVD between the PVD and the boot volume descriptor
2. **Joliet directory tree construction**: Iterates the PVD directory list and creates parallel Joliet `Directory` objects via `toJolietDirectory()`. Each directory gets its own block location, but file entries share the same data extent locations as the PVD tree
3. **Joliet path table**: `createJolietPathTable()` builds a separate path table with UCS-2 encoded directory names and Joliet-specific block locations
4. **Writing**: Joliet directories and path tables are written to disk after the PVD structures, followed by the SVD itself (which references the Joliet root directory entry, path table locations, and UCS-2 Level 3 escape sequences)

Helper functions added:
- `jolietFilename()` ??? appends `;1` version suffix to file entries (ISO 9660 requirement)
- `toJolietDirectoryEntry()` ??? creates a Joliet directory entry from finalization metadata
- `toJolietDirectory()` ??? builds a complete Joliet directory (self + parent + children)
- `calculateJolietDirectorySize()` ??? computes byte size accounting for block boundary alignment

### 3. Read Path (`iso9660.go`)

The `Read()` function was refactored to detect and load Joliet SVDs:

- **SVD detection**: During volume descriptor parsing, supplementary VDs are checked via `isJolietSVD()` which inspects the escape sequence bytes for UCS-2 Level 1/2/3 markers
- **Joliet loading**: `loadJoliet()` extracts the root directory entry and path table from the SVD
- **SUSP detection**: Extracted into `detectSUSP()` for clarity
- **Directory reading**: `readDirectory()` now dispatches to `readDirectoryJoliet()` when Joliet is enabled *and* Rock Ridge is not active. Rock Ridge takes precedence because it provides richer metadata (permissions, symlinks, etc.)

The `FileSystem` struct gained three new fields:
- `jolietEnabled bool`
- `jolietPathTable *pathTable`
- `jolietRootDir *directoryEntry`

### 4. Path Table Layer (`pathtable.go`)

Added Joliet-specific path table serialization and parsing:

- `toJolietLBytes()` / `toJolietMBytes()` ??? serialize path table entries with UCS-2 encoded directory names in little-endian and big-endian formats respectively
- `jolietPathTableName()` ??? encodes directory names as UCS-2, preserving the raw `0x01` byte for root entries
- `parseJolietPathTable()` ??? reads Joliet path table bytes, decoding UCS-2 directory names back to Go strings

### 5. Volume Descriptor (`volume_descriptor.go`)

- Added `volumeFlags` and `escapeSequences` fields to `supplementaryVolumeDescriptor` (previously not parsed)
- Updated `parseSupplementaryVolumeDescriptor()` to extract these fields
- Updated `toBytes()` to write volume flags, escape sequences, and file structure version
- Added `supplementary` field to `volumeDescriptors` struct
- Added `isJolietSVD()` detection function

### 6. UCS-2 Encoding (`util.go`)

Two utility functions handle the encoding boundary:

- `ucs2StringToBytes(s string) []byte` ??? converts a Go UTF-8 string to big-endian UCS-2 bytes
- `bytesToUCS2String(b []byte) string` ??? converts big-endian UCS-2 bytes back to a Go string

## Data Flow

### Writing a Joliet ISO

```
Finalize(Joliet: true)
  ???
  ?????? Build PVD directory tree (existing logic, 8.3 filenames)
  ?????? Build Joliet directory tree (parallel, original filenames as UCS-2)
  ???    ?????? File data locations shared with PVD entries
  ?????? Build Joliet path table (UCS-2 directory names)
  ???
  ?????? Write PVD directories
  ?????? Write Joliet directories
  ?????? Write PVD path tables (L + M)
  ?????? Write Joliet path tables (L + M)
  ?????? Write file data (shared)
  ?????? Write PVD
  ?????? Write SVD (points to Joliet root + path tables)
  ?????? Write terminator
```

### Reading a Joliet ISO

```
Read()
  ???
  ?????? Parse volume descriptors
  ???    ?????? PVD ??? rootDirEntry, pathTable
  ???    ?????? SVD with Joliet escape sequences ??? jolietRootDir, jolietPathTable
  ???
  ?????? detectSUSP() ??? check for Rock Ridge
  ???
  ?????? readDirectory(path)
       ?????? If Joliet enabled AND Rock Ridge not active:
       ???    ?????? readDirectoryJoliet() ??? parseDirEntriesJoliet() ??? UCS-2 filenames
       ?????? Otherwise:
            ?????? readDirectoryPVD() ??? standard or Rock Ridge filenames
```

## Design Decisions

1. **Rock Ridge takes precedence over Joliet**: When both are present, Rock Ridge provides richer metadata (POSIX permissions, symlinks, deep directory support). Joliet is only used as the filename source when Rock Ridge is absent.

2. **Shared file data extents**: File content is written once. Both PVD and Joliet directory entries point to the same data blocks. This matches how all major ISO authoring tools (xorriso, mkisofs) work and avoids doubling the ISO size.

3. **No SUSP in Joliet entries**: Per the Joliet specification, supplementary volume descriptor directory entries do not carry System Use extensions. The code explicitly guards against parsing or writing SUSP data for Joliet entries.

4. **UCS-2 Level 3**: The implementation uses UCS-2 Level 3 escape sequences (`%/E`), which is the most permissive level and what tools like xorriso produce by default.

## Testing

Integration tests (`joliet_xorriso_test.go`) verify interoperability with xorriso:

- **TestJolietGoReadXorrisoOutput**: Creates an ISO with xorriso using Joliet, reads it back with go-diskfs, verifies filenames and content
- **TestJolietRoundTrip**: Creates an ISO with go-diskfs Joliet, reads it back with go-diskfs, verifies filenames and content
- **TestJolietSVDComparison**: Creates equivalent ISOs with both go-diskfs and xorriso, compares the SVD fields byte-by-byte to ensure structural compatibility