rt_nle_datamodel. md

gistfile1.txt

# Data Model

Defines every data structure in the project. A rebuild AI reads this to recreate the exact project file format and in-memory model.

## Project File Format

Serialized as **MessagePack** (via `rmp-serde`). The schema below is defined in Rust-struct notation but represents the canonical format.

---

## Top-Level: Project

```rust
struct Project {
    version: u32,                    // Schema version for migration, current = 4
    name: String,
    resolution: Resolution,
    frame_rate: FrameRate,
    sample_rate: u32,                // Audio sample rate, default 48000
    compositions: Vec<Composition>,
    master_comp_id: CompId,          // Which composition is the root/master
    media_pool: Vec<MediaAsset>,     // All imported media
    workspace: WorkspaceState,       // Persisted editor/workspace state
    id_gen: IdGenerator,             // Monotonic ID allocator persisted with the project
}
```

`version` compatibility contract:
- `4` is the current project schema version.
- `1` is the minimum supported schema version.
- Versions `1`, `2`, and `3` are migrated in-memory to version `4` on load.
- Versions lower than `1` or higher than `4` are rejected.

## ID Generation

All entity IDs are `u64` newtypes. IDs are allocated from a single monotonic counter and must stay globally unique across the project.

```rust
struct IdGenerator {
    next: u64,
}

struct CompId(u64);
struct LayerId(u64);
struct LayerGroupId(u64);
struct ClipId(u64);
struct MediaAssetId(u64);
struct EasingPresetId(u64);
```

The `IdGenerator` itself is serialized with the project so newly-created entities after reload continue from the correct next ID.

## Resolution

```rust
struct Resolution {
    width: u32,
    height: u32,
}
```

Common presets: 1920x1080, 3840x2160

## FrameRate

```rust
struct FrameRate {
    num: u32,   // e.g., 24000, 30000, 60000, 24, 25, 30
    den: u32,   // e.g., 1001 for NTSC, 1 for integer rates
}
```

## Composition

A composition is a timeline with layers. The master composition is the final output. Sub-compositions can be nested.

```rust
struct Composition {
    id: CompId,
    name: String,
    resolution: Resolution,          // Can differ from project (sub-comps)
    frame_rate: FrameRate,
    // Duration is auto-calculated: max(end frame of last clip on each layer).
    // No manual duration setting. Comp grows/shrinks with content.
    layers: Vec<Layer>,              // Ordered bottom-to-top (index 0 = bottom)
    markers: Vec<Marker>,
    // Motion blur settings (per-composition)
    shutter_angle: f32,              // Degrees, default 180.0
    shutter_phase: f32,              // Degrees, default -90.0 (centered)
    layer_groups: Vec<LayerGroup>,
    linked_av_pairs: Vec<LinkedAvPair>,  // Video+audio clips from same source
}
```

## Layer

A layer holds a sequence of clips. It has compositing properties that act as **defaults** for all clips on the layer. Per-clip overrides take precedence when set (see Clip struct).

```rust
struct Layer {
    id: LayerId,
    name: String,
    layer_type: LayerType,           // Video, Audio, Text, Null
    clips: Vec<Clip>,                // Ordered by timeline position, non-overlapping
    visible: bool,
    locked: bool,
    solo: bool,
    muted: bool,
    // Layer-level defaults (overridden by per-clip values when set)
    motion_blur_enabled: bool,       // Per-layer default, uses comp shutter settings
    blend_mode: BlendMode,
    opacity: Animatable<f32>,        // 0.0 - 1.0, multiplied with clip opacity
    transform: Transform,            // Layer base transform (applied on top of clip transform)
    scale_interpolation: ScaleInterpolation,
    masks: Vec<Mask>,                // Layer-level masks (applied to all clips)
    parent_id: Option<LayerId>,      // AE-style parenting — inherits parent's transform
    effects: Vec<EffectInstance>,
    // Audio-only properties
    audio_fx_chain: Option<AudioFxChain>,
}
```

### Property Resolution Order

For visual properties that exist on both Layer and Clip:
1. **Transform**: Clip transform applied first (local space), then Layer transform on top (like AE's layer → parent chain)
2. **Opacity**: Clip opacity × Layer opacity (multiplicative)
3. **Scale Interpolation**: Clip value if set, otherwise Layer value
4. **Blend Mode**: Clip value if set, otherwise Layer value
5. **Motion Blur**: Clip value if set, otherwise Layer value
6. **Masks**: Clip masks applied first, then Layer masks on top
7. **Effects**: Clip effects applied first, then Layer effects on top
8. **Volume** (audio): Clip volume × Layer volume envelope (multiplicative)

## LayerType

```rust
enum LayerType {
    Video,   // Can hold video clips, images, image sequences
    Audio,   // Audio-only clips
    Text,    // Text elements
    Null,    // Transform-only layer (no visual content), used as parent for other layers
}
```

## Clip

A clip is a segment of source media placed on the timeline. Clips have their own visual properties that override layer defaults when set.

`scale_axes_linked` is editor-facing authoring metadata rather than a compositor input. It controls whether the inspector keeps X and Y scale edits in lockstep for that clip, defaults to `true`, and persists with the clip so the remembered link state survives reselection and reload.

Newly inserted visual media clips initialize `transform.anchor_point.default_value` to the center of the source media bounds. If source dimensions are unavailable at insert time, the fallback default is the active composition center.

```rust
struct Clip {
    id: ClipId,
    media_asset_id: Option<MediaAssetId>,  // None for gaps, text, or generated
    timeline_start: u64,                    // Frame position on timeline
    duration_frames: u64,                   // Visible duration on timeline
    source_in: u64,                         // In-point in source media (frames)
    source_out: u64,                        // Out-point in source media (frames)
    speed: f64,                             // 1.0 = normal, 0.5 = half speed, etc.
    time_remap: Option<Animatable<f64>>,    // Keyframeable time remap (overrides speed)
    transition_in: Option<Transition>,
    transition_out: Option<Transition>,
    // Per-clip visual properties (override layer defaults)
    transform: Transform,                   // Clip-local transform (applied before layer transform)
    scale_axes_linked: bool,                // Inspector authoring state for clip scale X/Y, default true
    opacity: Animatable<f32>,               // 0.0 - 1.0, multiplied with layer opacity
    scale_interpolation: Option<ScaleInterpolation>,  // None = use layer default
    blend_mode: Option<BlendMode>,          // None = use layer default
    motion_blur_enabled: Option<bool>,      // None = use layer default
    masks: Vec<Mask>,                       // Per-clip masks (applied before layer masks)
    effects: Vec<EffectInstance>,           // Per-clip effects (applied before layer effects)
    // Per-clip audio properties
    volume: Animatable<f32>,                // 0.0 - 1.0, multiplied with layer volume envelope
    audio_microfade_in_ms: f32,             // Non-keyframed edge fade-in in milliseconds, default 0.0
    audio_microfade_out_ms: f32,            // Non-keyframed edge fade-out in milliseconds, default 0.0
    audio_sync_mode: AudioSyncMode,         // Free / Bed / Connected (audio clips only)
    /// Sync point: sample offset within the source audio marking the "hit" or key moment.
    /// Auto-detected from first transient on import. When a connected clip is placed,
    /// the sync point aligns to the anchor frame. Sub-frame (sample-accurate) positioning.
    sync_point_samples: Option<u64>,        // None = no sync point set (defaults to clip start)
    // Connected clip: anchored to a frame on another layer.
    // When the anchor layer ripples, this clip shifts to maintain relative position.
    connection: Option<ClipConnection>,
    // Sub-composition reference (for nested comps)
    sub_comp_id: Option<CompId>,
}

struct ClipConnection {
    anchor_layer_id: LayerId,
    anchor_frame: u64,              // The frame on the anchor layer this clip is attached to
}

/// Audio-specific sync mode. Determines how an audio clip relates to video clips.
enum AudioSyncMode {
    /// No sync — fixed timeline position, unaffected by other layers' ripple.
    Free,
    /// Duration-synced to a contiguous range of clips on a reference layer.
    /// Bed auto-adjusts when clips in the range are split, deleted, or trimmed.
    Bed(BedSync),
    /// Position-synced to a specific frame (uses ClipConnection).
    /// For SFX, foley, stings. Moves with anchor on ripple.
    Connected,
}

struct BedSync {
    reference_layer_id: LayerId,
    /// The clip IDs this bed spans. Ordered by timeline position.
    /// Bed start = start of first clip, bed end = end of last clip.
    synced_clip_ids: Vec<ClipId>,
}

/// Linked A/V pair: video and audio clips from the same source file.
struct LinkedAvPair {
    video_clip_id: ClipId,
    audio_clip_id: ClipId,
}
```

## Transform

```rust
struct Transform {
    position: Animatable<Vec2>,      // (x, y) in pixels from comp center
    scale: Animatable<Vec2>,         // (x, y) as multipliers, 1.0 = 100%
    rotation: Animatable<f32>,       // Degrees
    anchor_point: Animatable<Vec2>,  // Pivot point relative to source/layer bounds
}

struct EvaluatedTransform {
    position: Vec2,
    scale: Vec2,
    rotation: f32,
    anchor_point: Vec2,
}

impl Transform {
    fn evaluate_at(frame: u64) -> EvaluatedTransform;
}
```

`audio_microfade_in_ms` / `audio_microfade_out_ms` are distinct from `volume` automation. They are tiny click-prevention fades applied at the clip's timeline edges in both live mixed playback and export audio mixing.

## ScaleInterpolation

```rust
enum ScaleInterpolation {
    NearestNeighbor,  // Crisp pixel art
    Bilinear,         // Smooth standard
    Bicubic,          // Sharp smooth
}
```

Set per-clip (with layer as fallback default). Honored during both preview and export.

## BlendMode

```rust
enum BlendMode {
    Normal,
    Add,
    Multiply,
    Screen,
    Overlay,
}
```

## LayerGroup

A group provides a shared base transform for member layers. Member transforms are relative to the group's transform.

```rust
struct LayerGroup {
    id: LayerGroupId,
    name: String,
    member_layer_ids: Vec<LayerId>,
    transform: Transform,            // Group base transform
    opacity: Animatable<f32>,        // Group opacity (multiplied with member opacity)
}
```

## Mask

Drawn directly on a layer. Multiple masks per layer, combined via mask mode.

```rust
struct Mask {
    name: String,
    path: Animatable<BezierPath>,    // Entire path is keyframeable (vertex animation)
    mode: MaskMode,
    opacity: Animatable<f32>,        // 0.0 - 1.0
    feather: Animatable<f32>,        // Edge softness in pixels
    inverted: bool,
}

enum MaskMode {
    Add,
    Subtract,
    Intersect,
}

struct BezierPath {
    points: Vec<BezierPoint>,
    closed: bool,
}

struct BezierPoint {
    position: Vec2,
    in_tangent: Vec2,                // Control handle toward previous point
    out_tangent: Vec2,               // Control handle toward next point
}
```

**Mask path animation**: The entire `BezierPath` is keyframed as a unit (like AE). At each keyframe, the full set of vertices is stored. Interpolation between keyframes lerps each corresponding vertex position and tangent. Keyframes must have the same number of vertices (adding/removing vertices creates a new keyframe with the new vertex count).

## Transition

```rust
struct Transition {
    transition_type: TransitionType,
    duration_frames: u64,
    params: TransitionParams,
}

enum TransitionType {
    CrossDissolve,
    LumaFade,
    DipToBlack,
    DipToWhite,
}

struct TransitionParams {
    // LumaFade-specific
    luma_gamma: Option<f32>,  // Controls threshold curve, default 1.0
}
```

## MediaAsset

```rust
struct MediaAsset {
    id: MediaAssetId,
    file_path: PathBuf,              // Relative when under the project base, absolute otherwise
    media_type: MediaType,
    duration_frames: Option<u64>,    // None for stills
    resolution: Option<Resolution>,  // None for audio-only
    frame_rate: Option<FrameRate>,
    sample_rate: Option<u32>,
    channels: Option<u16>,           // Audio channels
}

enum MediaType {
    Video,
    Audio,
    Image,
    ImageSequence,
}
```

## Workspace State

The project file persists enough editor state to reopen into the same working context instead of only restoring authored timeline data.

```rust
struct WorkspaceState {
    active_comp_id: Option<CompId>,          // Defaults to master_comp_id when migrated from v1
    playhead_frame: u64,
    timeline_scroll_x: f32,
    pixels_per_frame: f32,                   // Default 6.0
    timeline_track_height: f32,              // Default 40.0
    selected_media_asset_id: Option<MediaAssetId>,
    selected_clip: Option<WorkspaceClipSelection>,
    pane_layout: WorkspacePaneLayout,
}

struct WorkspaceClipSelection {
    layer_id: LayerId,
    clip_id: ClipId,
}

enum WorkspacePaneKind {
    ProjectBrowser,
    Viewer,
    Inspector,
    Timeline,
}

enum WorkspaceSplitAxis {
    Horizontal,
    Vertical,
}

enum WorkspacePaneLayout {
    Pane(WorkspacePaneKind),
    Split {
        axis: WorkspaceSplitAxis,
        ratio: f32,                          // Must be finite and strictly between 0.0 and 1.0
        a: Box<WorkspacePaneLayout>,
        b: Box<WorkspacePaneLayout>,
    },
}
```

Workspace validation rules:
- `active_comp_id`, when present, must refer to an existing composition.
- `selected_media_asset_id`, when present, must refer to an existing media asset.
- `selected_clip`, when present, must refer to an existing clip on an existing layer.
- `pane_layout` must contain each core pane kind exactly once: `ProjectBrowser`, `Viewer`, `Inspector`, `Timeline`.
- Split ratios must be finite and in the open interval `(0, 1)`.

## Animatable (Keyframe System)

```rust
struct Animatable<T> {
    keyframes: Vec<Keyframe<T>>,
    // If keyframes is empty, `default_value` is used as a static value
    default_value: T,
}

struct Keyframe<T> {
    frame: u64,
    value: T,
    easing: EasingType,
}

enum EasingType {
    Linear,
    Hold,                            // Step/hold — no interpolation
    Bezier(BezierEasing),
    Preset(EasingPresetId),          // Reference to named preset
}

struct BezierEasing {
    // Control points for cubic bezier, normalized 0-1 on time axis
    x1: f32,
    y1: f32,
    x2: f32,
    y2: f32,
}
```

**Evaluation behavior (`Animatable::value_at`)**
- If `keyframes` is empty: return `default_value`
- Before first keyframe: return first keyframe value
- After last keyframe: return last keyframe value
- Between keyframes:
  - `Linear`: lerp between values
  - `Hold`: return previous keyframe value (step)
  - `Bezier` / `Preset`: currently fall back to linear interpolation until custom curve evaluation is implemented

**Supported interpolation value types**
- `f32`, `f64`, `Vec2`: numeric lerp
- `BezierPath`: lerp each corresponding vertex/tangent; if keyframes have different vertex counts, hold previous shape

## Text Properties

```rust
struct TextProperties {
    content: String,
    font_family: String,
    font_size: f32,
    color: Color,
    alignment: TextAlignment,
    stroke: Option<Stroke>,
    shadow: Option<DropShadow>,
    background_plate: Option<BackgroundPlate>,
}

enum TextAlignment { Left, Center, Right }

struct Stroke {
    color: Color,
    width: f32,
}

struct DropShadow {
    color: Color,
    offset: Vec2,
    blur: f32,
    opacity: f32,
}

struct BackgroundPlate {
    color: Color,
    padding: f32,
    corner_radius: f32,
    opacity: f32,
}
```

## Marker

```rust
struct Marker {
    frame: u64,
    label: String,
    color: Color,
}
```

## Color

```rust
struct Color {
    r: f32, g: f32, b: f32, a: f32,  // 0.0 - 1.0 linear
}
```

## Internal Color Pipeline

The compositor works in **16-bit float (f16) per channel, RGBA, linear color space**. This applies to all intermediate textures, compositing operations, and effect processing. Source media is decoded and converted to linear on upload. Final output is converted to sRGB/target space on export or preview display.

## ID Types

All IDs are `u64` newtypes, monotonically incrementing within a project.

```rust
struct CompId(u64);
struct LayerId(u64);
struct LayerGroupId(u64);
struct ClipId(u64);
struct MediaAssetId(u64);
struct EasingPresetId(u64);
```