student_ai_video_pipeline_guide_0526

# Local AI Video Production Pipeline — Student Edition (May 2026)

---

## Overview

This guide describes a fully offline, modular AI filmmaking pipeline split across two machines:

| Machine | Role |
|---|---|
| **M2 Mac (96GB Unified Memory)** | LLM shot planning, FLUX image generation, multi-character identity injection, cinematic stills |
| **RTX 3080 Ti PC (16GB VRAM)** | WAN 2.1 video animation, RIFE frame interpolation, upscaling, batch rendering |

The architecture is **not** a single giant ComfyUI graph. It is five specialized graphs that pass frames sequentially through a shared storage location.

---

## Why Modular?

A single-pass SDXL-style workflow is suitable for isolated poster images but structurally unsuitable for:

- Multi-character identity continuity
- Cinematic shot iteration
- Storyboard orchestration
- Image-to-video animation
- Temporal consistency

The modular approach prevents graph instability, simplifies iteration, allows shot-by-shot regeneration, avoids VRAM fragmentation, and makes debugging manageable.

---

## Locked Model Stack

| Component | Model | Machine |
|---|---|---|
| **LLM** | `unsloth/Qwen3-35B-A3B-UD-MLX-4bit` (MLX backend) | M2 Mac |
| **Image Generation** | FLUX.2-dev | M2 Mac |
| **Skeletal Identity** | PuLID Flux2 (`pulid_flux2_klein_v2.safetensors`) | M2 Mac |
| **Facial Continuity** | IPAdapter FaceID | M2 Mac |
| **Pose Continuity** | OpenPose | M2 Mac |
| **Spatial Continuity** | Depth Anything | M2 Mac |
| **Video** | WAN 2.2 (image-to-video mode only) | RTX 3080 Ti |
| **Interpolation** | RIFE 4.x | RTX 3080 Ti |
| **Editorial** | DaVinci Resolve | Either |

> Do **not** use Juggernaut XL, SDXL, or Schnell for the cinematic pipeline.
> Do **not** use WAN 2.1 in text-to-video mode.

---

## Introduction

This guide teaches beginners how to build a fully local AI video-production workflow for short-form cinematic generation using:

- an Apple Silicon Mac for image generation and planning
- a Windows NVIDIA PC for local AI video generation
- ComfyUI for node-based workflows
- FLUX for cinematic still generation
- WAN 2.2 for image-to-video animation
- RIFE for frame interpolation

The goal of this document is NOT to create a one-click “generate movie” button.

The goal is to teach:

- structured local AI video production
- modular workflows
- shot planning
- identity consistency
- cinematic continuity
- reliable local generation
- beginner-friendly troubleshooting

---

# IMPORTANT EXPECTATIONS

## Local AI Video Production Is NOT Instant

In 2026, local AI video production still requires:

- experimentation
- rendering time
- troubleshooting
- iterative refinement

Even powerful hardware may require:

- several minutes per shot
- multiple regeneration passes
- manual editing
- shot-by-shot correction

Professional-looking results usually come from:

- disciplined shot planning
- short clips
- controlled camera motion
- consistent character references
- strong editing
- sound design

NOT giant prompts.

Most successful local AI video workflows in 2026 rely heavily on:

- modular generation stages
- image-to-video pipelines
- editorial assembly
- iterative shot correction

rather than fully automated generation systems.

---

# OVERALL PIPELINE

Your workflow is divided into 5 major stages.

## Stage 1 — Shot Planning

The local LLM converts screenplay text into:

- shot descriptions
- lens choices
- lighting notes
- motion prompts
- generation seeds

## Stage 2 — Environment Generation

FLUX generates cinematic still frames WITHOUT characters.

These become:

- master lighting references
- composition references
- environment plates

## Stage 3 — Structural Maps

Control maps establish:

- pose
- depth
- staging
- perspective

before character injection.

## Stage 4 — Character Injection

Characters are inserted independently using:

- PuLID
- IPAdapter FaceID
- regional masking

This helps reduce:

- face blending
- identity drift
- character corruption

## Stage 5 — Video Animation

WAN 2.2 converts still frames into short video clips.

RIFE then interpolates frames for smoother motion.

---

# WHY MODULAR WORKFLOWS MATTER

One of the biggest beginner mistakes is attempting:

- giant workflows
- one-click movie generation
- enormous prompts
- fully automated pipelines immediately

Do NOT do this.

Current practitioner workflows in 2026 commonly favor:

- small workflows
- modular graphs
- separate rendering stages
- independent validation
- shot-by-shot iteration

This improves:

- stability
- troubleshooting
- render consistency
- project organization
- workflow recovery

(Source: Official ComfyUI documentation and practitioner workflow tutorials)

Official references:

- https://github.com/comfyanonymous/ComfyUI
- https://docs.comfy.org/

---

# RECOMMENDED HARDWARE

## Mac System

Recommended:

- Apple Silicon M2/M3/M4
- 64GB–128GB unified memory preferred

Used for:

- FLUX image generation
- prompt encoding
- planning
- identity workflows

## Windows NVIDIA System

Used for:

- WAN 2.2 image-to-video
- interpolation
- upscaling
- video rendering

## Practical VRAM Guidance

| Tier | Practical Experience |
|---|---|
| 8GB VRAM | Entry-level experimentation only |
| 12–16GB VRAM | Functional for constrained 480p image-to-video workflows |
| 24GB+ VRAM | Recommended for stable iterative production |
| 32GB+ VRAM | Comfortable for larger modular pipelines |

VRAM requirements vary substantially depending on:

- ControlNet usage
- quantization strategy
- node overhead
- frame count
- preprocessing stages
- simultaneous model loading

Modern local AI video generation benefits heavily from CUDA acceleration.

(Source: Official WAN 2.2 ComfyUI documentation)

References:

- https://docs.comfy.org/tutorials/video/wan/wan2_2
- https://developer.nvidia.com/cuda-zone

---

# SOFTWARE OVERVIEW

| Software | Purpose |
|---|---|
| ComfyUI | Node-based AI generation |
| Open WebUI | Local LLM runtime for chatting|
| opencode | Local LLM runtime for building with agents |

---

## Machine Setup

### M2 Mac

The Mac uses its unified memory architecture to handle heavy T5 XXL prompt encoders and multi-character regional injection without hitting memory ceilings.

**ComfyUI launch flags:**
```bash
python main.py --use-split-cross-attention --force-fp16 --lowvram
```
`--lowvram` offloads heavy text encoders to system memory, preventing spikes during multi-stage sampling.

**M2 Mac optimizations:**

| Optimization | Purpose |
|---|---|
| FP8 FLUX | Lower memory usage |
| TeaCache | Faster diffusion |
| Split Attention | Better Metal efficiency |
| Tiled VAE | Prevent memory spikes |
| Sequential Rendering | Reduce thermal throttling |

**Models to download and place:**

| Model | Path |
|---|---|
| `flux2-dev.safetensors` | `models/unet/` |
| `pulid_flux_klein_v2.safetensors` | `models/pulid/` |
| '1k3d68.onnx' | 'models/insightface/models/antelopev2' |
| 2d106det.onnx | 'models/insightface/models/antelopev2' |
| genderage.onnx | 'models/insightface/models/antelopev2' |
| glintr100.onnx | 'models/insightface/models/antelopev2' |
| scrfd_10g_bnkps.onnx | 'models/insightface/models/antelopev2' |
| `EVA02-CLIP-L-14-336` | `models/clip/` |
| mistral_3_small_flux2_bf16.safetensors | 'models/text_encoders' |


**ComfyUI extensions to install:**
- ComfyUI Manager
- ComfyUI PuLID Flux
- ComfyUI Impact Pack
- ComfyUI LayerStyle
- ComfyUI ControlNet Aux

---

### RTX 3080 Ti PC

The PC is a CUDA-accelerated render node. NVIDIA CUDA massively outperforms Metal for video diffusion; tensor cores accelerate interpolation.

**ComfyUI launch flags:**
```cmd
python main.py --highvram --gpu-only --use-quad-cross-attention
```

**Models to download:**
- WAN 2.1 image-to-video checkpoint (quantized versions fit inside 16GB VRAM)
- RIFE 4.x model files

**PC optimizations:**

| Optimization | Purpose |
|---|---|
| xFormers | Faster attention |
| CUDA FP16 | Tensor acceleration |
| TensorRT (optional) | Faster inference |
| Batch Rendering | High throughput |

**ComfyUI extensions to install:**
- ComfyUI Frame Interpolation

---

## Pipeline Overview

```
M2 MAC
───────────────────────────────────────
Graph 1: LLM Shot Planner
  (Script → Structured JSON)
          ↓
Graph 2: Environment Plate Generation
  (FLUX.2 Dev — cinematic still, no characters)
          ↓
Graph 3: Structural Control Map Extraction
  (OpenPose + Depth Anything)
          ↓
Graph 4: Regional Multi-PuLID Character Injection
  (3-character identity lock)
          ↓
        [Shared Storage]
          ↓
RTX 3080 Ti PC
───────────────────────────────────────
Graph 5: WAN 2.1 Animation + RIFE Interpolation
  (Final composited still → cinematic clip)
          ↓
DaVinci Resolve (editorial assembly)
```

---


# THE LOCKED MODEL STACK

## Image Generation

### Recommended Model

FLUX.2-dev

FLUX models are widely regarded in current practitioner workflows as offering:

- strong cinematic realism
- high-quality lighting response
- coherent composition
- effective prompt interpretation for film-oriented still generation

These characteristics have made FLUX-based workflows common in cinematic ComfyUI pipelines.

(Source: Black Forest Labs model documentation and practitioner workflows)

Official reference:

- https://huggingface.co/black-forest-labs

Instead of relying entirely on external identity adapters, Flux accepts:

- a reference image
- a text instruction

simultaneously.

This allows:

- iterative edits
- character preservation
- localized image modification
- style continuity across revisions

Key capabilities include:

- character consistency from reference images
- local editing
- style preservation
- simplified workflow structure

## Comparison

| | PuLID + IPAdapter | FLUX.2 |
|---|---|---|
| Setup complexity | Higher | Lower |
| Regional multi-character control | Strong | More limited |
| Single-character consistency | Strong | Strong |
| License | Apache 2.0 | FLUX non-commercial license |
| VRAM usage | Moderate | Higher |

For single-character workflows, Flux may simplify setup.

For multi-character regional injection workflows, PuLID + masking remains widely used.

Official references:

- https://huggingface.co/black-forest-labs/FLUX.2-dev
- https://github.com/ToTheBeginning/PuLID

---

# VIDEO GENERATION

## Recommended Model

WAN 2.2 (image-to-video mode)

WAN 2.2 is an open-weight video-generation model commonly used in local ComfyUI video workflows.

The model supports:

- image-to-video generation
- cinematic motion generation
- quantized inference workflows
- FP8 workflows
- modular ComfyUI integration

WAN 2.2 uses a Mixture-of-Experts (MoE) architecture that separates broader motion/layout processing from later refinement stages.

(Source: Official WAN repository and ComfyUI documentation)

Official references:

- https://github.com/Wan-Video
- https://docs.comfy.org/tutorials/video/wan/wan2_2

---

# USE IMAGE-TO-VIDEO FIRST

Beginners should avoid starting with:

- text-to-video
- long prompts
- complicated choreography
- large action sequences

Current practitioner workflows strongly favor image-to-video generation for:

- continuity control
- shot stability
- identity preservation
- editorial consistency

Common production workflow:

FLUX keyframe → WAN image-to-video.

(Source: Official ComfyUI WAN tutorials)

Reference:

- https://docs.comfy.org/tutorials/video/wan/wan2_2

---

# KEEP MOTION SMALL

AI video models still struggle with:

- fighting
- running
- large crowds
- complex choreography
- aggressive camera movement

More reliable results usually come from:

- subtle blinking
- breathing
- slow dolly motion
- ambient movement
- gentle camera drift

Controlled motion remains one of the most important quality factors in local AI video generation.

---

# DIRECTORY STRUCTURE

Create this shared folder structure:

```text
/AI_FILM_PROJECT
    /scripts
    /shots
        /scene_01
            /plates
            /poses
            /depth
            /characters
            /final_frames
            /video_raw
            /video_interp
    /audio
    /resolve
```

This structure helps prevent:

- missing assets
- broken paths
- accidental overwrites
- organizational chaos

---

# USING OPENCODE IN THE PIPELINE

## Shot Planning & Scripting

Use `opencode` to generate structured shot lists, narrative breakdowns, and cinematic prompts. This directly supports the guide's emphasis on shot planning and modular workflows.

**OpenCode Execution Steps:**

1. Run `/new` to isolate this phase, then create `shots.md`.
2. Paste your screenplay scene and prompt OpenCode with the cinematic storyboard system rules to output valid JSON shot descriptions.
3. Use `@shots.md` to reference the generated list, then run `/compact` to summarize narrative beats and verify shot lengths (3–6 seconds) before exporting.

---

## Prompt Engineering & Identity Consistency

Leverage it to draft, refine, and version-control prompts for FLUX and WAN 2.2, ensuring character consistency and cinematic continuity across shots.

**OpenCode Execution Steps:**

1. Create a `prompts/` directory and initialize version tracking.
2. Feed OpenCode your character references and continuity rules (wardrobe, lighting, environment) to draft base prompts.
3. Use OpenCode's `@prompts/` autocomplete to swap parameters for FLUX and WAN 2.2, then run `/undo` if identity drift occurs, while logging stable sequential seeds in your version files.

---

## ComfyUI Workflow Assistance

Deploy agent capabilities to automate repetitive node configurations, parameter adjustments, or graph validation. This aligns with the guide's recommendation for independent validation and modular graphs.

**OpenCode Execution Steps:**

1. Break your ComfyUI graph into modular subgraphs and save each stage as a separate JSON or YAML config.
2. Prompt OpenCode to generate node configuration snippets for each modular block, referencing the guide's validation standards.
3. Use OpenCode's shell prefix (`!`) to run quick ComfyUI API tests or validate outputs, and toggle `/details` to inspect how the model maps your modular requirements to node parameters before merging stages.

---

## Troubleshooting & Iteration

Use it to analyze ComfyUI logs, suggest fixes for memory/render errors, and guide shot-by-shot iteration, which the guide highlights as essential for stability.

**OpenCode Execution Steps:**

1. Paste raw ComfyUI error logs directly into the TUI and prompt OpenCode to cross-reference with the guide's stability and troubleshooting recommendations.
2. Enable `/thinking` to trace the model's diagnostic logic, then apply suggested fixes incrementally.
3. Use `/undo` to revert any breaking changes, maintain a `troubleshooting.md` log for shot-by-shot iteration notes, and run `/compact` to archive resolved issues before proceeding to the next shot.

---

# GRAPH 1 — LLM SHOT PLANNER (Mac)

## Purpose

Convert screenplay text into:

- cinematic shot lists
- motion prompts
- lighting descriptions
- structured metadata

---

## Recommended LLM

`unsloth/Qwen3-35B-A3B-UD-MLX-4bit`

Run locally using:

- Open WebUI
- MLX backends

---

## Beginner Advice

Do NOT ask the LLM to:

- generate entire films
- write giant prompts
- create 30-second shots

Instead generate:

- short shots
- simple actions
- clear compositions
- stable continuity


This is **not** a ComfyUI graph. It runs via your local LLM runtime using `unsloth/Qwen3-35B-A3B-UD-MLX-4bit` on the MLX backend.

**Purpose:** Convert screenplay pages into structured shot metadata, lens language, motion prompts, continuity seeds, lighting instructions, and FLUX prompt packets.

### System Prompt

```
You are a cinematic storyboard planner for a local AI filmmaking pipeline.
Your job is to convert screenplay scenes into structured JSON shot descriptions
optimized for:
- FLUX.2 Dev image generation
- PuLID multi-character identity consistency
- WAN 2.1 image-to-video animation

Rules:
- Keep shots between 3 and 6 seconds.
- Prefer cinematic cuts over long continuous shots.
- Maintain continuity of wardrobe, lighting, and environment.
- Use professional cinematography terminology.
- Describe camera movement precisely.
- Include lens selection.
- Include environmental lighting.
- Include motion prompts suitable for subtle AI animation.
- Use stable sequential seeds.

Output ONLY valid JSON.
```

### Output Schema

```json
{
  "scene": 1,
  "shot": 12,
  "duration": 4,
  "camera": "slow dolly-in",
  "lens": "85mm anamorphic",
  "lighting": "warm sunset practicals",
  "environment": "",
  "characters": ["Char_A", "Char_B"],
  "composition": "",
  "motion": "subtle breathing and blinking",
  "seed": 10421042,
  "clip_style": "",
  "flux2_prompt": "",
}
```

### User Prompt (Feed with Screenplay Pages)

```
Convert this screenplay scene into a cinematic shot list optimized for:
- FLUX.2 Dev
- PuLID
- WAN 2.1 image-to-video

Prioritize:
- cinematic realism
- shot continuity
- subtle motion
- visual storytelling
- manageable local generation complexity

Avoid:
- impossible motion
- excessive choreography
- crowded compositions
- long continuous shots
```

---

# GRAPH 2 — ENVIRONMENT PLATE GENERATION

## Purpose

Generate:

- lighting
- atmosphere
- framing
- composition
- cinematic environments

WITHOUT characters.

These become:

- master cinematography plates
- lighting references
- composition references

---

## Why Separate Environment From Characters?

Separating environments from character injection improves:

- composition stability
- lighting consistency
- regeneration flexibility
- identity control

This workflow structure is widely used in modular ComfyUI video pipelines.

---

## Beginner FLUX Settings

The following settings are practical beginner-oriented starting points derived from commonly shared community workflows. They are not universal requirements.

| Parameter | Recommended |
|---|---|
| Steps | 28–32 |
| CFG | 1.5–2.0 |
| Resolution | 1024×576 |
| Sampler | Euler |
| Scheduler | Beta |

---

# GRAPH 3 — CONTROL MAP EXTRACTION

## Purpose

Generate geometrically absolute control maps by rendering 3D human meshes directly from live-action reference photographs, bypassing 2D skeletal estimation entirely to preserve true volume, occlusion, and shape prior to character injection.

## Core Pipeline: Live-Action to Direct Render

1. **Reference Capture:** Photograph actors performing the target scenes.
2. **3D Reconstruction:** Process the RGB photographs through **SAM 3D Body** to generate full-body Momentum Human Rig (MHR) 3D meshes.
3. **Map Extraction (Direct Render):** Instead of extracting 2D joints, render the 3D mesh directly into dense, pixel-perfect control formats:
* **Normal Maps:** Render the surface normals of the 3D mesh to provide the diffusion model with absolute structural lighting, mass, and contour data.
* **Subject Depth Maps:** Render the literal Z-depth of the 3D mesh, ensuring flawless occlusion handling for complex poses.
* **DensePose Maps (Optional):** Render UV surface mapping of the mesh to track exact surface points, which is highly effective for locking clothing and texture placement.

---

## Recommended Tools

| Tool | Purpose | Pipeline Role |
| --- | --- | --- |
| **SAM 3D Body** | 3D Mesh Reconstruction | **Primary Engine:** Converts actor photos into accurate 3D meshes (GLB/PLY), decoupling skeleton structure from surface shape. |
| **3D Render Engine (e.g., ComfyUI 3D nodes / Blender)** | Mesh to Map Conversion | **Map Generator:** Renders the SAM 3D meshes directly into 2D Normal, Depth, and DensePose maps for ControlNet. |
| **Depth Anything / MiDaS** | Environment Depth | **Background Map:** Generates depth continuity for the *scene/background*, onto which the absolute SAM 3D subject depth is composited. |

### Node Flow

```
Load Image (Master Plate) ─→ OpenPose Preprocessor ──→ Save Image: /poses/
Load Image (Master Plate) ─→ Depth Anything Preprocessor ──→ Save Image: /depth/
```

**Official References:**

* Meta AI / SAM 3D Body (Hugging Face / GitHub)
* [https://github.com/LiheYoung/Depth-Anything](https://github.com/LiheYoung/Depth-Anything)
* [https://github.com/isl-org/MiDaS](https://github.com/isl-org/MiDaS)

---

## Why this architecture wins

* **Volume and Contour Preservation:** Stick figures provide structure but lose all body mass. Normal maps rendered directly from a 3D mesh force the diffusion model to respect the exact physical dimensions and muscle contours of your actor.
* **Flawless Occlusion:** When an actor crosses their arms, interacts with props, or turns away, 2D pose estimators often guess or drop joints entirely. A direct 3D depth render mathematically knows exactly which limb is in front of the other.
* **Temporal Coherence:** For sequential frames or video workflows, rendering a smooth 3D mesh eliminates the high-frequency jitter common in 2D pose detection. The stable surface normals and depth maps lock the character's geometry in place across time, preventing the "boiling" effect in generated output.
---

# GRAPH 4 — CHARACTER INJECTION

This is one of the most important workflow stages.

---

## Purpose

Inject characters independently while preserving:

- identity
- facial structure
- consistency
- spatial separation

---

## Core Beginner Rule

Do NOT inject multiple characters simultaneously without masks.

This increases the risk of:

- face blending
- anatomy corruption
- identity drift

Use:

- regional masks
- separate character passes
- separate seeds

---

## Beginner Character Advice

Start with:

- one character
- medium shots
- stable lighting
- frontal faces

Attempt:

- multiple characters
- profile angles
- dramatic motion

only after the pipeline becomes stable.

---

## Recommended Identity Settings

The following settings are practical beginner-oriented starting points derived from commonly shared community workflows. They are not universal requirements.

| Shot Type | PuLID | IPAdapter |
|---|---|---|
| Wide | 0.60 | 0.20 |
| Medium | 0.75 | 0.25 |
| Close-Up | 0.85 | 0.35 |

---

### Required Nodes (Per Character)

| Node | Purpose |
|---|---|
| Load Image | Character face reference |
| Load PuLID Flux Model | Loads PuLID |
| Load EVA Clip | Loads EVA02 encoder |
| InsightFace Loader | Facial analysis |
| Apply PuLID Flux | Identity lock |
| Apply IPAdapter FaceID | Organic facial detail |
| Regional Mask | Spatial isolation |

### Node Flow

```
Load Diffusion Model ──────────────────→ Apply PuLID Flux (Char A)
                                                  ↓
                                         Apply PuLID Flux (Char B)
                                                  ↓
                                         Apply PuLID Flux (Char C)
                                                  ↓
                                             KSampler
                                                  ↓
                                            VAE Decode
                                                  ↓
                                    Save: Final_Composited_Still.png

Per character:
  Face_Ref_X.png ──────────→ Apply PuLID Flux (image input)
  Load PuLID Flux Model ───→ Apply PuLID Flux (pulid_flux)
  Load EVA Clip ───────────→ Apply PuLID Flux (eva_clip)
  InsightFace Loader ──────→ Apply PuLID Flux (face_analysis)
  Regional Mask ───────────→ Apply PuLID Flux (mask)
  Apply PuLID Flux ────────→ Apply IPAdapter FaceID
  Apply IPAdapter FaceID ──→ Regional Composite
```

### Seed Families (Prevents Identity Blending)

| Character | Seed Range |
|---|---|
| Character A | 1000–1999 |
| Character B | 2000–2999 |
| Character C | 3000–3999 |

### Identity Weight Settings

| Shot Type | PuLID Weight | IPAdapter Weight |
|---|---|---|
| Wide | 0.55–0.70 | 0.15–0.20 |
| Medium | 0.70–0.80 | 0.20–0.30 |
| Close-up | 0.80–0.90 | 0.30–0.40 |

### Critical PuLID Parameters

| Parameter | Value |
|---|---|
| start_at | 0.0 |
| end_at | 0.85 |
| weight | 0.75 |

### Refinement Pass (Post-Injection)

After compositing all characters, run a low-denoise KSampler pass to merge lighting, atmosphere, grain, and cinematic tone into one coherent hero frame.

| Parameter | Value |
|---|---|
| Denoise | 0.20–0.35 |
| Shared seed | Yes |

Recommended nodes: Image Composite, Ultimate SD Upscale, Color Match, Film Grain Node, Low-Denoise KSampler.


# GRAPH 5 — VIDEO GENERATION

## Purpose

Convert still frames into:

- cinematic clips
- subtle motion
- temporally coherent shots

---

## Current Model: WAN 2.2

WAN 2.2 is commonly used in local ComfyUI image-to-video workflows.

Official references:

- https://github.com/Wan-Video
- https://docs.comfy.org/tutorials/video/wan/wan2_2

---

## Model Files

Choose model variants based on available VRAM.

| VRAM | Recommended Approach |
|---|---|
| 24GB+ | FP8 workflows |
| 12–16GB | Quantized workflows |
| 8GB | Lightweight model variants |

Place diffusion models in:

```text
ComfyUI/models/diffusion_models/
```

Additional components commonly include:

- text encoders
- VAE files
- quantized checkpoints

Always verify:

- current file naming
- model compatibility
- workflow support

from official repositories before installation.

---

## Native WAN Support

ComfyUI includes native WAN support.

Some practitioners additionally use Kijai’s WanVideoWrapper workflows for expanded routing and workflow compatibility.

Official references:

- https://docs.comfy.org/tutorials/video/wan/wan2_2
- https://github.com/kijai/ComfyUI-WanVideoWrapper

---

## Beginner WAN Settings

The following settings are practical beginner-oriented starting points derived from commonly shared community workflows. They are not universal requirements.

| Parameter | Recommended |
|---|---|
| FPS | 8 |
| Duration | 3 sec |
| Resolution | 480p |
| Denoise | 0.80–0.90 |

These settings are generally more stable for beginners than aggressive high-resolution workflows.

---

## Motion Prompt Example

```text
subtle breathing,
soft blinking,
ambient hair movement,
slow cinematic drift,
natural posture adjustment
```

### Animation Node Flow

```
Load Image (Final_Composited_Still.png)
          ↓
WAN 2.1 Image-to-Video Model
  (motion prompts from Graph 1 JSON)
          ↓
Raw Video Tensor
          ↓
RIFE 4.x Interpolation Node (2x or 4x multiplier → 60fps)
          ↓
Save Video (ProRes or DNxHR, high-bitrate)
          ↓
/shots/scene_XX/video_interp/
```

---

# RIFE INTERPOLATION

## Purpose

Convert low-FPS clips into smoother playback.

This is usually less expensive than generating higher FPS directly.

---

## Beginner Recommendation

Generate:

- 8 FPS

Then interpolate to:

- 24 FPS

24 FPS already produces cinematic motion for most workflows.

Official references:

- https://github.com/hzwer/ECCV2022-RIFE

---

# BEGINNER TROUBLESHOOTING

## Problem: Faces Keep Changing

Possible causes:

- weak identity weights
- no masks
- excessive motion

Possible fixes:

- increase PuLID weight slightly
- simplify motion
- use stronger face references

---

## Problem: Video Looks Melty

Possible causes:

- excessive motion
- overly aggressive denoise
- long clips

Possible fixes:

- shorter shots
- subtler movement
- lower denoise

---

## Problem: ComfyUI Keeps Crashing

Possible causes:

- VRAM exhaustion
- oversized workflows
- incompatible nodes
- dependency conflicts

Possible fixes:

- reduce resolution
- restart ComfyUI periodically
- simplify graphs
- validate environment compatibility

---

## Problem: Characters Blend Together

Possible causes:

- overlapping masks
- shared seeds
- weak regional isolation

Possible fixes:

- separate masks completely
- use independent seed families
- inject one character at a time

---

# WHAT PROFESSIONAL RESULTS ACTUALLY LOOK LIKE

Most successful local AI video workflows in 2026 use:

- short clips
- controlled movement
- heavy editing
- strong sound design
- image-to-video pipelines
- modular workflows
- iterative shot correction

Common production combinations include:

- FLUX keyframes
- WAN image-to-video
- RIFE interpolation
- Resolve editing
- Blender camera planning

rather than relying on a single model to generate entire finished sequences.

---

# FINAL RECOMMENDATIONS FOR STUDENTS

## Learn In This Order

1. Learn ComfyUI basics
2. Generate still images
3. Learn masking
4. Learn identity workflows
5. Learn image-to-video
6. Learn interpolation
7. Learn editing

Do NOT skip stages.

---

# BEST PRACTICES SUMMARY (MAY 2026)

## Recommended

- FLUX for still generation
- WAN image-to-video workflows
- modular graphs
- short clips
- subtle motion
- editorial control
- 480p starting workflows
- independent shot iteration

## Avoid

- giant workflows
- one-click movie generation
- long shots
- text-to-video first
- excessive motion
- too many custom nodes initially
- huge resolutions immediately

---

# OFFICIAL REFERENCES

## ComfyUI
https://github.com/comfyanonymous/ComfyUI

## ComfyUI WAN Documentation
https://docs.comfy.org/tutorials/video/wan/wan2_2

## WAN Official Repository
https://github.com/Wan-Video

## PuLID
https://github.com/ToTheBeginning/PuLID

## IPAdapter
https://github.com/tencent-ailab/IP-Adapter

## FLUX Models
https://huggingface.co/black-forest-labs

## PyTorch Installation Matrix
https://pytorch.org/get-started/locally/

## NVIDIA CUDA Documentation
https://developer.nvidia.com/cuda-zone

## DaVinci Resolve
https://www.blackmagicdesign.com/products/davinciresolve

## RIFE
https://github.com/hzwer/ECCV2022-RIFE

---

# FINAL THOUGHT

The strongest AI video creators in 2026 are usually not the people with the biggest prompts.

They are the people who:

- plan shots carefully
- control motion deliberately
- iterate patiently
- edit professionally
- build modular workflows
- understand cinematic language

AI generation is only one part of filmmaking.

Storytelling, pacing, composition, sound, and editing still matter most.