Playwright Video → LLM Filmstrip Pipeline: Turn Playwright recordings into frame-by-frame LLM design reviews

playwright-video-filmstrip.md

Playwright Video → LLM Filmstrip Pipeline

Step 1: Record video

// In your test file or playwright.config.ts
test.use({ video: 'on' });

test('your UI flow', async ({ page }) => {
  // Walk through the flow — every frame is captured
  await page.goto('/your-page');
  await page.getByRole('button', { name: /submit/i }).click();
  await page.waitForTimeout(3000); // Let async transitions complete
});

Step 2: Extract frames on scene change

VIDEO=$(find test-results -name "*.webm" -type f | xargs ls -t | head -1)
mkdir -p /tmp/filmstrip

# Coarse pass — meaningful UI changes only
ffmpeg -i "$VIDEO" -vf "select='gt(scene,0.01)'" -fps_mode vfr /tmp/filmstrip/frame_%04d.jpg

# Detail pass — 30fps around a specific transition (adjust -ss/-to)
ffmpeg -i "$VIDEO" -ss 14.5 -to 16.5 -vf "fps=30" /tmp/filmstrip-detail/frame_%04d.jpg

Step 3: Build manifest

ffmpeg -i "$VIDEO" -vf "select='gt(scene,0.01)',showinfo" -fps_mode vfr -f null - 2>&1 \
  | grep "pts_time" | sed 's/.*pts_time:\([0-9.]*\).*/\1/' \
  | awk '{printf "{\"frame\": \"frame_%04d.jpg\", \"ts\": %s}\n", NR, $1}' \
  > /tmp/filmstrip/manifest.jsonl

Step 4: Feed to LLM

Read the JPEG frames sequentially with the manifest timestamps. Ask the LLM to identify:

• State transition flickers (intermediate states flashing between expected states)
• Missing loading indicators
• Layout shifts during async operations
• Progress bars that don't update

Threshold tuning

Threshold	Sensitivity	Use case
0.005	Very high	Spinners, subtle color shifts
0.01	Default	Meaningful UI state changes
0.05	Low	Page navigations only