Skip to content

Instantly share code, notes, and snippets.

@renezander030

renezander030/README.md

Last active June 12, 2026 15:39
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save renezander030/866bd85789c5902471f8f5fc86d09342 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save renezander030/866bd85789c5902471f8f5fc86d09342 to your computer and use it in GitHub Desktop.
Download ZIP
Driving CapCut / JianYing video drafts from an LLM agent (Claude, DeepSeek, GLM) with a zero-dep Node CLI
Raw
README.md

Driving CapCut / JianYing video drafts from an LLM agent (Claude, DeepSeek, GLM) with a zero-dep Node CLI

Driving CapCut / JianYing video drafts from an LLM agent (Claude, DeepSeek, GLM) with a zero-dep Node CLI

Production AI Automation Notes #4 — how an LLM agent edits a draft_content.json over JSON-in / JSON-out, no MCP server, no Python.

Last tested: May 2026 (capcut-cli v0.5.0, CapCut 9.x, JianYing 5.9.0). See Changelog at the bottom.

If this saves you setup time, follow @renezander030 for more production AI automation notes — agent approval gates, MCP server design, video pipelines.

Full repo with scripts: github.com/renezander030/capcut-cli

TL;DR cheat sheet

Agent goal capcut command What it writes in draft_content.json
Discover the project capcut info ./project reads id, name, duration, fps, canvas_config, track counts
List every subtitle capcut texts ./project walks tracks[].segments[] of type text, joined with materials.texts[]
Replace a subtitle capcut set-text ./project <segment-id> "New line" rewrites materials.texts[].content.text + adjusts the range byte offsets
Add a speech-bubble capcut bubble-text ./project <segment-id> --bubble heart sets effect/bubble_id on the text material
Fade audio capcut audio-fade ./project <segment-id> --in 0.5 --fade-out 1.0 appends to materials.audio_fades[]
Set blend mode capcut mix-mode ./project <segment-id> screen sets segment.mix_mode on the video segment
Set draft cover capcut add-cover ./project ./cover.png sets the draft's cover frame at time 0
Discover what's possible capcut enums --filters --jianying dumps the 468-slug filter catalogue for JianYing namespace
Run an agent loop capcut serve reads JSONL on stdin, emits JSONL on stdout stateless, one process per loop

JSON in, JSON out, exit code 0/1/2 for CI. Zero runtime dependencies (Node ≥ 18 stdlib only). Both CapCut and JianYing namespaces in one binary — --jianying toggles enums.

Recommended setup

One CLI + one model + one wrapper. No MCP server, no HTTP daemon, no Python venv.

# install
npm i -g capcut-cli

# verify
capcut version ./path/to/draft_content.json
# → {"app":"cc","app_version":"9.0.0","support":{"status":"tested"},
#    "schema":{"mask_field":"common_masks","has_text_ranges":true}}

That's it. The agent now has a deterministic surface: every command is JSON-in / JSON-out, every error has an exit code, every enum is discoverable from enums.

The 4-step pipeline

┌─────────────────┐   ┌────────────────────┐   ┌─────────────────┐   ┌────────────┐
│ 1. cut source   │ → │ 2. LLM emits JSON  │ → │ 3. shell runs   │ → │ 4. review  │
│ (capcut cut)    │   │ (Claude/DeepSeek)  │   │ capcut commands │   │ (capcut    │
│ → short.mp4     │   │ → array of edits   │   │ → draft updated │   │ info/lint) │
└─────────────────┘   └────────────────────┘   └─────────────────┘   └────────────┘

The agent never touches draft_content.json directly. It emits structured edit instructions. The shell wrapper validates them against the CLI's known command set, runs them, captures JSON output, feeds it back to the agent on the next turn.

This is the same pattern as agent-approval-gate (PAAN #1): the LLM proposes, the CLI executes, the schema gates the contract.

Why a CLI instead of an MCP server or HTTP API

Surface What it costs What it gives you
HTTP server (Flask, CapCutAPI, capcut-mate) Process to keep alive, port to manage, state to debug One canonical instance, hot in-memory cache
MCP server stdio/HTTP transport, tool schema, model-specific quirks Tighter integration with one ecosystem (Claude)
Python library (pyJianYingDraft, pyCapCut) Python venv, dependency resolution, agent has to wrap stdin/stdout itself Mature, well-documented, large community
Zero-dep CLI (this) One npm install Same agent can drive it whether it's Claude, DeepSeek, GLM, Kimi, or a shell script. No transport layer. Exit codes work. Every command pipes to jq.

The CLI bet: if your agent can write JSON and your shell can read it, you don't need a protocol on top. capcut serve is the only concession — a stateless JSONL stdin/stdout streamer for n8n / Coze / 扣子 / Make workflows. No port, no daemon.

The agent's discoverable surface

The hardest part of agent-driving any tool isn't the commands — it's the enums. CapCut and JianYing ship hundreds of slugs across transitions, filters, bubble shapes, sticker IDs, animation presets. They differ between platforms. They change between releases.

capcut enums is the discovery API:

# CapCut namespace defaults
capcut enums --filters
capcut enums --bubbles
capcut enums --transitions
capcut enums --animations
capcut enums --effects

# JianYing namespace
capcut enums --filters --jianying    # 468 slugs
capcut enums --bubbles --jianying

Feed the JSON output back into the agent's context. It now knows exactly which strings are legal for the next bubble-text or add-filter call. No guessing, no hallucinated slugs.

v0.5 — six new commands the agent now has

Shipped this week from Discussion #1 community vote:

  1. mix-mode <id> <mode> — blend mode on a video segment (normal · multiply · screen · overlay · soft-light · hard-light · color-dodge · color-burn · darken · lighten · difference · exclusion).
  2. audio-fade <id> --in 0.5 --fade-out 1.0 — real materials.audio_fades[] entries, not faked with volume keyframes.
  3. add-cover <project> ./cover.png --time 0 — set the draft's cover frame.
  4. add-filter <slug> <start> <duration> + enums --filters — colour-filter chain separate from add-effect. 10-slug starter for CapCut, 468 for JianYing.
  5. bubble-text <id> --bubble heart + enums --bubbles — 花字 / speech-bubble shapes. 7-slug starter, or --effect-id / --resource-id for your own IDs.
  6. import-ass <project> ./sub.ass — ASS / SSA subtitle import alongside import-srt. Same --track-name / --style-ref / --time-offset flag set.

Combined with v0.4: caption (whisper → real caption objects, not SRT-as-text), migrate (5.9 ↔ 9.6 mask ↔ common_masks), lint (schema-aware caption checks, exit code for CI), version (app detection + schema flags), translate (whole-draft clone via Anthropic API), serve (JSONL queue), add-sfx, chroma, export --batch. The agent now has a 30-command surface to compose against.

Smoke test — a real agent-style sequence

# 1. create a project from scratch
capcut init smoke-test
# → {"ok":true,"path":"~/Movies/CapCut/User Data/Projects/.../smoke-test/draft_content.json"}

# 2. add a video, a text overlay, an audio track
capcut add-video ./smoke-test ./clip.mp4 0 5000000
# → {"ok":true,"segment_id":"a1b2c3d4-...","track_name":"video"}

capcut add-text ./smoke-test 500000 2500000 "Welcome to the video"
# → {"ok":true,"segment_id":"e5f6a7b8-...","material_id":"...","track_name":"text"}

capcut add-audio ./smoke-test ./voiceover.wav 0 5000000
# → {"ok":true,"segment_id":"...","material_id":"...","track_name":"audio"}

# 3. apply v0.5 polish — bubble the text, fade the audio
capcut bubble-text ./smoke-test e5f6a7b8 --bubble heart
# → {"ok":true,"id":"e5f6a7b8-...","bubble":"heart","effect_id":"..."}

capcut audio-fade ./smoke-test <audio-segment-id> --in 0.5 --fade-out 1.0
# → {"ok":true,"id":"...","fade_in_ms":500,"fade_out_ms":1000}

# 4. lint before opening in CapCut
capcut lint ./smoke-test
# → {"errors":0,"warnings":0,"checks":["overlap","line-length","missing-files","missing-refs"]}
# → exit 0

# 5. agent inspects what it built
capcut info ./smoke-test -H
# human-readable summary table for the agent's context

Five commands, one shell loop, zero draft_content.json editing by hand.

Recipe — a deterministic shell wrapper

#!/usr/bin/env bash
# agent-edit.sh — wrap an LLM into a capcut-cli edit loop
set -euo pipefail

PROJECT="${1:?path to draft}"
PROMPT="${2:?what the agent should do}"

# 1. give the agent the current state + the legal enums
STATE="$(capcut info "$PROJECT")"
TEXTS="$(capcut texts "$PROJECT")"
BUBBLES="$(capcut enums --bubbles --jianying)"

# 2. ask the LLM for a plan as a JSON array of capcut commands
PLAN="$(claude -p --model opus <<EOF
You are editing a CapCut/JianYing draft via the capcut-cli.
Current state: $STATE
Current texts: $TEXTS
Legal bubbles: $BUBBLES
User goal: $PROMPT
Emit a JSON array of capcut command arrays, e.g.:
  [["set-text", "<id>", "New caption"], ["bubble-text", "<id>", "--bubble", "heart"]]
Only emit the JSON array. No prose.
EOF
)"

# 3. execute each command, capture exit codes
echo "$PLAN" | jq -c '.[]' | while read -r cmd; do
  ARGS=$(echo "$cmd" | jq -r '.[]')
  capcut $ARGS "$PROJECT" || { echo "FAIL: $cmd"; exit 1; }
done

# 4. lint + report
capcut lint "$PROJECT"

That's the entire agent loop. Substitute claude -p with deepseek-cli / glm-cli / kimi-cli — same shape. Anything that emits JSON works.

What doesn't work for agent-driving

  • Don't ask the agent to edit draft_content.json directly. It's deeply nested, microseconds-typed, with text buried inside escaped JSON-in-JSON inside materials.texts[].content. Even a 1M-context model will mis-edit it 1 call in 3.
  • Don't bundle 12 alternative tools and ask the agent to choose. Pick capcut-cli (or pyJianYingDraft, or CapCutAPI — whichever you standardise on) and put only that command surface in the prompt. The agent will compose well within one tool, badly across three.
  • Don't skip capcut version at the start of a session. JianYing 6.0+ encrypts the draft. If the agent tries to edit an encrypted file, it will fail in confusing ways. capcut version exits cleanly with support.status set — branch on that.
  • Don't trust hallucinated slugs. Always pass capcut enums output into the agent's context before asking for a bubble / filter / transition. The slug namespaces between CapCut and JianYing are not interchangeable.

Series

This is Production AI Automation Notes #4. The series covers gates, contracts, and deterministic interfaces for AI agents in production.

Follow @renezander030 for the next entry.

Sources

Reader contributions

Drop a comment with your setup:

  • Model (Claude Opus / Sonnet / DeepSeek-R1 / GLM-4.7 / Kimi-K2 / local Qwen)
  • App + version (CapCut 9.x / JianYing 5.9 / JianYing 10.x encrypted)
  • The command sequence you're driving
  • What worked / what the agent got wrong

If you've wired capcut serve into n8n / Coze / 扣子 / Make, a workflow JSON export is gold for the next reader.

Changelog

2026-05-25

  • Initial publication. capcut-cli v0.5.0 + the six commands from Discussion #1.
  • Hook map: this gist (curiosity-driven), capcut-cli repo (search-driven), v0.5 release notes (pain-driven) — three distinct hook types across the surface.

Series — full list (updated 2026-05-25)

The Production AI Automation Notes series has grown to 7 entries:

  1. Agent Approval Gates — proposed actions, schema validation, audit log
  2. Token Budgets — per-step, per-pipeline, per-day enforcement
  3. Agentic Task System — your task app as the agent's context layer, with pluggable storage adapters
  4. Driving CapCut / JianYing from an LLM agent — deterministic JSON command boundary
  5. SQLite dedup + crash safety — WAL mode, seen_items, audit log
  6. Prompt-injection defense — input sanitization, schema validation, deterministic boundary
  7. PDF cite verification — auditable LLM extraction with per-fragment bounding boxes

Reference implementation for entries #1, #2, #5, #6, #7: draftcat (Go, MIT).

Follow @renezander030 for new entries.

Independent project — not affiliated with or endorsed by ByteDance. "CapCut"/"JianYing" are trademarks of their owner; used nominatively.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment