The Harness
How an LLM Understands Video
video-use / SKILL.md
Don't show pixels. Show a text transcript from audio + visual composites on demand. The LLM never watches the video. It reads it.
What the LLM actually sees
timeline_view.py — C0103.MP4 — 42.0s → 48.0s
ON-DEMAND COMPOSITE PNG
SPEAKER 0 SPEAKER 1 Ninety percent of what ...wasted. We fixed this. When a web agent comes... 620ms silence 880ms silence (laughter) 42.00s 43.20s 44.40s 45.60s 46.80s 48.00s CUT CUT Silences ≥ 400ms = safe cut candidates. Words give boundary precision. Audio events mark beats to preserve.
How it gets there
Transcribe Scribe, parallel Pack phrase-level .md LLM Reasons reads transcript calls timeline_view at decision points EDL .json Render ffmpeg pipeline Self-Eval timeline_view on output at every cut point issue? fix → re-render (max 3)
Audio transcript is the primary surface. • Visual composites are on-demand at decision points. • Verify output before presenting.