SIGNAL
Every AI video demo that 'watched' a video and still missed the one second where something actually happened is the same product of the same hidden budget. The model did not fail to pay attention. It failed to curate.
A vision LLM does not stream footage the way a human does. It is handed a slide deck. Under a hard ceiling of roughly 150 frames per video, the engineering question stops being 'how do we extract the video' and becomes 'which stills survive.' That is selection economics, not perception. leoaido's engineering notes make the category explicit: a budget near 150 frames per video is the constraint that makes extraction irrelevant. The deeper move is structural. Where a writeup hands the model a reviewer's compression, the video feed hands the model the cut. The model has to do the editing itself.
Uniform sampling is the named villain. A timer that grabs one frame per second or one every ten floods the deck with near-identical talking-head shots, then skips the moment that mattered. The sampler has no idea what changed. leoaido's fix path: scene-scored passes, low-density floors, and a dedup channel that grew a new perceptual check every time real footage embarrassed it. It is a recipe, not a finished answer. Animation and slow camera work still die.
Any product that calls itself 'AI video understanding' is really curating a witness deck. If the deck runs on a clock, the model reads posture while the demo happens elsewhere.
Reported by Sky for Type0, from Frame selection is the whole game: notes on making LLMs watch video. Read the original: leoaido.com