Skip to content

fix: seek off the frame boundary, so the importer stops losing frames (#124) - #364

Merged
difficcd merged 1 commit into
mainfrom
fix/seek-off-the-frame-boundary
Oct 6, 2026
Merged

difficcd merged 1 commit into
mainfrom
fix/seek-off-the-frame-boundary

Conversation

@difficcd

@difficcd difficcd commented Oct 6, 2026

Copy link
Copy Markdown
Owner

Closes #124.

The issue said "before changing anything, get a concrete case — the dedupe is a threshold, so 'looks off' could be either direction, and the two need opposite fixes." That was the right instruction and it was what had kept the issue open. So rather than guess, I built the case.

The reproduction

A one-second 30fps clip where every frame is a different colour (ffmpeg, lossless, so compression can never make two frames genuinely identical). In exact mode nothing in it can legitimately be merged, so skipped must be 0 at any import rate.

BAD  fps=30  kept=21  skipped=9  2091ms  holds=[1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,1,1,1,2]
ok   fps=15  kept=15  skipped=0   894ms
ok   fps= 6  kept= 6  skipped=0   455ms

Nine provably-distinct frames merged, in an alternating 1,2,1,2 pattern. And only at a rate near the video's own — which is exactly why it read as "slightly" off.

Cause — not the threshold, and not the signature

The merged captures really were byte-identical, so the dedupe was doing its job correctly on the pixels it was given. The fault was upstream: seeking to exactly a frame boundary is ambiguous. The frame ending at t touches t as much as the one starting there, and the browser may hand back the earlier one. The extractor steps by 1/fps, so at a matching rate every other seek returned the previous picture.

Isolated to that single variable, waiting only for seeked:

seek offset distinct frames consecutive repeats
0 (on the boundary) 21/30 9
0.0001 30/30 0
1/60 (half a frame) 30/30 0

The theory that was wrong

My first guess was that seeked fires before the new frame is presented, and the fix was requestVideoFrameCallback. It changed nothing — still 9 merges — and it cost 16× the time (133ms → 2123ms over 30 seeks). It is also unsafe on its own: rVFC never fires when a seek lands on the frame already showing, so waiting on it without a timeout hangs the import. Recording this because it is the plausible-looking fix someone reaches for first.

With the nudge, seeked alone is exact.

Change

seekTarget nudges 1ms into the frame. Far below any real interval (8.3ms even at 120fps, so it cannot reach the next frame) and far above the float error in start + i * step. Applied before the end clamp, which exists because seeking to the duration fires no seeked in some browsers — so the nudge can never walk back into that hang.

The import also got faster: 2091ms → 252ms at 30fps, because it no longer spends full-resolution reads and byte compares on frames that only looked identical.

Tests

test/canvas/canvasHelpers.test.js:

  • a seek is nudged off the frame boundary it was asked for
  • the nudge is far smaller than a frame and far larger than float error — pins both ends of the size, since the value is only correct within a range
  • the nudge can never push a seek past the point that hangs — the ordering against the end clamp

And scripts/video-dedupe-check.mjs keeps the whole reproduction, exiting non-zero if any rate merges a frame. Needs ffmpeg and the dev server; not in npm run check, as it drives a browser.

1300 tests pass; npm run check green.

Also in here

vite.config.js moves the dev server to 5175 with strictPort: true. 5173 is permanently held by another project on this machine, and the clash was silent rather than loud: that one listens on ::1, this one on ::, localhost resolves to ::1 first, so Vite printed http://localhost:5173/ and the browser got the other app. strictPort turns the next such clash into a refusal to start — the one outcome that cannot be mistaken for working.

🤖 Generated with Claude Code

…#124)

#124 sat open as "the exact-duplicate dedupe looks slightly off", with no way to say which
direction the threshold erred in - the reporter had a video, the repository did not. So this
builds one: a second at 30fps where every frame is a different colour, which in `exact` mode
can never legitimately be merged.

Importing it at 30fps kept 21 of 30 and merged 9, in the giveaway pattern
holds=[1,2,1,2,1,2,...]. At 15 and 6fps nothing merged, which is why it only ever looked
*slightly* off.

The signature was innocent. The merged captures really were byte-identical, because seeking
to exactly a frame boundary is ambiguous - the frame ending at t touches t as much as the one
starting there - and the browser may hand back the one before. The extractor steps by 1/fps,
so at a rate near the video's own, every other seek returned the previous picture. Isolated
to the single variable, waiting only for `seeked`:

    offset 0        21 of 30 distinct, 9 consecutive repeats
    offset 0.0001   30 of 30 distinct, 0 repeats
    offset 1/60     30 of 30 distinct, 0 repeats

So seekTarget nudges a millisecond into the frame. That is far below any real interval -
8.3ms even at 120fps - and far above the float error in `start + i * step`. It is applied
before the end clamp, which exists because seeking to the duration fires no `seeked` in some
browsers, so the nudge can never walk back into that.

Waiting longer was the obvious other theory and it is wrong: with the nudge, `seeked` alone
is exact, and waiting on requestVideoFrameCallback instead cost 16x the time (133ms to
2123ms over 30 seeks) and fixed nothing. rVFC also never fires when a seek lands on the frame
already showing, so without a timeout it hangs the import outright.

The import also got faster - 2091ms to 252ms for the 30fps case - because it no longer spends
full-resolution reads and byte compares on frames that only looked identical.

scripts/video-dedupe-check.mjs keeps the reproduction, so the next report of this shape has a
number instead of an impression.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
easy-mv-maker Ready Ready Preview Oct 6, 2026 10:52am UTC

@difficcd
difficcd merged commit 8a807dd into main Oct 6, 2026
6 checks passed
@difficcd
difficcd deleted the fix/seek-off-the-frame-boundary branch October 6, 2026 10:56

This branch was successfully deployed

1 active deployment
Preview — be73c260 Deployed Oct 6, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exact-duplicate frame merging looks slightly off

1 participant