Repository navigation
fix: seek off the frame boundary, so the importer stops losing frames (#124) - #364
Merged
Merged
Conversation
…#124) #124 sat open as "the exact-duplicate dedupe looks slightly off", with no way to say which direction the threshold erred in - the reporter had a video, the repository did not. So this builds one: a second at 30fps where every frame is a different colour, which in `exact` mode can never legitimately be merged. Importing it at 30fps kept 21 of 30 and merged 9, in the giveaway pattern holds=[1,2,1,2,1,2,...]. At 15 and 6fps nothing merged, which is why it only ever looked *slightly* off. The signature was innocent. The merged captures really were byte-identical, because seeking to exactly a frame boundary is ambiguous - the frame ending at t touches t as much as the one starting there - and the browser may hand back the one before. The extractor steps by 1/fps, so at a rate near the video's own, every other seek returned the previous picture. Isolated to the single variable, waiting only for `seeked`: offset 0 21 of 30 distinct, 9 consecutive repeats offset 0.0001 30 of 30 distinct, 0 repeats offset 1/60 30 of 30 distinct, 0 repeats So seekTarget nudges a millisecond into the frame. That is far below any real interval - 8.3ms even at 120fps - and far above the float error in `start + i * step`. It is applied before the end clamp, which exists because seeking to the duration fires no `seeked` in some browsers, so the nudge can never walk back into that. Waiting longer was the obvious other theory and it is wrong: with the nudge, `seeked` alone is exact, and waiting on requestVideoFrameCallback instead cost 16x the time (133ms to 2123ms over 30 seeks) and fixed nothing. rVFC also never fires when a seek lands on the frame already showing, so without a timeout it hangs the import outright. The import also got faster - 2091ms to 252ms for the 30fps case - because it no longer spends full-resolution reads and byte compares on frames that only looked identical. scripts/video-dedupe-check.mjs keeps the reproduction, so the next report of this shape has a number instead of an impression. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #124.
The issue said "before changing anything, get a concrete case — the dedupe is a threshold, so 'looks off' could be either direction, and the two need opposite fixes." That was the right instruction and it was what had kept the issue open. So rather than guess, I built the case.
The reproduction
A one-second 30fps clip where every frame is a different colour (ffmpeg, lossless, so compression can never make two frames genuinely identical). In
exactmode nothing in it can legitimately be merged, soskippedmust be 0 at any import rate.Nine provably-distinct frames merged, in an alternating
1,2,1,2pattern. And only at a rate near the video's own — which is exactly why it read as "slightly" off.Cause — not the threshold, and not the signature
The merged captures really were byte-identical, so the dedupe was doing its job correctly on the pixels it was given. The fault was upstream: seeking to exactly a frame boundary is ambiguous. The frame ending at
ttouchestas much as the one starting there, and the browser may hand back the earlier one. The extractor steps by1/fps, so at a matching rate every other seek returned the previous picture.Isolated to that single variable, waiting only for
seeked:The theory that was wrong
My first guess was that
seekedfires before the new frame is presented, and the fix wasrequestVideoFrameCallback. It changed nothing — still 9 merges — and it cost 16× the time (133ms → 2123ms over 30 seeks). It is also unsafe on its own: rVFC never fires when a seek lands on the frame already showing, so waiting on it without a timeout hangs the import. Recording this because it is the plausible-looking fix someone reaches for first.With the nudge,
seekedalone is exact.Change
seekTargetnudges 1ms into the frame. Far below any real interval (8.3ms even at 120fps, so it cannot reach the next frame) and far above the float error instart + i * step. Applied before the end clamp, which exists because seeking to the duration fires noseekedin some browsers — so the nudge can never walk back into that hang.The import also got faster: 2091ms → 252ms at 30fps, because it no longer spends full-resolution reads and byte compares on frames that only looked identical.
Tests
test/canvas/canvasHelpers.test.js:a seek is nudged off the frame boundary it was asked forthe nudge is far smaller than a frame and far larger than float error— pins both ends of the size, since the value is only correct within a rangethe nudge can never push a seek past the point that hangs— the ordering against the end clampAnd
scripts/video-dedupe-check.mjskeeps the whole reproduction, exiting non-zero if any rate merges a frame. Needs ffmpeg and the dev server; not innpm run check, as it drives a browser.1300 tests pass;
npm run checkgreen.Also in here
vite.config.jsmoves the dev server to 5175 withstrictPort: true. 5173 is permanently held by another project on this machine, and the clash was silent rather than loud: that one listens on::1, this one on::,localhostresolves to::1first, so Vite printedhttp://localhost:5173/and the browser got the other app.strictPortturns the next such clash into a refusal to start — the one outcome that cannot be mistaken for working.🤖 Generated with Claude Code