Repository navigation
perf(core): the preview mounts and draws only the scenes near the playhead - #6189
miguel-heygen wants to merge 48 commits into
Conversation
Edit accuracy: accurate 271 (base branch 271), smooth 133 of thoseThe gate passes. Quarantined, measured but not gated (0) Unstable (1)
|
…ount one per turn
…w that unmounts scenes
…ked scene keeps its styles
…caches or old roots
…markup it mounts with
miguel-heygen
changed the title
…ones slowed each mount
…n font loads can unmount
…ted scene keeps showing
…r scene unmounting
…kip-render # Conflicts: # packages/core/src/runtime/timedClipHide.ts
… after a font wait
…kip-render # Conflicts: # scripts/package-size-budgets.json
…, no forced layout at the cut
…kip-render # Conflicts: # scripts/package-size-budgets.json
…kip-render # Conflicts: # packages/core/src/runtime/webAudioTransport.test.ts # scripts/package-size-budgets.json
…preview runtime inlined
![heygengenesis[bot]](https://avatars.githubusercontent.com/in/2913576?s=60&v=4){kind=small}
There was a problem hiding this comment.
Verdict: REQUEST CHANGES · 1 blocker · 1 important · 0 nits · head e1ab276a
The bar for this PR is play and scrub fps above main, and no scene cut more than 33 ms slower than main. Fps passes. The cut part fails on the body's own data:
- blocker — cuts on the 50-second film are well over 33 ms slower than main. In the Scene cuts on the 50-second film table (taken at 7752323; the far-scene and reveal code has not changed since), the median cut frame is 222 vs 102 ms at 23.17 s (+120), 205 vs 88 ms at 30.5 s (+117) and 136 vs about 100 ms at 45.0 s (+36). The longest task is still +49 / +36 ms at 23.17 s and 30.5 s. The newer 23-minute film at this head cut table doesn't make up for this: it times the 50-second film's cut times (17.6, 23.17, 30.5, 30.7, 40, 45 s) on the 23-minute film, and the body says only one of those is a cut there. That film's real cuts, such as 630 s / 633.9 s, aren't measured. To clear this, either get those cuts under +33 ms here, or measure every cut at this head and show each one within +33 ms. If the plan is to accept the regression and fix it in the follow-up PR, that call belongs to whoever set the bar, not to this review.
- The after-a-seek fps rows still show "third run running". Please fill them in before re-review.
CI was still running at review time (CI, regression, Windows render, CodeQL, Player perf); I haven't judged it.
Scope
Read end to end: runtime/init.ts (the window, mount/retire, seek, audio horizon and built-root changes), sceneWindow.ts, farScenes.ts, firstRenderReveal.ts, webAudioTransport.ts, timeline.ts, timedClipHide.ts. Checked how far I could trust them: the bundler gate, Studio sceneHold and hydration, and the tests. Confirmed the change since c17c3df2 is comments only.
| const landed = windowScenes.find( | ||
| ({ name, start, end }) => start <= time && time <= end && parked(name), | ||
| ); | ||
| if (windowWork) return landed !== undefined; |
There was a problem hiding this comment.
important — a seek can report done before its scene mounts. If a seek lands in a parked scene while another mount is already running, this line returns true and registers nothing with registerSeekCompletion. That happens whenever a neighbour mount is in flight, which during play is every scene 10 s ahead, and it takes about 165 ms. The finally only registers the landing later, so a __hfWaitForSeekCompletion call made in between returns while the old frame is still on screen. That breaks the body's promise that "a seek is not done until the scene it lands in is mounted". Fix: on this path, when landed is set, also register a promise that resolves once that scene has mounted, for example windowWork.then(() => <the landed scene's mount>). Add a test that seeks into scene B while scene A is mounting.
What
The Studio preview of a long film now keeps only the scenes near the playhead mounted, skips drawing the near ones it is not showing, and schedules audio just ahead of the playhead. Renders and exports are unchanged: everything here acts only on preview bundles, whose scene hosts carry
data-hf-scene.Far scenes skip rendering. Mounted scenes beyond the playing scene and one neighbour on each side get
content-visibility: hidden(runtime/farScenes.ts), with their measured size pinned throughcontain-intrinsic-sizeso scripts that measure them read the same box. Only out-of-flow hosts are skipped: layout containment would move the children and siblings of an in-flow host. A scene's size and position come from its last render, reported by aResizeObserverafter layout, so the frame where a scene leaves reads nothing from it. Before,measureLeavinginfarScenes.tscalledgetComputedStyle(host).position,offsetWidthandoffsetHeighton the leaving host while the cut had the page's style dirty, which forced a full style pass inside the cut frame.The first render reveals only what it renders. A first-time seek used to show every hidden clip of the film for one GSAP render. It now shows only the clips holding targets of tweens that render for the first time in the jump (
runtime/firstRenderReveal.ts).Only scenes near the playhead are mounted. At boot, before any scene script runs, each scene that can be swapped is kept as markup and the ones away from the start are replaced by an empty copy of their host that keeps its timing (
runtime/sceneWindow.ts). The window follows play, seek and scrub:__hfWaitForSeekCompletion) until the scene it lands in is mounted; other scenes mount one per turn after it, so a later seek overtakes them;A scene parks only when it can run again cleanly: it passes the swap gate, and every media element and nested composition in it has an authored
data-duration(otherwise its timeline length would depend on loaded metadata or a timeline that only exists while mounted). The swap gate now also allows a scene to:document.fonts.readyordocument.fonts.load("…")with literal font strings;document.createElement("canvas").getContext("2d")in one expression, with nothing reading.canvas); every othergetContextstays refused, so a scene that draws on a canvas stays mounted;animate(t)method (the browser'sElement.animate(keyframes, timing)stays refused);gsap.timeline(…)right where it registers it in__timelines[…](a chain that is never registered stays refused: that rule exists because a swap only stops registered timelines);Everything else the gate refused stays refused. At run time, a scene that also published its timeline under another scene's id is never unmounted, because unmounting would leave a stopped timeline registered there.
Audio follows the window. Clips are scheduled through Web Audio only when they start within 10 s of the playhead, and the tick schedules the rest as it comes near; clips that already ended are skipped. A scene mounting during play joins the mix without stopping or re-seeking what already plays. A source that a play and a seek both ask for is fetched and decoded once. Decoded audio of a scene that leaves the window is released.
A scene can mount on its own and shows once mounted. A helper script that several scenes load now runs once for the film instead of only with the first scene that loaded it, so any scene can mount without it. A host whose stand-in the visibility pass hid with
display: noneis shown again when it mounts.A play frame that draws nothing new skips the hidden-clip lookup. The first-render reveal already knows which hidden clips hold a tween drawing for the first time; when there are none, the page-wide
[data-hf-clip-hidden]query is skipped. On a synthetic 20,000-element film of plain timed clips (no unmounting), 10 s of play on the software-rendered machine at 8351fe3, 3 runs each: the query ran 600-601 times and cost 0.068-0.079 ms per frame before, 6 times and 0.001 ms per frame after; main-thread script time per frame went from 0.26-0.29 ms to 0.17-0.24 ms.An unmounted scene is freed. Six references kept the whole tree of every unmounted scene alive: a captured
<audio>(Chrome keeps it for the audio context's lifetime, and it held its scene through its parent), a cache of composition hosts, the clip tree while posts are held during play, the player's first root timeline, root timelines the runtime built and replaced, which stayed on GSAP's global timeline, and timelines a scene's script builds after a font wait, which stayed there too. The script's recorder only sees what it starts before returning, so a mount now also records what starts on GSAP's global timeline until fonts settle (the gate allows no other wait), and stopping a scene also stops the timelines its own timelines drive (a tween of a timeline's time, astweenFromTomakes), drivers first: a driver's revert sets the driven timeline's time, which would put it back on the global timeline.Measurements
Synthetic 45-scene, 22.5-minute film built in code: 290,868 elements (about 6.4k SVG elements per scene), 736
<audio>clips (466 at the root, 270 in scenes), a GSAP timeline per scene built by a project helper script, and a font-ready wait in each scene. Served as the Studio preview bundle. Base ismainat the merge base; each row is the median of 3 fresh headless Chrome 152 runs per variant, alternating variants (the 10-minute play is one run each).Protocols:
Software-rendered machine (4 CPUs of a shared Linux box, no GPU: SwiftShader; 1-minute load 5-16 during the runs). "Earlier head" is this PR before the window change (4283542); "this PR" is 17a68b1, before the gate and remount changes above.
Against the earlier head, 17a68b1 is better on the after-seek and warm worst play frames, every scrub row, every seek row and memory. It is worse on the open play's worst frame (100 against 50 ms, one long task) and the 10-minute play's worst frame (one 1,250 ms frame in a single run, while the box's load rose from 6.7 to 11). On main the 10-minute play there reached only 39 s of film.
GPU machine (Apple M5 Pro laptop, Chrome's Metal backend), measured on the earlier head 4283542; GPU rows for this head are not measured yet.
<audio>/<svg>mountedReal films
Two real projects, served as the Studio preview bundle from a copy, on the software-rendered machine above. Base is current
main(683e9b1); this PR is its head (1da8643). Each row is the median of 3 fresh headless Chrome 152 runs per variant, alternating variants; the 5-minute play is one run each. The machine was shared and loaded during these runs; each table gives the 1-minute load range.A 23-minute film: 45 scenes, 289,512 elements, 573
<audio>clips, GSAP from a CDN. Main refuses all 45 scenes; this PR refuses none and unmounts 43 at open. Load 2.2-12.8.Play does not reach 30 fps on either side; an earlier 6 s trace of play after a seek put this PR's limit in software rasterization of the film's SVG artwork, not script. Not improved: the seek to 700 s right after open or after a seek, which mounts the scene it lands in (252 → 438 ms, 99 → 550 ms), and the warm worst scrub frame (217 → 233 ms).
A 50-second 3840x2160 film: 7 scenes, each drawing 2D particle canvases from its own GSAP timeline, GSAP from a file in the project. Main and this PR both refuse all 7 scenes ("its script uses getContext"), so none unmounts: what changes here is far scenes skipping rendering and the first render revealing only what it renders. Measured through a 1920x1080 window onto the stage, with the protocol's times scaled to the film's length (scrub from 15.0 s, seeks to 25.4 s and 47.1 s, play across the cut at 23.2 s); the 5-minute play loops the film. Load 3.1-16.9.
Warm play is the same on both (32 fps). Not improved: the seek to 25.4 s right after open or after a seek (158 → 317 ms, 164 → 526 ms), the warm worst play and scrub frames, and renderer memory after the protocol.
Memory over long runs. Each sample follows a forced full garbage collection; both variants ran side by side. The JS heap, DOM nodes and live canvases, media elements and decoded audio buffers stay flat on the 50-second film; the renderer's own memory (raster, decoded images and the 54 canvases' backing stores at 3840x2160) follows what is on screen: in looping play it peaks near the same point of the film each time (about 17-19 s, 1.0-1.3 GB on both in the 5-minute runs) and falls after it, and the seek loops level off after about 10 rounds. So the rise on that film is bounded and the same on main; it is not a leak.
On the 23-minute film this PR's DOM count moves between 131k and 223k from sample to sample with no trend over the 30 rounds: a sample can land while the window still mounts a neighbour. Before the leak fix in item 7, the same loop grew by about 125k nodes and 10 MB of JS heap per round.
Frames after unmounting and mounting again. 36 frames per film, 12 per mode: play past scenes as they unmount, then sample back inside them; seek in, far away and back; scrub across with one seek per frame, then sample inside. Each sample is its own seek, taken once two screenshots in a row agree, with software compositing (as the renderer uses, because SwiftShader's GPU compositor re-presents stale raster after a seek). Main against itself, same sequence twice (683e9b1): 36 of 36 identical on both films.
Scene cuts on the 50-second film
Each time below is a scene start in this film. The frame that crosses each cut, during play, through the film copy with its own
[style*="visibility: hidden"] *rule removed. Same method on both sides: play from about 0.5 s before the cut, the first frame whose film time passes the cut is stamped, and its main-thread task is read from a trace. Software-rendered machine, 3 runs each, alternating, load 2.4-13.0. Main is 89af290; this PR is 7752323 (its runtime is the same at the current head).The forced read named above is gone: against the PR before it (f91ddc7, main 683e9b1 at the time, 3 runs), the forced style pass inside the cut frame went from 32-52 ms to none at 17.46 s, from 145-176 ms to 9 ms at 23.17 s (what remains is the visibility pass's in-flow position read) and from 80-104 ms to none at 30.5 s. The cut frames did not get shorter: the same restyle now runs in the frame's own render. Three cuts are still slower than main; see Known limits.
The single cut frame is a noisy number on both sides: main's measured 88 to 1,466 ms at 23.17 s over 11 runs on one day. Counting the whole second around each cut instead (every main-thread task that ends within 0.5 s of the stamped frame; blocking time is each task's time over 50 ms, summed), main 89af290 against this PR at defa68e, 3 runs each, alternating, load 8-20 on 8 cores, medians:
Around every cut this PR blocks less in total: main runs several tasks of about 100 ms each there, this PR fewer. Its longest single task is still longer than main's at 23.17 s and 30.5 s.
The 23-minute film at this head
Main is 26e3045; this PR is be9a987 (this head's runtime is the same; what came after is a merge of a test-only main commit and comment edits). Software-rendered machine (shared Linux box, 8 cores, SwiftShader), 3 runs each, alternating main and this PR.
Play and scrub. Open: play 6 s from the start right after the page is ready, then scrub ahead from 700 s at 2 s of film per frame for 5 s. After a seek: the same, after a seek to 4 s before the cut at 633.9 s. Warm: seek 2 s past that cut and settle, scrub ahead from 300 s, then play from 4 s before the cut. Load 7.6-17.9.
The open play run at 8.1 fps ran at the highest load of the six (10.3 at its start, 12.5 at its end).
Frame at the 50-second film's cut times, on the 23-minute film. The main-thread task of the first frame whose film time passes each time, during play from 0.5 s before it. These are not cuts in this film: its only scene cut in this range is at 23.0 s, 0.17 s before the 23.17 s frame, and the other times fall inside scenes. Load 5.5-19.3.
At 17.6 s this PR's frames are its ordinary play frames there (52-91 ms each); main's frames nearby include tasks of 170-255 ms.
Package size
This PR adds about 32 KB to
@hyperframes/core's built files (its runtime bundle grows by 9 KB, 1.5%, and ships in several formats) and about 8 KB to@hyperframes/producer's packed tarball, which carries that runtime. With main already close to both budgets, the merged head at e2bbe2d measured@hyperframes/core4,017,248 bytes unpacked, over by more than 1%; that budget is raised to 5% above it, the headroom the size check uses. Main has since lowered@hyperframes/producer's budgets; at be9a987 CI measured it at 782,289 bytes packed and 2,841,105 unpacked, over them by 1.5% and 1.1%, and both are set to those sizes.Known limits
[data-hf-clip-hidden][style*="visibility: hidden"] *rule, and 15-110 ms with that rule removed. perf(core): Safari stops re-serializing inline styles on every GSAP write #5974, merged in from main, replaces that rule with an attribute-only one; the mount cost after it is not measured separately (a 420 ms timer task remained in a trace of play across the cut at e2bbe2d).[style*="visibility: hidden"] *rules. On the 50-second film's cut at 23.2 s with the film's own rule, the cut frame's task took 351 ms at e2bbe2d (one trace), almost all of it a 279 ms restyle of the 8,647 elements whose visibility the cut switches; 190-200 ms with the film's own rule removed at 8351fe3.data-duration.position: fixedelement inside a scene skipped from rendering measures against its host.Tests
Each new test fails with its part of the change removed; checked one by one.
farScenes.test.ts,coreRuntimeBrowser.test.ts(Chrome): far-scene marks follow play, cut, seek and scrub; measured boxes match with the skip on and off; render bundles get no marks.farScenes.test.ts: a leaving scene's size and position come from its last render, with nogetComputedStyleoroffsetWidthread in the cut frame (fails on the earlier code with the read).firstRenderReveal.test.ts,init.test.ts: a first render reveals only the clips its tweens reach.htmlBundler.test.ts: the gate allows a project helper, the font-ready wait and named font loads, a canvas made only to measure text, a timeline chained where it is registered, a scene's ownanimate(t)and a GSAP file copied into the project; it still refuses timers, fetch, other awaits and promises, a named canvas or one handed back through.canvas, drawing on the stage's canvas, WebGL, WebGPU,bitmaprendererand computed context types, chains that are never registered or are set aside first, bare GSAP tweens andElement.animatewith timing. A helper that scenes share runs once for the film.init.swapScenes.test.ts: far scenes park at boot without running their scripts; a seek waits for the scene it lands in; the last frame holds while it mounts, also when a frame draws between the mount's tasks; playing, the next scene mounts once its cut is within the lead and not at the cut before it; Studio holds scenes; the timeline keeps parked clips; edits to parked scenes (markup and styles) reach the next mount and unparkable edits are refused; parked scenes still mount after an edit to a mounted one; scenes whose clip lengths come from media stay mounted; a scene mounting during play joins the mix without a stop; decoded audio of a parked scene is released; Studio gets a parked scene's markup.init.test.ts: Web Audio schedules a clip only as the playhead nears it, and none that ended; the hidden-clip lookup runs only on a jump where some tween draws for the first time.init.swapScenes.test.ts: a mounting scene records what it starts after its font wait as its own, never the runtime's root timeline (which a rebuild can put back on GSAP's global timeline meanwhile); a Studio edit swaps a scene whose timeline drives one its script built later, and stops that one too (refused before: "an animation outside it moves its elements"); a scene whose timeline is also published under another id stays mounted; a mounted scene whose stand-in the visibility pass hid by display shows, also after it unmounts and mounts again.coreRuntimeBrowser.test.ts(Chrome): a scene that draws on a 2D canvas is refused and never unmounts, its canvas unchanged after the window moves past it; a scene whose timeline is chained where it is registered unmounts and mounts again with the same transform.webAudioTransport.test.ts: one fetch and decode per source; sources of media gone from the page stop without going to the destination.coreRuntimeBrowser.test.ts(Chrome, GSAP, Web Audio running): after five seeks during play, the scenes the window mounted and left are freed, also when a scene's timeline drives one its script built after a font wait, and when its script builds one after a font wait to pose itself once (2,811 to 4,415 DOM nodes kept before the fix, under 800 after); each of the first five retainers, restored alone, keeps them.sceneHold.test.ts,useDomSelectionSelectionGuards.test.ts(selection waits on a mount only when the preview has one),timelineSyncHydration.test.ts(rows of an unmounted scene).@hyperframes/coresuite (5,293 passed, 1 skipped) and the Chrome contract file (77 passed), three runs each, and the core typecheck. The Chrome scene-memory test, which failed 3 of 10 runs under load before the last fix, passed 15 of 15 at load 11. The Studio suite, lint, format and the fallow audit run in CI.Before
Main: playing across the cut at 630 s after a seek, GPU machine. The overlay is drawn by the page and counts frames per second.
before.mp4
After
This PR at the earlier head 4283542, same run.
after.mp4