Repository navigation
[Feature] The headset's frame cost, on the Swift side: the pyramid's gate, the sun, the look pass, the fences, the textures - #74
Closed
miogds wants to merge 9 commits into
Conversation
added 9 commits
October 15, 2026 13:06
…odel A model loaded from a file is one entity per node under the entity it was set on, so the viewer's single dartboard put many mesh entities into the culling set, and the pyramid's gate (minimumEntities, 2 by default) built the depth copy and the pyramid every frame, and ran the test for an object that hides nothing but itself: on the headset 1 to 2.5 ms of a 13 to 15 ms frame. The culling set now counts objects: an entity with its children, the snapshot's object(of:) being the entity's scenegraph parent or the entity itself. Both culling paths set hzbCandidateCount to that count; the gate, the setting and the accessor are documented in those terms. A render test puts three parts under one entity beside a box and sees two objects: the pyramid at a minimum of two, none at three, where four entities would have built it.
…draw The pass made its encoder, loading and storing the lit colour and the depth of the whole frame, before it looked for a transparent surface: with none, every eye paid the encoder, its bindings and three light uploads to draw nothing. The draw list is gathered first now and the pass returns when it is empty, as the wireframe and occluder-shell passes do. A render test reads the GPU's own pass labels: no "Transparency" with an opaque box, the pass once its material blends.
…ge, not their vertex stage Every scene pass waited on the shared fence before its vertex stage, after the previous pass updated it after its fragment stage: the tiler of one pass could not run while the renderer of the previous one shaded, and the Forward pass's binning of a model's many parts (3.5 ms of vertex work on the headset) sat behind the shadows' and culling's fragments. A pass's vertex stage reads nothing a previous pass's fragments wrote (meshes, quads, joint streams), and every resource is hazard-tracked, so the wait moves to the fragment stage at all 23 sites, as the temporal resolve already had it; every update stays as it was.
…ay compress them The lit colour (the MSAA resolve target), the environment, the composite, the look output and the anti-aliasing output were created with shaderWrite usage, which no kernel uses and which, per Apple's texture guidance, forbids the GPU's lossless compression of a texture: every pass that resolved into, read or wrote them moved uncompressed pixels. They keep their shared storage, so the tests' CPU reads stay as they were.
The look pass drew through the post-process descriptor, which carries the frame's depth loaded and stored for the passes that read it as an attachment; the look pass never tests or writes depth, so each eye loaded and stored a full-size depth texture for nothing. It has its own descriptor now, colour alone, its pipeline built with no depth attachment; executePostProcess takes the descriptor to draw into and sets the depth load only when there is one. The other post-process passes keep the frame's depth. A render test checks the descriptor and that the look output still carries the lit frame.
The viewer's sun switch leaves the directional light in place at intensity 0 with castsShadow on, and the cascades were drawn every frame regardless: two 2048x2048 depth maps of every part of the board, 0.5 to 2.9 ms a frame on the headset with no sun. The shading multiplies the sun's shadow by its intensity, so the cascade update now refuses a sun at zero and the shadow passes stand down; the picture is the same. A render test dims the scene's sun to zero and sees no "Shadows" pass, then lights it again.
…um, and the frame's own GPU time The per-second line named only the three heaviest passes, so the device's numbers could not be read against the frame's span: the shadows, the look, the culling and the transparency passes were timed and never shown. A second well past the deadline now lists every pass, heaviest first, with their sum (a label sums both eyes); the rest of the span lies between passes. The GPU figure adds the frame's own time after the frame before finished: a queued frame's command buffer is taken before the previous one is done, so the raw span overstates the frame's work.
…at held, what remains in the shaders
…set starts without one The renderer's setup made a camera and a directional light for every path, the headset included: a sun at intensity 1 casting cascades in a passthrough room that has its own light, until the app replaced it. The defaults are one function now (createDefaultEntities), the camera on every path, since the headset writes the head pose into it, and the sun by UntoldRendererConfig.defaultSun: unset, the platform decides, a sun everywhere but the headset, where the passthrough room is lit by its probe and the immersive environment by its own light; a project sets it to keep a sun on the headset (for shadows its environment cannot cast) or to start without one elsewhere. A render test covers both cases, and the XR architecture doc's comparison table has the row.
Member
Author
|
Closed without merging here: the headset's frame cost on the Swift side (feature/xr_frame_cost at f82664c) is on develop since #76 (merge 0b6348e of feature/xr_shader_cost at eac4e3e, which carries #73–#75), and develop now also holds the head of untoldengine#1359 through #78 and #79. The branch stays for the upstream pull request once untoldengine#1359 lands. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on
feature/presentation_time_animation(#73): the headset's frame cost, on the Swift side. Nine commits.Why
With the frame posed for its presentation time and paced evenly, the viewer's dartboard still stepped on Apple Vision Pro in the heavy configuration (MSAA 4× with TAA, the occlusion cull, foveated 2048×1984 per eye): the GPU took 13 to 15 ms a frame against an 11.1 ms refresh, so the display could show every other frame at best. The per-pass timing on the device named the costs, and each cut was refuted against the code before it was taken (five readers, three verifiers per finding).
What changes
castsShadowon, and the cascades were drawn regardless: 0.5 to 2.9 ms a frame with no sun. The shading multiplies the shadow by the intensity, so the cascade update refuses a sun at zero.UntoldRendererConfig.defaultSun): unset, a sun on every path but the headset, whose passthrough room is lit by its probe and whose immersive environment by its own light. The camera stays on every path; the headset writes the head pose into it.shaderWrite, which no kernel uses and which forbids the GPU's lossless compression per Apple's texture guidance; storage stays shared for the tests' CPU reads.On the device
The heavy configuration, the sun off: 17 ms a frame paced to half rate before; after, 6.8 to 9 ms, full rate at 90 and at 100 Hz, no refresh skipped after the placement, the pacing never engaged. Forward 10.40 → 8.02 ms (both eyes), TAA 2.50 → 2.06, Post 0.76 → 0.45, the shadows and the transparency pass gone, the cull 0.14 → 0.01.
Tests
Render tests for the pyramid's gate (a model's parts count as one object), the shadow gate (a sun at zero draws no cascades), the look pass's descriptor, the transparency gate and the default entities, by the GPU's own pass labels where a pass is the question.
docs/proposals/PresentationTimeAnimation.mdrecords the measurements and the reviewers' findings; the XR architecture doc's comparison table has the default-scene row.