Skip to content

[Feature] The headset's Forward pass and temporal resolve do less per pixel, and the sun casts in one cascade - #75

Closed
miogds wants to merge 4 commits into
feature/xr_frame_costfrom
feature/xr_shader_cost
Closed

miogds wants to merge 4 commits into
feature/xr_frame_costfrom
feature/xr_shader_cost

Conversation

@miogds

@miogds miogds commented Oct 15, 2026

Copy link
Copy Markdown
Member

Stacked on feature/xr_frame_cost: the headset's Forward pass and temporal resolve do less per pixel, in the shaders, with the kernel libraries rebuilt (buildkernels.sh, every platform); and the sun made cheap for the projects that switch it on. Four commits.

Why

After the Swift-side cuts the heavy configuration held 90 Hz with 8 ms of GPU a frame, 5.5 of it the Forward pass's fragment work. The viewer switches its sun off, half its materials lack a map, the room probe is never rotated, and most of a passthrough frame is cleared; the shaders paid for all four regardless. With the sun on, the frame no longer fitted at 100 Hz.

What changes

  • The sun by its intensity. The cascade shadow taps and the sun's Cook-Torrance BRDF run only when the sun's intensity is not zero, in the forward shader, both light passes and the transparency shader, on a per-frame constant (a uniform branch). Any other intensity takes the old path in the old order of products, bit for bit.
  • Each material map sampled only when the material has it, the normal map's decode and tangent transform included; the flags are per-draw constants, so the implicit-derivative samples stay in uniform control flow. The reviewers put this at nothing measurable; it stays as a cleanup.
  • Once per pixel instead of twice: the view vector, the environment's rotation in radians, the BRDF lookup table's pair (both reflectances derive from it); the rotation of the lookup direction is skipped at angle zero, the room probe's case. The normalize is kept, because the simulator's light pass hands in a G-buffer normal as stored and the equirectangular latitude's asin is not scale-invariant; that asin is clamped now, which removes a pre-existing chance of a NaN pixel at the poles.
  • The temporal resolve leaves early for a pixel with no history or at the clear depth, before the rate-map decodes, the reprojection and the velocity fetch. The foveation rates are read as texels per screen pixel over the same screen interval from both maps, a second decode of each, instead of derivatives across the quad, which forbade any early return: the ratio is exactly 1 wherever the two maps agree, zone boundaries included.
  • The sun's cascades are a setting, setRendering(.shadowCascades(1 or 2)): one on the headset by default, where a room or a tabletop never shows the distance the second covers; with maxShadowCastingDistance set to the scene's size one cascade is sharp on it. The count flows through the splits, the fitting, both shadow passes and the shader's uniforms.
  • A sun that casts no shadow shadows nothing. When a sun stops casting the cascades are no longer drawn, but the shader kept sampling the last maps; its matrices now land every point outside the maps, which the shader reads as lit.

Twelve reviewers (Metal semantics, picture equivalence, performance, three per cut) tried to refute each shader cut; what they found is folded in.

On the device

The sun off, 100 Hz: Forward 8.02 → 6.56 ms (the fragment part 5.51 → 4.15), TAA 2.06 → 1.92 (the empty frame 1.81 → 1.47), the frame 6.8 ms against the 10 ms refresh, no late frame after the placement. The sun on, one cascade: Shadows 0.85 → 0.28 ms, the sun about 0.9 ms in all; full rate at 90 Hz.

Tests

The material shading, per-fragment lighting, glass, probe, post-effects and temporal tests pass on the rebuilt library (148 in the targeted run); the cascade setting, its headset default and that one cascade still casts on a floor are covered by render tests, the non-casting matrices by a unit test.

Javier Segura added 4 commits October 15, 2026 16:29
… pixel: the sun by its intensity, the maps by their flags, the lookups once, the resolve's early-out

On the headset the Forward pass took 8.0 ms for both eyes (5.5 of it fragment
work) and the temporal resolve 2.1 against an 11.1 ms refresh, with a sun the
viewer switches off, materials without every map, a room probe that is never
rotated, and a frame that is mostly cleared. Four cuts in the shaders, each a
branch on a per-draw or per-frame constant (uniform across the draw) or a value
computed once:

- The sun's cascade shadow taps and its Cook-Torrance BRDF are evaluated only
  when the sun's intensity is not zero: in emissiveWithIndirect and
  computeDirectLighting (the forward path and the tile light pass), the texture
  light pass, lightTBDRSample and the transparency shader. Any other intensity
  takes the old path in the old order of products, bit for bit. The CPU's
  cascade gate (ShadowSystem.updateCascades) says the same: not zero.
- Each material map is sampled only when the material has it (base colour,
  normal with its decode and tangent transform, roughness, metallic, emissive);
  the flags are per-draw constants, so the implicit-derivative samples stay in
  uniform control flow, as the height-map branch already was. The reviewers put
  this at nothing measurable (the removed samples were on nil textures); it
  stays as a cleanup.
- The view vector, the environment's rotation in radians and the BRDF lookup
  table's pair are computed once per pixel (brdfIntegration, with both
  reflectances derived from the pair; the texture-taking signatures stay as
  wrappers); the rotation of the lookup direction is skipped at angle zero, the
  room probe's case, with the normalize kept: the simulator's light pass hands
  in a G-buffer normal as stored, and the equirectangular latitude's asin is
  not scale-invariant. That asin is clamped now, which removes a pre-existing
  chance of a NaN pixel at the poles.
- The temporal resolve returns early for a pixel with no history or at the
  clear depth, before the rate-map decodes, the reprojection and the velocity
  fetch. The foveation rates are read as texels per screen pixel over the same
  screen interval from both maps, a second decode of each, instead of
  derivatives across the quad, which forbade any early return: the ratio is
  exactly 1 wherever the two maps agree, zone boundaries included; where they
  differ, a band at most a pixel wide before a boundary weighs a little
  differently than the quad's span did.

The kernel libraries are rebuilt for every platform (buildkernels.sh). Twelve
reviewers tried to refute the cuts on Metal semantics, picture equivalence and
performance; what they found is folded in above. The material shading, lighting,
glass, probe, post-effects and temporal tests pass on the rebuilt library.
…ault

A sun switched on in a headset scene cost about 1.5 ms a frame, half of it the
second cascade: every caster drawn once more into a 2048x2048 depth map, and the
band where the two cascades meet sampled twice, for a distance a room or a
tabletop never shows. The cascade count is a setting now,
setRendering(.shadowCascades(1 or 2)): unset, the headset draws one and the
desktop every cascade the maps hold. The count flows from the setting through
the cascade splits (one cascade reaches the whole shadow distance), the frustum
fitting, both shadow passes and the shader's uniforms, which already read the
count; the depth array keeps its slices, and the slots past the cascades in use
repeat the last one. With maxShadowCastingDistance set to the scene's size, one
cascade is sharp on a small scene.

A render test checks the setting, the splits and the headset default; the
proposal and the XR architecture doc record the device measurements behind it.
…is shown to cast

When a sun stops casting (castsShadow off, or no sun active) the cascades are
no longer drawn, but the shader kept sampling the last maps through the last
matrices, so the stale shadows stayed on the scene. The uniforms of a sun that
does not cast now carry matrices that land every point outside the maps, which
the shader already reads as lit.

Found by a render test that renders a box on a floor with the sun casting and
not, and counts the floor pixels the shadow darkens: with two cascades and with
one, which must shadow about the same pixels; the unit test of the uniforms'
packing states that its system is an active one, and a sibling checks the
non-casting matrices.
@miogds

miogds commented Oct 16, 2026

Copy link
Copy Markdown
Member Author

Closed without merging here: the Forward pass and resolve doing less per pixel (feature/xr_shader_cost at eac4e3e) is on develop since #76 (merge 0b6348e of feature/xr_shader_cost at eac4e3e, which carries #73–#75), and develop now also holds the head of untoldengine#1359 through #78 and #79. The branch stays for the upstream pull request once untoldengine#1359 lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant