Fix XR over-culling: correct combined frustum and per-eye gsplat visibility#8889
Merged
Conversation
…bility Two related XR culling fixes: Frustum.add (used to build the combined XR culling frustum) merged per-plane by distance only, assuming identical plane normals. Real headsets report asymmetric per-eye projections, so matching planes have different normals and the picked plane cuts into the combined volume - which plane won even depended on the camera's world position. Rewritten as a conservative corner-based merge: the other frustum's 8 corners are computed and this frustum's planes are pushed outward to contain them, correct for arbitrary frusta. The XR combined-frustum logic is extracted from the renderer into Camera#updateXrFrustum and reused by the gsplat interval culling, which previously culled XR with a mono symmetric frustum. The hybrid gsplat projector culled per-splat against eye 0's viewport only, dropping splats visible only near the other eye's edge (missing splats on the right edge of the right eye). The compute cull now projects eye 1 (reusing the result for the cache write, so net cost is ~zero) and rejects a splat only when it is off-screen in both eyes. The foveated culling radius likewise uses the smaller of the two eyes' radii, so splats near the centre of either eye are protected. Fixes #8449
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes XR-specific culling errors affecting both general scene frustum culling and hybrid GSplat stereo rendering. It makes XR “combined frustum” construction conservative for asymmetric per-eye projections (real headset behavior) and updates GSplat culling/projection to keep splats visible in either eye.
Changes:
- Reworks
Frustum#addto conservatively merge frusta via corner containment rather than per-plane distance selection. - Extracts combined XR frustum update into
Camera#updateXrFrustumand reuses it from bothRenderer.updateCameraFrustumand GSplat interval culling. - Updates GSplat hybrid projector WGSL to do stereo union visibility (and foveation radius handling) using both eyes, reusing the eye-1 projection for cache writes.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/core/shape/frustum.js | Replaces Frustum.add with conservative corner-based expansion using 3-plane intersections. |
| src/scene/camera.js | Adds Camera#updateXrFrustum helper to build a combined XR frustum across views. |
| src/scene/renderer/renderer.js | Switches XR frustum update path to call camera.updateXrFrustum() (removes inlined logic). |
| src/scene/gsplat-unified/gsplat-frustum-culler.js | Adds setFrustumPlanes(frustum) and routes matrix-based computation through it. |
| src/scene/gsplat-unified/gsplat-manager.js | Uses combined XR frustum (with per-view transform refresh) for GSplat interval culling. |
| src/scene/shader-lib/wgsl/chunks/gsplat/compute-gsplat-common.js | Adds eye-1 projection + stereo-union offscreen test and min foveation radius logic under GSPLAT_XR. |
| src/scene/shader-lib/wgsl/chunks/gsplat/compute-gsplat-project-common.js | Threads viewProj1 into shared projection helper when GSPLAT_XR is enabled. |
| src/scene/shader-lib/wgsl/chunks/gsplat/compute-gsplat-projector.js | Passes viewProj1 to projection helper and reuses computed eye-1 NDC for stereo cache write. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two related XR culling fixes, found while testing the hybrid gsplat renderer in stereo on Quest 3 / Apple Vision Pro.
1.
Frustum.addover-culls with real headset projections (fixes #8449)The combined XR culling frustum (introduced in #8393) merged the two eye frusta per-plane by keeping the larger
distancescalar, assuming "frustum plane normals are identical" for parallel-axis stereo. That holds for the view direction, but real headsets report asymmetric per-eye projections (e.g. ~49° outer / ~37° inner on Quest-class devices), so the matching planes of the two eyes have different normals. The two candidate planes cross, and the picked one cuts into the true combined volume beyond the crossing — and because thedistancescalars shift differently with camera world position when normals differ, which plane won depended on where the camera was in the world. This is why #8449 reproduced on hardware in some scenes but not in the simulator (symmetric, identical eye projections) or at other locations.Fix:
Frustum.addis rewritten as a conservative corner-based merge — the other frustum's 8 corners are computed (three-plane intersections) and this frustum's planes are pushed outward just enough to contain them. Plane orientations are preserved, making the merge correct for arbitrary frusta with no parallelism assumptions. Verified with a standalone reproduction (Quest-like asymmetric frusta, IPD 64 mm): the old merge over-culled 35/70 true-union edge points across 7 camera world positions; the new merge over-culls none, and still excludes behind-camera points.The XR combined-frustum construction is also extracted from
Renderer.updateCameraFrustuminto a sharedCamera#updateXrFrustum, now additionally used by the gsplat interval culling — which previously culled XR frames against a mono symmetric frustum.2. Hybrid gsplat projector culled per-splat against eye 0 only
The GPU-sort projector's per-splat screen cull tested only eye 0's viewport, dropping splats visible only near the other eye's edge (visible as missing splats at the right edge of the right eye). The compute cull now also projects eye 1 — reusing the result for the cache write that needed it anyway, so the net cost is ~zero — and rejects a splat only when it is off-screen in both eyes. The foveated culling radius similarly uses the smaller of the two eyes' radii, so splats near the centre of either eye are protected from the foveated threshold boost.
Changes:
Frustum#add— corner-based conservative merge (behaviour fix; API unchanged).Camera#updateXrFrustum(ignored/internal) — shared combined-XR-frustum helper, used byRenderer.updateCameraFrustumand gsplat interval culling.GSplatFrustumCuller#setFrustumPlanes— accepts a prebuilt frustum;computeFrustumPlanesdelegates to it.GSplatManager— XR interval culling uses the combined frustum (with per-view transform refresh, matching the projector dispatch).computeSplatCov/projectSplatCommon/ projector): stereo union visibility test via eye-1 projection (GSPLAT_XR-gated; mono and compute-local paths unchanged), eye-1 NDC reused for the cache write, per-eye min foveation radius.Fixes #8449