Skip to content

Unified GSplat Picking Support#8421

Merged
mvaligursky merged 1 commit into
mainfrom
mv-unified-picking
Jan 30, 2026
Merged

Unified GSplat Picking Support#8421
mvaligursky merged 1 commit into
mainfrom
mv-unified-picking

Conversation

@mvaligursky

@mvaligursky mvaligursky commented Jan 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds picking support for Gaussian Splats in unified rendering mode
  • Introduces centralized PickerId class for consistent ID allocation across pickable objects
  • Extends the Picker class to return GSplatComponent for unified gsplats alongside MeshInstance for other objects

Changes

Centralized ID Generation

  • New PickerId class (src/scene/picker-id.js) with static get() method for allocating unique IDs
  • MeshInstance now uses PickerId.get() instead of internal counter
  • GSplatComponent allocates its own ID at construction, passed to placement

GSplat Unified Picking

  • GSplatParams.id property enables per-component ID storage in the work buffer
  • When enabled, a pcId stream (R32U format) is added to the work buffer format

Picker Updates

  • Picker.getSelection() and getSelectionAsync() now return (MeshInstance | GSplatComponent)[]

Examples

  • Updated gaussian-splatting/picking.example.mjs to demonstrate unified picking
  • Created gaussian-splatting-legacy/picking.example.mjs for non-unified picking reference

Public API Changes

New: GSplatParams.id

// Enable gsplat ID for unified picking
app.scene.gsplat.id = true;

Once enabled, cannot be disabled. The ID stream persists for the lifetime of the app.

Updated: Picker.getSelectionAsync() / Picker.getSelection()

Return type changed from MeshInstance[] to (MeshInstance | GSplatComponent)[]:

picker.getSelectionAsync(x, y, 1, 1).then((results) => {
    const picked = results[0];
    if (picked instanceof pc.GSplatComponent) {
        // Unified gsplat component
        console.log('Picked gsplat:', picked.entity.name);
    } else {
        // MeshInstance
        console.log('Picked mesh:', picked.node.name);
    }
});

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds GPU picking support for Gaussian Splats in unified rendering by introducing a shared ID allocation path and wiring those IDs through the work buffer into the pick pass, so Picker can return GSplatComponent alongside MeshInstance.

Changes:

  • Introduces centralized PickerId generator and migrates MeshInstance / GSplatComponent to use it.
  • Adds optional pcId work-buffer stream (GSplatParams.id) and shaders to write/read that ID for pick rendering.
  • Extends picker render pass + examples to support unified gsplat picking and maintains a legacy picking example.

Reviewed changes

Copilot reviewed 21 out of 25 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/scene/shader-lib/wgsl/chunks/gsplat/vert/gsplat.js Passes per-splat/component pick ID through a flat varying during pick pass.
src/scene/shader-lib/wgsl/chunks/gsplat/frag/gsplatCopyToWorkbuffer.js Writes uId into the pcId stream when enabled.
src/scene/shader-lib/wgsl/chunks/gsplat/frag/gsplat.js Encodes vPickId into pick output for unified gsplat pick pass.
src/scene/shader-lib/wgsl/chunks/common/frag/pick.js Adds encodePickOutput(id) and makes meshInstanceId optional via PICK_CUSTOM_ID.
src/scene/shader-lib/glsl/chunks/gsplat/vert/gsplat.js GLSL equivalent: passes flat vPickId during pick pass.
src/scene/shader-lib/glsl/chunks/gsplat/frag/gsplatCopyToWorkbuffer.js GLSL equivalent: writes uId into pcId stream.
src/scene/shader-lib/glsl/chunks/gsplat/frag/gsplat.js GLSL equivalent: encodes vPickId during pick pass when unified ID enabled.
src/scene/shader-lib/glsl/chunks/common/frag/pick.js GLSL equivalent: adds encodePickOutput(id) and PICK_CUSTOM_ID gating.
src/scene/picker-id.js Adds centralized picker ID allocator.
src/scene/mesh-instance.js Migrates mesh instance IDs to PickerId.get().
src/scene/gsplat-unified/gsplat-work-buffer.js Enables GSPLAT_ID define when pcId stream exists so copy shader writes IDs.
src/scene/gsplat-unified/gsplat-work-buffer-render-pass.js Sets uId per placement when rendering into the work buffer.
src/scene/gsplat-unified/gsplat-renderer.js Enables unified-ID shader defines (GSPLAT_UNIFIED_ID, PICK_CUSTOM_ID) when pcId exists.
src/scene/gsplat-unified/gsplat-placement.js Adds placement id and propagates it through placement creation / inheritance.
src/scene/gsplat-unified/gsplat-params.js Adds GSplatParams.id to append pcId stream (R32U) to the work buffer format.
src/scene/gsplat-unified/gsplat-info.js Carries placement ID into render info for shader uniform setup.
src/framework/graphics/render-pass-picker.js Extends picker pass to map placement IDs to GSplatComponent.
src/framework/graphics/picker.js Updates picker documentation/types to return `MeshInstance
src/framework/components/gsplat/component.js Allocates a stable component pick ID and passes it into unified placement creation.
examples/thumbnails/gaussian-splatting_picking_small.webp Adds updated thumbnail for unified picking example.
examples/thumbnails/gaussian-splatting_picking_large.webp Adds updated thumbnail for unified picking example.
examples/thumbnails/gaussian-splatting-legacy_picking_small.webp Adds thumbnail for legacy picking example.
examples/thumbnails/gaussian-splatting-legacy_picking_large.webp Adds thumbnail for legacy picking example.
examples/src/examples/gaussian-splatting/picking.example.mjs Updates example to enable unified mode + scene.gsplat.id and handle GSplatComponent results.
examples/src/examples/gaussian-splatting-legacy/picking.example.mjs Adds a separate legacy (non-unified) gsplat picking example.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/framework/graphics/render-pass-picker.js
Comment thread src/scene/picker-id.js
@mvaligursky mvaligursky requested a review from a team January 30, 2026 11:38
@mvaligursky mvaligursky merged commit 5b2a9ac into main Jan 30, 2026
13 checks passed
@mvaligursky mvaligursky deleted the mv-unified-picking branch January 30, 2026 12:31
}

// Enable gsplat ID for unified picking
app.scene.gsplat.id = true;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I hadn't noticed this property before. Couple of things:

  1. I still feel like configuration props for rendering gsplats should live on the GSplatComponentSystem.
  2. id suggests to me it's an actual ID (e.g. a GUID or number). So naming feels weird. Something like enableIds would be better IMHO.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. perhaps.
  2. will do, great point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: graphics Graphics related issue enhancement Request for a new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants