Neural radiance fields (NeRFs) and Gaussian splatting recently received a lot of attention. These are 3D representations that can be optimized from posed image collections via differentiable rendering yielding near-photorealistic results.

There are a large number of follow-up works that adopt the main idea from the original works, but modify the network architecture, sampling procedure, exact rendering equation, relax the assumption of posed images, etc.

This makes it more difficult to support these directions out-of-the-box.

Describe a solution you'd like
In my opinion the best (and maybe only) way to add support is via plugins that allow custom datatypes (e.g., logging the 3D Gaussians, network weights, or whatever is the underlying representation) and custom rendering (e.g., given the camera parameters for the 3D view, and the logged data, let the plugin render RGB image + Z buffer, which are then combined with supported primitives on the Rerun side).

NeRFs can typically not be viewed in real-time, thus an adaptive rendering scheme should be implementable (i.e., resolution could easily be increased when the camera does not move, I believe nerfstudio does this).

Describe alternatives you've considered
An option might be to support whatever comes closest to a reference implementation for Gaussian splatting (e.g. this one). But I'm not convinced this is a viable solution at this point, when there's still a lot of research at the renderer level going on.