NeRF-Herding

• 

  Generated 3D Model

Challenge 6

Inspiration

Inspired by the heroic amounts of effort that into diagnosing the problems caused during the Apollo 13 disaster.

What it does

Using short, low quality videos, generate a complex three dimensional scene of the location.

How we built it

Using the basic technology Neural Radiance Fields, and optimized using Instant Neural Graphics Primitives to enable it to run on a laptop and produce high resolution meshes.

Challenges we ran into

• Hard to get access to Engineering cameras on Spacecraft to demonstrate the capabilities of the model. Ended up using videos of random locations in the Hackathon arena.
• Laptop kept getting thermally throttled and running out of RAM while trying to get the models to converge.

Accomplishments that we're proud of

• Despite a severe lack of quality videos, generated enough models to demonstrate a solid proof-of-concept.
• Tuned hyperparameters perfectly for some tricky models and got a fantastic model of the Perseverance rover being deployed from the skycrane.

What we learned

NeRF is a viable alternative solution to traditional photogrammetric methods, especially when the scene being depicted is complex structurally and lightwise.

What's next for NeRF-Herding

Test it on real-world spacecraft videos!

Built With

• deeplearning
• instantngp
• nerf
• python