Project LARP

Inspiration

Orienting and accurately geo(luna?)locating yourself on the moon is a hard problem that only gets harder on Mars. On Earth, it requires overhead satellites for triangulation. However, these satellites do not exist for the moon. For real time navigation, astronauts cannot rely on ground control. However, they still need to be able to navigate to destination locations avoid hazards, terrain changes. Additionally, whatever navigation tool they use must be seamless and integrated into their helmet's HUD.

What it does

Project LARP (Lunar Augmented Reality Paths) solves this problem by having the astronaut simply take a picture of the horizon and click a button saying where they want to go. An ML algorithm analyzes the image of the horizon and maps it to terrain maps generated through overhead satellite data (NASA's DEMs). Then, the user can select a location from an AR drop-down menu. The HUD will then calculate the least-cost path from the astronaut's calculated position to the area of interest, automatically avoiding difficult terrain (anything steeper than 15 degrees up or down). It will then display an augmented reality map showing where the astronaut needs to go.

How we built it

There are three main components: location recognition, pathfinding, and the AR interface. We used an Inception Version 3 CNN algorithm to train the ML on 8 images of the horizon. We then used Python's sci-kit image to find the shortest paths. For the AR, we used Unity with a Vuforia engine to generate the XR UI.

Challenges we ran into

We had never used Unity before, so we encountered several challenges with getting it to work. We also had problems with map projection and 3d modeling.

Accomplishments that we're proud of

We have a functional AR display!! We are also very proud that our ML model has a 70% success rate after just one day of working on it.

What we learned

We learned how to make projects in AR, learned how to create topography maps from elevation color maps, learned more about areas of interest at the lunar south pole, learned about the challenges of integration, and so much more.