Hololens City

Use Case

In the city of Karlsruhe, citizens can report problems to the city's administration office, like defective street lamps or bumps on the roads. Today, the data is available as a flat CSV file, which is not optimal to get an overview on the problem hot spots. Thus, status tracking of these issues is not easy.

We improve this by using a cutting edge holographic mixed-reality application (which is quite cool by itself :-).

This helps to visualize the problems, get an overview of hotspots and track the status of each of them.

Challenges that had to be solved:

Create a holographic map of Karlsruhe and render it in the mixed-reality world
Enable the user to interact with this map, e.g. place it on a table, rotate it etc.
Place the feedback data on the map. As the feedback data only contained the address of the location where the problem occurred, we also had to enrich this with geolocation information (latitude and longitude)
Use speech recognition to show the data overlay
Enhance the gesture recognition capabilities of Hololens which were not sufficient for our use case by integrating it with a Leap Motion device

Github repository contents

In the repository, you will find:

A Unity game engine project that is deployed on the Hololens. This also contains an extension for enhanced gesture recognition based on a Leap Motion device,
A utility application that reads the feedback data from a CSV file, enriches it with geolocation information and writes the results back to a JSON file to be imported by Unity.

Our experiences

It was quite exciting and fun to work with such a technically challenging environment
Due to the experimental status of the Hololens SDKs, we had to find a lot of work-arounds
A lot of existing Unity community libraries were not working when deployed on the Hololens device. This blocked the integration of the Leap Motion device which worked very well in the emulator (and was a real eye-catcher there), but did not work on the actual device.