The Kinect Awakens

Inspiration

We're inspired by the possibilities of augmented and virtual reality with a realistic user interface.

What it does

The player wears a Google Cardboard, which places the user into a virtual reality world, facing a monster the enemy to battle. The user holds the PS Move controller as a light saber and swings it to fight the monster. The perspective of the user changes as they walk around, move and duck.

How we built it

Behind the scenes, a Microsoft Kinect computes the coordinates in 3D space of the player's head and hands. The PS Move controller and the IMU of the phone provide the orientations of the camera and weapon, to be rendered with Unity. Also, the computations are distributed over several machines and platforms, connected by a C# client/server.

Challenges we ran into

One of our goals was to use the ARToolkit to display the player's surroundings in the Google Cardboard, creating an augmented reality game. We also wanted to use OpenCV to track the top of the PS Move controller, but integration proved very difficult.