CanYouDoWhatICanDo

CoopLevel

Inspiration

The inspiration comes from a Japanese game show: https://www.youtube.com/watch?v=zL4HSk4MUUw The players are standing in front of a moving wall which only has small cut outs through which the players have to squeeze through in order not to get pushed in a pool behind them.

We wanted to play this game without a massive hardware set up and easier way to create new challenges that don't require a full workshop to build the walls.

What it does

The installation records the pose of a player with a 3d camera and uses a projector to show the current player a body pose (or a shape) that he has to achieve as close as possible. The player needs creativity and for some poses quite some sense of balance.

A game consists of multiple levels and the player loses health points in each level until she fills the given shape and held this posture for some seconds.

New levels can be create by drawing them by hand or by storing the current pose of the player so that players can challenge each other. (The reason for the project name)

How we built it

The camera is positioned underneath the projection area and the projector is located so that the player does not cast shadows.

We use an Asus Xtion 3D Camera and the Openni2-based ROS-Driver to capture the pointcloud data. As the camera and the projector are facing each other, an automated calibration would need artificial targets so we decided to rely on a manual calibration. The most important degree of freedom is the angle of the camera relative the floor which can be estimated in only a view tries (using interactive markers in RVIZ to define the transform).

The pointcloud is read by a C++-Node that applies the transformation into the correctly rotated frame, does a z-thresholding to suppress people standing behind the player, projects it into the xy-plane and publishes it as an image and jpeg-stream. (The jpeg-stream was needed to get the data into Processing, an approach we stopped after some hours of development due to missing computer vision functions in Processing).

The pre-processed depth-mask is then read by a python-program that contains the game logic and computes the score by comparing the current depth-mask (created by the user) with a level.

The game logic is a state-machine with these states:

Waiting for Player
Game is played
Player has won
Player has lost

Depending on the game state, UI-Interfaces like Health-Bars or messages to the player are selectively shown.

While the installation is waiting for a player, a start-screen is shown and simple task (clearly recognizable human shape) is shown. If a person is crossing the playing area, his shape is visualized and the overlapping region drawn in green so that the game idea is easy understandable even without having to read a text. The player starts a game by matching the shape so that no other interface like a button or keyboard is needed. Therefore, the installation has no physical interface which could be damaged.

If there are multiple people within the playing area when the shape is matched, a multi player match is started.

The player has a global health-level that is decreased continuously (with a grace period of some seconds after each level-start). She also has a progress-bar for each level that fills the faster the better she is matching the given shape. As soon as the progress-bar is filled up, a new level is started (or the game is won). The game ends as soon as the global health-level is at zero. Each successful level increases the health a bit again.

Challenges we ran into

Getting images from ROS to Processing was harder than expected and we missed basic Computer Vision functions in Processing so that we decided to remove Processing again and did all visualizations with OpenCV.

Accomplishments that we're proud of

Responsive system, Easily understandable, no expensive components, we are only two Computer Scientists with no design experience, but the game looks nice and is easy to understand.

What we learned

Processing is really limited...

What's next for CanYouDoWhatICanDo

Full port to Raspberry Pi, so setup is independent from laptop (this would most likely reduce the framerate)
Automatic Calibration of the system
Parallel installations so that two players in different locations can play together. In this setup, the shape of the distant player is projected next to the local player so that the players have to fill shapes together, but can only communicate via pantomime.