RobotAR

RobotAR

Inspiration

It is very complex to program robots. There is the possibility with quaternions, Euler Winkel and a lot of code. Or to control it remotely with a complex controller. Or per 0-force control. However, these are no easy tasks especially in human inaccessible areas.

What it does

We wanted to revolutionize how we program robots. Per mapping of the hand of the user (which is captured with the Holo Lens), we are moving the ABB YuMi Robot. We are able to move the robot in X,Y and Z directions. We are able to manipulate the grabber with a pinch. Therefore we are able to fully program the robot with Holo Lens. A new area in robotics programming.

How we built it

With a lot of tears and sweat. And the Microsoft Mixed Reality Toolkit, Unity, Azure, RobotStudio, Lots of Math and some dedication.

Challenges we ran into

Just a little collection of problems:

Visual Studio did not work
Unity did not work
Complex movements of the robot arm and the pairing to the hand of the user with Holo Lens
Holo Lens didn't connect to the internet from inside the app
Holo Lens didn't connect to the robot
Sleep deprivation ;)

Accomplishments that we're proud of

It is a working prototype for an innovative way to program a robot.

What we learned

That we can achieve anything if we work together as a team and all give our best to contribute to the project.

What's next for RobotAR

The product is scale-able and could be used for the training of any robotic arm at production sites. But also for tasks which cannot be performed by humans, for example: in a submarine, defusing of a bomb, in space or in case of a catastrophe.