RoboTherapy Doggo

Inspiration

"Did you know that an epidemic of loneliness plagues America, and that students often bear the heavy weight of stress? Enter Rover, the robotic therapy dog designed to address both of these pressing issues. Rover isn't just a machine; it's a lifeline for those in need.

What it does

This empathetic companion alleviates loneliness with its warm presence, empathetic listening and various faces in a world where loneliness and stress often intertwine, Rover offers a ray of hope, turning challenges into opportunities for personal growth and connection. Join us on this journey to redefine well-being and education with Rover, your constant companion in a sometimes turbulent world."

How we built it

The project is broken down into two separate parts: software and hardware. For software, we use Open CV for image processing. The library provides an easy way to access the computer's camera for image input. After capturing a frame of the user's face, the program relies on DeepFace for emotion detection to gives back an emotion. Based on the emotion, the program calls Llama 2 and gets the empathetic response back. We make an interactive websites on Streamlit application to display the front-end and handle the user interaction with the program.

For the hardware, we use embedded systems to build the robot. We also use 3d-printing to print out the frame for the dog robot.

Challenges we ran into

It was not possible to set up tensorflow locally on Mac since the M1 chip does not allow the use of DeepFace for emotion detection. Moreover, for large language model, we need to find different methods to make API call to the free model.

Our camera broke when we do the hardware.

Accomplishments that we're proud of

We were able to set up the Streamlit application that handles a lot of image processing and emotion detection. We also set up successfully the motor and LCD for the robot.

What we learned

What we learned is that doing embedded system requires a lot of patience and dedication in order to debug and make progress. Moreover, it was very cool to see how the data transferred from the Streamlit application to the robot dog.

Built With

• autodesk-fusion-360
• c
• deepface
• flask
• llama
• opencv
• python
• streamlit