Inspiration
Physical therapy can feel repetitive and discouraging for kids, which leads to lower engagement and missed progress. We wanted to reimagine rehab as something children want to do so we built Bowser.
What it does
Bowser is an AI-powered pediatric therapy companion that turns rehab exercises into an interactive game loop. It tracks patients movements and posture in real time, counts reps, and encourages children by using Bowsers(From Mario Kart) voice.
How we built it
We built this by using the base of a Roomba 690, and connecting it to a ESP32 through a 7 pin mini din cable. For the perception, we used OpenCV + MediaPipe + YOLO + Depth estimation to track body posture, as well as people in the area. For the main intelligence, we used Google ADK, building 4 agents that run together to achieve its purpose.
-> Patient Record Agent: Retrieving Hospital Database by scanning the QR Code
-> Clinical Agent: Emotionless agent strictly to calculate
-> Therapy Assistant Agent: it takes in live microphone data and generates outputs to encourage the kids to complete their therapy
-> Robotic Pacer Agent: takes in the data and outputs directions for roomba to move.
Challenges we ran into
One of our biggest challenges was the tracking of the roomba. We had a hard time finding the perfect combination of libraries to map out and keep track of the kids and not crash into objects when it drove autonomously.
Accomplishments that we're proud of
We are very proud of being able to build our first hardware hack, espically being autonomous.
What we learned
We learned how to connect a roomba to a ESP32, and actually working with the chip, also working on the autonomous features. A lot of trial and error helped us understand the logic on how to make a successful program.
What's next for Bowser
In the future we hope to add a fully autonomous multi robot system thats a bigger size.
Log in or sign up for Devpost to join the conversation.