Inspiration
As a student, I’ve seen teachers struggle with switching slides, using laser pointers, or writing on smartboards that only work in one spot of the classroom. Many schools can’t afford expensive interactive whiteboards or high-power projectors. I wanted to create a low-cost, energy-efficient solution that allows teachers to interact naturally with their presentations anywhere on the wall.
What I Learned
How to use MediaPipe Hands + Ultralytics YOLO + OpenCV to detect and classify hand gestures.
The concept of homography mapping to calibrate fingertip touches on a wall to projected coordinates.
How to design a 3D-printed enclosure with passive cooling and proper cable routing.
The importance of balancing hardware constraints (power, heat, weight) with usability in real-world teaching environments.
How to assemble an embedded computer with an onboard NPU.
How I Built It
Hardware Setup: I combined a Jetson Orin Nano, a 50W Yaber E1 projector, and an Anker C200 webcam into one 3D-printed box.
Gesture System: Using MediaPipe and Ultralytics, I mapped universal gestures and mode-specific actions for Presenting, Drawing, and Normal navigation.
Calibration: I built a system where the user taps projected points on the wall, and OpenCV computes a homography matrix to ensure drawings align perfectly with the projection.
Enclosure Design: In CAD, I designed a vented case printable on a Bamboo Labs A1 that keeps airflow, secures components, and positions the camera to avoid projector glare.
Startup Integration: I scripted the Jetson to auto-run both the Python program and the presentation on boot, so setup is as seamless as plugging it in.
Challenges I Faced:
Performance Constraints: The Jetson had to run real-time hand tracking and projection simultaneously without overheating, so I optimized code and ensured proper ventilation in the case.
Lighting Conditions: Projector glare sometimes blinded the webcam. I solved this by adjusting the camera angle and adding shielding in the enclosure design.
Time Pressure: Building both software and hardware solo in 24 hours required rapid prototyping and prioritizing features that had the most impact.
Built With
- embedded-computing
- linux
- machine-learning
- python
- roboflow
- ubuntu
Log in or sign up for Devpost to join the conversation.