UT-Glasses

Inspiration

Communication is one of the most fundamental human needs, yet millions of Deaf and hard‑of‑hearing individuals face barriers when others cannot understand sign language. Our team wanted to create something that makes everyday interactions more natural and inclusive. That vision led us to build a pair of smart glasses capable of translating sign language into text in real time.

What it does

We developed smart glasses equipped with a front‑facing camera and a companion mobile app. The camera captures hand gestures, our model interprets them, and the translated text appears instantly on the app.

Real‑time translation with low latency
Wearable and portable hardware
Accessible design for both signers and non‑signers
Scalable architecture for adding new signs and languages

How we built it

Our system combines computer vision, machine learning, and embedded hardware:

Trained a gesture‑recognition model using a curated dataset of sign‑language videos
Integrated the camera module with a microcontroller
Built a companion app to display translated text in real time
Designed a clean UI prioritizing clarity and accessibility

Challenges we ran into

Building a real‑time translation system on wearable hardware came with several hurdles:

Lighting variability affected gesture detection
Model size constraints required aggressive optimization
Similar gestures caused occasional misclassifications
Latency needed to be extremely low for natural conversation
Hardware integration demanded careful tuning of sensors and power usage Each challenge pushed us to iterate, refine, and learn quickly.

Accomplishments that we're proud of

Built a fully working end‑to‑end prototype of smart glasses capable of capturing and transmitting sign‑language gestures in real time.
Achieved low‑latency translation, bringing our inference time down to a level that feels natural in conversation.
Optimized our ML model using techniques like quantization and pruning, reducing size while maintaining strong accuracy.
Created a clean, accessible companion app that displays translated text instantly and clearly.
Integrated hardware and software seamlessly, from camera modules and microcontrollers to wireless communication and UI design.
Developed a scalable architecture that can support additional signs and languages.

What we learned

This project strengthened our skills in:

Real‑time computer vision
Model compression and optimization
Embedded systems and sensor integration
Designing for accessibility and inclusivity Most importantly, we learned how impactful technology can be when built with empathy and purpose.