Signify

SIgnify dashboard for new user
Signify post practice session screen
Onboarding screen describing some of what Signify does
Signify stats page for experienced user
Teaching Card in Signify Lesson
Live practice during Signify Lesson

Inspiration

Our team's original inspiration for this project was to bridge the communication gap so we could reduce the isolation of the deaf community and empower inclusive everyday communications.

What it does

Signify teaches ASL through structured, beginner -> advanced lessons that lets students learn in a friendly environment through video demonstrations and skill mastery tech. The platform uses a live camera feed to analyze the student's own technique to improve their ASL skills with no delay.

How we built it

We built the platform using Nuxt for a fast, responsive frontend and developed our own custom ASL recognition model with TensorFlow to power real-time feedback. The model was trained and optimized using high-performance computing resources on HiPerGator, along with additional web and AI technologies to ensure accurate gesture detection and seamless user interaction.

Challenges we ran into

One major challenge we faced was that training our ASL recognition model required many iterations before it produced accurate and consistent results. We had to repeatedly adjust the dataset, fine-tune hyperparameters in TensorFlow, and retrain the model on HiPerGator to improve performance and reduce misclassifications.

Accomplishments that we're proud of

We’re especially proud of leveraging NVIDIA B200 GPUs on HiPerGator to train our custom ASL recognition model at scale. Having access to high-performance computing resources allowed us to experiment, iterate quickly, and push our system toward real-time, production-level performance — turning our idea into serious, large-scale computing.

What we learned

Through this project, we strengthened our understanding of advanced data analysis techniques such as dynamic time warping for comparing temporal gesture sequences and z-score normalization for improving model consistency and feature scaling. Learning and applying these mathematical methods helped us significantly improve the accuracy and reliability of our ASL recognition system.

Z-score normalization rescales a value based on the mean and standard deviation of the dataset:

$$ z = \frac{x - \mu}{\sigma} $$

x = original data value

μ = mean of the dataset

σ = standard deviation

z = normalized value

For two sequences

X=(x1,x2,…,xn) X=(x1,x2,…,xn) Y=(y1,y2,…,ym) Y=(y1,y2,…,ym)

The DTW distance is defined recursively as: $$ DTW(i,j) = d(x_i, y_j) + \min \begin{cases} DTW(i-1, j) \ DTW(i, j-1) \ DTW(i-1, j-1) \end{cases} $$

What's next for Signify

We plan to expand Signify to let users learn phrases, master words, and especially create a more effective model that is capable of phrases & words and body language too.