Sign atlas

Home page
Gesture recognition
3D hand model in course example

Inspiration

Sign Atlas was born from a passion for inclusivity and accessibility. Witnessing the transformative power of sign language in bridging communication gaps, we were inspired to create a platform that empowers everyone, especially those with disabilities. By combining advanced 3D visualization and gesture recognition technology, Sign Atlas aims to make learning sign language engaging, effective, and inclusive. Our mission is simple: to break down barriers and foster a world where communication knows no bounds.

What it does

Sign Atlas offers a unique and immersive learning experience through its interactive 3D visualization tool. Users can explore hand signs from various angles, enhancing their understanding and mastery of sign language gestures.
Leveraging cutting-edge gesture recognition technology, Sign Atlas enables users to practice sign language with real-time feedback. The platform utilizes computer vision to accurately track users' hand movements, providing immediate guidance to improve accuracy and fluency.
Sign Atlas provides a range of courses tailored to different skill levels and learning objectives. Each course is designed to be engaging, interactive, and accessible to users of all abilities.

How we built it

Languages: TypeScript, Python, HTML/CSS
Libraries: MediaPipe, TensorFlow, Mantine UI, ThreeJS (React Three Fiber), Mongoose
Database: MongoDB
Frameworks: NextJS, React
Deployment: Vercel

Challenges we ran into

Dependency hell with MediaPipe and Python
React state management and garbage disposal problems with ThreeJS
React Three Fiber type errors
Importing 3D models (STL) into ThreeJS
MongoDB database models
Vercel deployment with MongoDB not working
User authentication not working with Vercel deployment

Accomplishments that we're proud of

We're proud of the reliable efficiency of the gesture recognition model. It is smooth, consistent and transparent. Furthermore, we are proud to have successfully integrated 3D hand models with three.js to help facilitate the learning experience.