AI & Machine Learning Track

Inspiration

We noticed a massive gap in digital accessibility: while AI is booming, students with hearing impairments are still struggling with auto-captions that lack context or tone. We wanted to move "Beyond Limits" by building a tool that doesn't just transcribe words, but translates the experience of learning into a visual, interactive format.

What it does

Our project, EduBridge AI, is a real-time learning assistant. It uses AI to ingest live lecture audio and instantly generates: 3D Sign Language Avatars: A virtual interpreter that signs the lecture. Contextual Summaries: Breaking down complex jargon into simple, digestible bullet points. Visual Mind-Maps: Automatically mapping out the lecture’s main concepts for visual learners.

How we built it

The core engine is powered by Gemini 1.5 Flash for high-speed audio-to-text and contextual analysis. We used Python and TensorFlow to map text to sign-language gestures. The frontend is built with React, and we utilized Three.js to render the 3D avatar animations directly in the browser.

Challenges we ran into

The biggest hurdle was latency. Translating audio to sign language in real-time requires immense processing power. We had to optimize our API calls and use "Streaming Data" techniques to ensure the avatar’s movements didn't lag behind the professor's voice. We also struggled with technical jargon (like Calculus or Physics terms) that doesn't have a direct 1-to-1 sign language translation. Accomplishments that we're proud of We successfully reduced the processing lag to under 1.5 seconds, making it feel like a live conversation. We are also proud of our "Jargon-Simplifier" feature, which detects when a student is confused and offers a simpler explanation of the topic on the fly.

What we learned

We learned that AI isn't just about automation; it's about empathy. Building for accessibility taught us to think about UI/UX from a completely different perspective. Technically, we deepened our knowledge of Natural Language Processing (NLP) and real-time data streaming.

What's next for AI & Machine Learning Track

The next step is to expand the sign language library to include different regional dialects (ASL, BSL, etc.). We also plan to integrate Computer Vision so the student can sign back to the AI to ask questions, creating a truly two-way, inclusive educational environment.

Built With

• ai-agent
• api-key
• app.js
• assets-folder
• automation
• capcut-editing
• cloud
• computer-vision
• core
• css
• devpost-submission
• engineering-safety
• error-detection
• flask
• gemini-live-api
• gemini-pro
• github-repo
• google-cloud
• html
• iso-standards
• javascript
• languages
• machine-learning
• main.py
• mit-license
• multimodal-reasoning
• opencv
• python
• react
• readme.md
• real-time-monitoring
• requirements.txt
• robotics
• spatial-analysis
• sub-second-latency
• system-architecture
• vertex-ai
• video-demo
• vision-ai