Aspectum

• 

  Homepage

• 

  Inserting phone

• 

  Image recognition

• 

  Color transform

Inspiration

Colorblind and visually impaired people face daily challenges, from reading signs to recognizing objects around them. We wanted to create a solution that doesn’t just tell you what is there, but enhances their perception of real time, making environments more understandable and accessible for them.

What it does

Aspectum uses real-time video processing and VR/AR display to help colorblind and blind users understand their surroundings. For colorblind users, it corrects colors and increases perception. For blind users, it identifies objects and announces them through audio feedback. The result is an intuitive and low-cost system that makes visual information actionable and accessible for the visionally impaired.

How we built it

Color Correction: implemented advanced HSV-based true inverse algorithms to enhance color perception for different types of colorblindness, Object Recognition: used Gemini’s API to recognize multiple object types in real-time, VR/AR Display: shown through Google Cardboard-style VR to in the corrected visual environment, Audio Feedback: pressing a button triggers spoken object descriptions using a multi-threaded TTS system. Hardware: ran on phones and low-cost cardboard headsets, keeping accessibility and deployment simple.

Challenges we ran into

Some challenges included real time integration with the hsv correcting and shifting algorithm, balancing the VR with the algorithm, and designing a UX for both colorblind and blind users.

Accomplishments that we're proud of

We successfully integrated color correction, object detection, and audio feedback into a single system that runs on low cost and easy to get hardware, demonstrated smooth, real-time video feed in VR, showing low latency and comfort for users, and developed a solution that directly improves accessibility for colorblind and blind people.

What we learned

Some things we learnt were visual and audio solutions required careful UX, real world conditions like lighting affecting the algorithm's performance, a balance of speed and accuracy, and integrating AI with hardware and CV efficiently.

What's next for Aspectum

Some of the next steps for Aspectum include object detection for a wider range and if possible, offline, stronger physical model for the application, and AI enhancements for depth perception.

Built With

• ar
• c++
• css
• cv
• html
• javascript
• numpy
• python