-
The Sight iOS App
Inspiration
Many visually impaired individuals face challenges due to their dependence on others. They lack the ability to do things on their own and often miss out on the simple pleasures of life. We wanted to create an app that helps the visually impaired fully experience life -- on their own terms.
What it does
"Sight" is essentially a navigation app for the blind. It uses image recognition and ultrasonic sensors attached to an Arduino in order to help visually impaired people to "see" around them. It also provides audio-based directions to the user.
How we built it
Our team first designed visual and audio based interfaces with a visually-impaired audience in mind. We then 3D-printed an iPhone case which was designed to hold an Arduino board/ultrasonic sensor. Then, our team developed an image recognition model using Apple's CoreML. Lastly, the model was implemented in the iOS application and any remaining flaws were removed.
Challenges we ran into
The main issue our team ran into regarded implementing our ML models in the final iOS application. We had a TensorFlow model already trained, however, our team was not able to use this model in the final application.
Accomplishments that we're proud of
Although our team was not able to use the TensorFlow model as initially planned, we were able to come up with an alternate solution that worked. Our team is proud that we were able to come up with a working app that has potential to impact the modern world.
What we learned
Our team primarily learned how to complement iOS development and ML models and how Arduino/iOS compatibility functions.
What's next for Sight
Moving forward, our team needs to improve Sight's image-recognition software. We created our own dataset since we lacked the computing power to use larger, more official datasets. We used CoreML's pre-trained MobileNet model. Although the model works to an extent, the safety of the blind individual is paramount and improving the image-recognition software directly benefits the safety of the user.
Log in or sign up for Devpost to join the conversation.