VisionMate

Wallet detection

Inspiration

The daily struggles of being a visually impaired person inspired us to create this app. Visually impaired people may find the task of locating lost items incredibly difficult compared to unimpaired people. We wanted to give visually impaired people the ability to locate lost items as efficiently and easily as possible. This project is the second version of our original app, which was an American Sign Language interpreter. Although the original product was functioning, due to time constraints, we decided to pivot to this idea instead.

What it does

Our app uses the camera to capture images of the user's surroundings. It then feeds the images into an image classification model that determines if a given object is in the image or not. If the given object is detected, it notifies the user by vibrating and emitting a beeping noise.

How we built it

We built it with Apple's Vision framework and Apple's CoreML framework. The Vision framework was used to manage interactions with the camera and the CoreML framework was used to classify the images from the camera. We built the image classification model through CustomVision, a cloud-based machine learning service provided by Microsoft. We integrated this model into our app as a CoreML model.

Challenges we ran into

Some challenges we ran into were issues with the machine learning model. Its accuracy was poor at first due to the lack of diversity in the provided datsets. However, we managed to improve its accuracy by providing the model with more diverse datasets.

Accomplishments that we're proud of

Some accomplishments that we're proud of is how we improved the machine learning model and how we managed to utilize the classifications made by the model to give notifications to the user.

What we learned

We learned about the importance of good data in training machine learning models. We also learned about how to integrate machine learning models into applications.