EyeExplore

Inspiration ✨

Exploration is more than just being called Dora and staying away from Swiper. Yet the struggle to truly live out this experience continues to be a struggle for those with disabilities. We wanted to build a project that helps the visually impaired explore the world. We wanted to be their eyes (hence the play on words in our name).

What it does 🤳

EyeExplore starts with a facial recognition login–a way for the visually impaired to log in without having to type out a password with the small letters on their phone. This information is stored in the Estuary database to be accessed in the future. It later brings them to a camera that detects surrounding obstructions with a text-to-speech function in order to read the names of these obstructions to know how to steer clear. This is done with the co:here API.

How we built it 🔧

Building off many different API’s, there were many unique technologies and frameworks we had to build around. For the back end, we used Face++ API for facial recognition sign-in, Estuary/Firebase for facial recognition storage, co:here API for text-to-speech features, and the Local Python server for the backend. For the front end, we used Figma and React Native first designing and then implementing the designs into an Expo app.

Challenges we ran into 🏃

We didn’t just run into challenges, the challenges chased us down. Whoever said it was a good idea for 4 first-years to build an almost functioning React Native app despite never having touched Native, must’ve been crazy (no one did by the way).

Halfway in, we realized the backend coders could not incorporate their code to React Native so we all had to scrap our old code and write new ones to be compatible with Native.

On the same note, we wanted to implement Tensorflow with Expo however they were incompatible and that resulted in our code not working out.

The Estuary database server also went down and we forgot to check the discord server for updates so we had to use firebase.

We struggled with “Possible unhandled promise rejection” and still don’t know how to solve this issue

JSON file syntax error which we did not find and that is what stopped us from getting further with our code. First, it was a broken promise but after fixing that error, we were unable to find the syntax error in all the JSON files. We were also too hyper-fixated on individual problems so it was hard to move on from the code we took a long time to build.

Accomplishments that we're proud of 🎉

Trying project ideas we have 0 experience in developing! Every hackathon we try to hack something new and time and time again we underestimate its difficulty. Halfway through this hackathon, we did lose a bit of hope, unsure if we were able to successfully develop the app by the deadline. However, we took a break, worked on our planning, and were able to voice out our concerns, resulting in us having something we were decently satisfied with (so close to full integration).

We are also very proud that we were able to get the app going in the first place as we, once again, had no experience with Native and we’re also new to fronted integration in general. To be able to even configure buttons, bars with input and output, and authentication is already crazy to us that we were able to figure it out.

What we learned 🌱

We learned that while aiming high is good, we lose sight of feasibility. We were definitely in a time crunch toward the end, especially after pretty much restarting the project halfway through. We were also able to work on managing databases, more API implementation, and debugging (especially with Native).

What's next for EyeExplore 🤩

Full integration of all parts. We all had the same clear vision for this project hence all parts of this project have been done by respective members. However, due to technical issues, we were unable to finish. We also hope to implement a translation feature for international users and a way to measure the distance between the person and the objects. The opportunities for EyeExplore are endless and we hope to finish our project in the near future!