LinguaScan

Real time object detection
Translation

Inspiration

We believe the best way to learn a new language is to immerse yourself in it. That’s why we created LinguaScan. Through learning translations of everyday objects and places, you’ll pick up a new language in no time!

What it does

Lingua Scan translates virtually anything around you! Scan anything with the app’s camera and it’ll use trained machine learning models to recognize the object, provide a translation, and even provide a short description of it.

But what’s learning a new language without being able to reinforce your learning? In the future, LinguaScan will also record every object you’ve seen the translation of and test your knowledge by chatting with a personalized chatbot that crafts conversations based on your knowledge of the language so far. You can also upload pictures of your travels and learn even more through interactive learning.

How we built it

We built this app using a frontend in Flutter and the services of Authentication and Firestore from Google Firebase. We implemented OpenCV library with real time translation integration.

Challenges we ran into

One of the major challenges that we ran into was Authentication with Firebase and Flutter. Much of the documentation was not in sync with the most recent versions of Flutter and Firebase, causing a great deal of confusion. At the end, we were able to use basic email authentication. In the future, we would like to seamlessly integrate Google authentication and other methods as well. Additionally, we had issues trying to generalize translation to specific objects.

Accomplishments that we're proud of

We are all proud of learning about Flutter and using new technologies. We are also proud of being able to submit a project amidst our busy schedules and the stresses of the school year.

What we learned

Since most of us didn’t have experience in Firebase or Flutter, we spent most of the time perusing documentation and guides on how the 2 frameworks and technologies were intertwined and integrated together. We really enjoyed the experience at HackMIT and would definitely do this experience again. We also learned about the architecture for a computer vision based project.

What's next for LinguaScan

We hope to bring LinguaScan past the MVP stage and revamp the feature set that LinguaScan offers as well. Future feature updates are planned to include a speech-to-text feature that tests the user’s proficiency in a certain language. We are excited about the future of LinguaScan!