LangBot

Inspiration

All of us in the team lived in a bilingual family. Growing up in first generation households, we were able to understand our parents' language, but we couldn't speak our thoughts in the language we were raised under. So, our team decided to develop a machine that could simulate this experience for users. We start off with English and Spanish words into its memory and translate Spanish to English if the word exists in the memory database.

What it does

Our team's goal was to develop a machine learning bot imitating a person's brain as they grow up. The bot can learn, and store learned foreign terms into a text file serving as a database using the front camera to scan a word. This machine is able to translate all English terms to Spanish, but it can't translate from Spanish to English unless the word is stored in the text file.

How we built it

For the backend side, we used the OpenCV library to capture text through the front camera of the user's laptop. Once the device reads the text, it will translate through an open translation API and print the English and Spanish translation into a text file. Each time the capture text will look for similar Spanish term in the text file if it exists. If the word does not exist, our program would return "me no understand English". For the frontend side, we used HTML, CSS, and a little JavaScript to design the website, and then we used Flask attempting to connect the backend to the frontend.

Challenges we ran into

This was our very first hackathon as a team, so we don't have that much experience with this event. We were trying to do deep learning of the image capturing feature with OpenCV library in order to improve the accuracy of the text through the camera, but we did not have that much time to develop a deep learning model. We also learned and used OpenCV for the first time, incorporating the use of a webcam. We also didn't have a lot of experience with front-end development so it was a steep learning curve for us to design the website and connect it to the backend through flask (which we weren't able to successfully do).

Accomplishments that we're proud of

This is our first non-school project as a team. We were facing a lot of challenges through our project, but we were proud of making our algorithm work during this project. We were also so proud of staying in the hackathon live event place for 16 hours and watching the sunrise through the window.

What we learned

We delved further in openCV and flask. We learned how to stay consistent with various coding languages and we learned how to undergo the full stack development process. We learned that a lot of research must be done before, throughout, and even after the product has been created. At the end, we were trying to implement our algorithm into html using Flask, but we did not have enough time for that. We would definitely look into that topic after this hackathon.

What's next for sample

Looking ahead, we plan to further develop our bot and optimize it to make it more efficient and smoother. We want to successfully implement the python script into our website server through Flask, and we want to continue building our website, so it is more polished and more user friendly.