code_snippet_reader - Brown Munde

extracted text from the code snippet along with the audio file
image upload webpage

Inspiration

One of our friends at the university is visually impaired and hence has accessibility issues with some of the online quizzes. Like the ones containing screenshots of python code and questions based on it. The screen reader software he uses cannot read text from the image.

What it does

Extracts code from code snippets and converts it into speech.

How we built it

We came up with this website where one can upload code screenshots from an ide and extract the text from it. The text is extracted through python libraries like OpenCV and pytesseract. We pass this text through Google's text-to-speech api using python and a library called gtts and generate an audio file. The audio file is then uploaded onto the website using the django framework where one can play it.

Challenges we ran into

Finding a better logic to accurately detect text and extract it from the code snippet. Receiving the image from website users on our host, processing it and uploading the generated audio file. Deploying it on Heroku and pythonanywhere.

Accomplishments that we're proud of

Being moderately accurate in extracting text from the code snippet. We got to learn so much about image processing libraries python has to offer such as OpenCV and pytesseract. Able to build a basic website that can receive image input from a user, process it and upload an audio file that can be played.

What we learned

We learnt the basics of image processing OpenCV. We learnt about threshold values for the different aspects of an image such as the values of its individual elements on the rgb and hsv colour space. We learnt how to dissect elements from the image by setting threshold values of the text in the snippet. We also learnt the basics of managing the backend of a site using the django framework and how to play audio in a webpage using html5