Objectives of the Project:

Android app for the project using Android Studio and building the Yolo-Object detection model which will be responsible for helping the blind to search and find out objects around them through live video capture. An interface interactive with a voice that guides them to navigate throughout the application. A virtual assistance using the concepts of Natural Language processing that can listen to the user and speak to the user and help him to do multiple tasks that a normal person can do. Sending emails through voice command, make calls, send SMS, cracking jokes for the user, open sites such as Google, Wikipedia, YouTube, take notes using voice notes, play music and a lot more A module that helps the blind to read text from any source. e.g: read books/ medicine names etc.

Tools and Technologies Used:

Android Studio for app development: XML for frontend and Java for Backend Image processing and Computer Vision Yolo Object Detection and deep neural network Optical Character Recognition Natural Language Processing using Python Voice recognition API's like JavaMail API, SMS Manager API

Motivation of the Project:

As we already know, visually impaired people have to be dependent on the people around them for their everyday needs. They cannot comprehend things around them without being told or without touching them, cannot read text written somewhere without it being read to them by someone, and they won't be able to operate a phone. If no one is available to them, it makes them helpless. Hence, the objective of the project is to develop a product like an android app that could help the visually impaired people to overcome all the above-mentioned problems that they face in their daily life. Visually impaired people with the help of our app can become more self-reliant and it will make their life a lot simpler. They will be able to identify objects around them, find the object they need, read the text in an object, make calls to number, send emails, SMS, know the battery level of their device, current time and operate a phone easily through voice assistance; without the presence of a live person with them.

Individual Contributions:

The entire project involves emerging technologies like Deep Learning, Image Processing and Computer Vision, Natural Language Processing and App Development. Both of us have equally contributed to the domain we are skilled. We planned together to make this project - a successful product for society.

Hariharan-

I contributed in developing an android app for the project using Android Studio and building the Yolo-Object detection model which will be responsible for helping the blind to search and find out objects around them through live video capture. I contributed to developing the module of the project responsible for helping the blind to read text from any source. Image to text and text to speech will be the two main features I will be learning and developing. This module that I am working will make blind to read books/ medicine names etc.

Nikitha-

I was responsible for making virtual assistance using the concepts of Natural Language processing that can listen to the user and speak to the user and help him to do multiple tasks that a normal person can do. Along with this, I contributed for modules responsible for making calls, sending SMS, knowing the battery level, and the current time.

Both of us worked on UI design of the app and once the app is made, we ensured that the app is perfect for all visually impaired people to use easily.

Challenges we faced while developing this Project:

We both have explored Deep Learning, Image processing and App development separately as two different domains. This is our first project implementing image processing in app development to help blind to detect and find out the object around us. Learning the requirements and implementing them is the main challenge we faced along with the dependencies and Gradle issues in android studio. At the end of this hack, we are glad that we came up with the best final product for the blind as we imagined. Thanks to the Team for organizing this hack which served as a motivation to come up with our ideas.

Future Scope of the Project:

A smart stick can also be developed in the future, in which, a camera will be present in a stick and the first three functionalities mentioned in the contributions of the project (mentioned later) can be included, i.e., the application has a scope of evolution in the future. Smart spectacles, similar in function to the smart stick can also be developed in the future, in which, a camera will be present in the specs. Again, object identification and finding, text recognition and dictation will all be enabled.