pennapps-xv-project

Usage

To open gui and draw

$ python start.py

Inspiration

We have specifically designed it for education purpose. For a case, sometimes it slightly challenging to understand the writing of autistic children. In that case, our application can simplify the process of understanding their work.

Location

We are located on the third floor lobby in front to Room 321.

What it does

Allows a user to draw an image and we interpret what the drawing is, based on the data sets available.

How we built it

Using OpenCV, we provide interface to an user to draw. We used Python to render the training data sets into pickles. Pickles helped in serialization and we can directly load the training data from memory, which makes the program fast in execution. We then apply Machine learning technique (specifically KNN) to find out the top N images similar to the user input.

Challenges we ran into

The edge detection technique was initially using dots. A human mind connects the dots to draw a line. So, we converted the dots into squares and made them overlap to make sure that a concrete line is drawn to be considered as edge. Another challenge was the execution time of the application. We initially stored the training datas as the converted 2-D binary array list. This process needed to be done everytime we execute the program. So, we decide to store the data sets as pickle, which can load the data from memory saving the time.

What we learned Color filters are the absolute worst thing ever. I suggest things like keypoint matching even in a time constraint. Color filters can completely freak out by the slightest change in lighting condition, both sun and lamp.

What's next for Draw

Modifying the input to take RGB colors. We can use Leap motion which can sense your hand motion and gives much accurate input data.