-
-
-
Hand recognition
-
-
Forks Up
Inspiration
Autism is one of the fastest growing mental disabilities in the world today. So we talked to someone who knew more about this and turns out she is an A.B.A, which is an Applied Behavior Analysis. An Applied Behavior Analysis teaches and helps kids with autism fit in to our society by using special teaching styles that normally wouldn't be necessary for kids in the regular classroom. Families come to them in tears of joy after they have worked with their child because they have seen what an A.B.A can do to help them. Their work and dedication has inspired us to reach out to an A.B.A and asked them what can we do to help them. Which is what we did.
What it does
This basic version of what we want to do will create a controlled room which will have objects that a child can point at. once the object is pointed at, it will respond with what the object is and a description of the object to help the child under stand the object better. This device will implement NET (natural environment teaching) and other types of teaching methods used by A.B.As to help the child learn in a safe environment. Since A.B.As have to go out into the world (parks,malls,stores) to implement NET, this device will allow them to do that in the safety of their own home.
How we built it
We used the unity engine to create the controlled environment and the leap motion to detect the child's hands. and allow the child to point at the objects. we linked it to google's text to speech technology and that way it will read the object's description to the child.
Challenges we ran into
there were many challenges we ran into. Due to our limited skill set and time, we we're having trouble implementing google's TTS API and getting the leap motion to work. After that, there was having unity respond to pointing and reacting to it was a challenge too.
Accomplishments that we're proud of
Getting the hand tracking in unity and getting the pointing done was a big step for us. But over all, getting one step closer towards getting something done for people with speech disorder is what we're all really proud of.
What we learned
We learned so many things researching the topic of autism and what an A.B.A does. We learned about cool technologies, like the leap motion, that we didn't know about that are out there for people to mess around.
What's next for A.B.A.A.R
Our main goal is to make NET (Natural Environment Teaching) and the other teaching techniques used by A.B.A as realistic as possible. So the next step is would be to implement the same concept into an augmented reality head set that could do the same thing but in the real world, hence natural environment teaching. We would implement the same concept into it and get it to recognize real world objects. if the object isn't recognized then a parent or teacher could add it to it's cloud data base where all headsets are linked and updated so that the head set is always set for anything that pops in it's way.
Log in or sign up for Devpost to join the conversation.