Aspen

• 

  Mira displaying the letter V in ASL on the webcam

• 

  The app's home page

Inspiration

We recognized that many deaf people may have trouble communicating with others, especially if either they or the person they're talking with don't know ASL. The same goes with mute people, where communication could be awkward. We wanted to create an application that would enable these different kinds of people to communicate with each other. This would be useful in many different settings such as in the workplace, schools, and even just in public. We also wanted to create a user-friendly interface for people with various disabilities such as physical/motor disabilities, autism, and being Deaf, so our initial interface design reflects that.

What it does

Aspen uses the user's camera or webcam to detect the signs that a person would be holding up. Then it outputs this as text. It also has a speech to text feature, which could further aid in the communication of a deaf and a hearing person in the event that the camera does not work. It also has a text to speech feature, so that people with language impairments can communicate effectively.

How we built it

Aspen uses a TensorFlow model that is trained in all ASL letters and some others such as "delete"; this allowed us to and this would potentially output as English text. For speech to text and text to speech, we used the speech API that comes standard on computers.

Challenges we ran into

We ran into several challenges, which led to us having to completely restart our project from scratch several times. One error we ran into was an issue with our model where it was incorrectly recognizing ASL signs, which meant that we had to start over.

Accomplishments that we're proud of

One thing that we are proud of is that our initial UI design was very visually pleasing and clean. We are also proud of the fact that we were able to turn around and significantly improve our project from where we had previously had to start from square one, even though some features didn't end up working.

What we learned

We learned that it would probably be best to have a rock-solid plan and resources when we create this, as well as trying to collaborate more than arguing.

What's next for Aspen

We would probably want to try and implement our initial UI/X, as well as features such as creating an account to sign in and multiple kinds of interfaces based on user preferences.

Built With

• css
• github
• html
• javascript
• tensorflow
• vscode
• webkit