Autism Assist

Social Confidence, One Cue at a Time: Paving a path for roleplays and ASD community support

Comment 1

Inspiration

Schools in our community are having a harder time providing adequate resources for developmental disorders. There is not enough support for autism, and we wanted to provide an additional resource to address this gap. We wanted to bring attention to the under development of the understanding of social cues in children with ASD. Clearly, our society is not teaching social cues in an effective manner; for instance, by using worksheets and outdated online programs.

Figuratively, you can think of the aforementioned as trying to learn how to drive a car by using pen and paper, rather than actually getting in the driver’s seat. One of our primary motivators in choosing this project was our team member’s experience with talking to a child with autism and learning about the various programs for children with ASD. We want to put children with autism in the driver's seat of their non-verbal social skills.

What it does

An app that uses technology to its highest, using the help of AI face recognition to help students learn social cues through role-playing them. Alongside that, we will have a special community feature where we curate special information regarding ASD and how to use therapy for ASD.

The solution is an innovative and unique mobile app tailored for autistic children, leveraging AI face and cue recognition to facilitate real-time learning of social cues through role-playing. This innovative approach is complemented by a dedicated community feature, providing curated information on ASD and therapeutic strategies. The app’s design prioritizes a calming and consistent user interface with clear instructions, utilizing soothing colors like green, blue, and pink, which are known to have a calming effect on those with autism. This ensures that the app is accessible and engaging for both children and their parents, making it a unique app for empowering students with autism.

How we built it

We used React Native as our frontend framework for the Autism Assist app. This allowed us to build for both iOS and Android at the same time. As the AI face detection model we were using was built for Python, we needed to build the backend using Python. Our team used Flask on Google Cloud Run for the Python backend.

Challenges we ran into

Nobody on our team had ever used Docker to deploy apps on Google Cloud Run. We needed to learn how to effectively deploy the backend to make our AI feature accessible to the end users. We were able to split up the deployment by having one team member build the Docker images and another team member set up Google Cloud Run. This technique allowed us to get through this challenge faster as we were able to collaborate on the problem.

Accomplishments that we're proud of

We are really proud of our app’s AI feature because we have never seen any computer vision model being used for the purpose of addressing a lack of resources in autism. We were able to create an effective way to detect social cues by using a computer vision model and applying a novel technique to the field.

What we learned

We learned a lot about app development in terms of frontend and backend technologies. Many of the team members had never used technologies like Flask and React Native, so it was a great learning experience in terms of learning new skills.Beyond the technical learning though, we learned how to collaborate and create solutions by using various perspectives. By combining team member’s inputs on the design and functionality of the app, we were able to create a very useful end result.

What's next for Autism Assist

We would like to expand our collection of community features and ques and make Autism Assist available to the public. Another one of our goals is discussing our app with child psychologist’s and special needs counselors at schools to find new features to add to the mobile app. We think that this app could really help a lot of children with ASD and we want to spread access to such a great tool.

Built by Sarthak Gupta, Aditya Mehta, Gaurika Gupta, Aarush Jugdar

Share this project:

Updates