TalkTiles

Inspiration

One of our team member's family member's is non verbal. Growing up they always had communication boards to communicate their needs. Communications boards have been around for a long time. However, these boards are typically made of paper so a caregiver or guardian has to be next to the individual to understand what they need or want. Talk tiles takes this retro technology and elevates it by sending alerts to a caregiver or guardian, and also says their need aloud.

What it does

First, create or log in to your TalkTiles account. You’ll be prompted to choose the tiles you want on your communication board. Once your board is set up, simply tap a tile to trigger an email notification to the caregiver, and the system will speak the selected tile aloud for additional support.

How we built it

We built TalkTiles using a full-stack MERN architecture. The backend uses Node.js, Express, and MongoDB to manage users, tiles, and communication board configurations. We implemented authentication, email notifications with Nodemailer with Gmail’s SMTP API, and text-to-speech features using the ElevenLabs API. On the frontend, we built an accessible React interface where users can select tiles and interact with their personalized board; this followed WCAG 3.1.5 and several WCAG 2.1 AA accessibility criteria to hone our project to those with limited dexterity, low-vision, or cognitive disabilities. We also integrated real-time visual and audio feedback to enhance communication for nonverbal users.

Challenges we ran into

We initially wanted to send SMS to the caregivers, but realized this would take a few weeks to receive approval; so we pivoted to sending email notifications. We also faced challenges with saving the users' selected tiles and displaying them in the next page or after login.

Accomplishments that we're proud of

We are incredibly proud of our finished product and the meaningful impact it can have in healthcare for the non-verbal and AAC community. Throughout the hackathon, both team members stepped far outside their comfort zones—learning new tools and collaborating across backend and frontend. Together, we built a fully functioning communication board that empowers individuals who cannot speak to express their needs independently.

What we learned

Our team learned how to create API endpoints and integrate them with React frontend, and best practices for augmentative/alternative communication (AAC) apps. We gained experience working with assistive technologies to support users with non-verbal communication needs.

What's next for TalkTiles

Next, TalkTiles will need an "edit" feature to add or remove tiles from the communication board. We would also like to implement SMS notifications to caregivers for quicker support.

Built With

• express.js
• mongodb
• node.js
• react