SEMA

SEMA means “speak” in Swahili, our mother tongue. It reflects our mission to give every child a voice by using responsible AI to make speech therapy accessible, inclusive, and human.

Comment

Inspiration

Our inspiration comes directly from hands-on experience in speech therapy at a non-profit rehabilitation center serving children with autism, cerebral palsy, and other neurological and physical conditions. One of our team members volunteered and assisted specifically in speech therapy sessions, working closely with children who were learning to communicate for the first time or regain lost speech abilities.

Despite the dedication of therapists, progress was often slow not because of a lack of effort, but because therapy did not continue beyond the clinic. Many children had little to no speech practice at home. The families served by the center were primarily low-income, and parents often lacked the training, time, or resources to support speech exercises outside of therapy sessions.

This experience made it clear that speech therapy cannot succeed in isolation. Children need frequent, engaging practice, and families need tools that are simple, affordable, and accessible. Witnessing these challenges firsthand motivated us to build a solution that extends speech therapy into the home, supports caregivers, and helps children practice communication consistently regardless of their family’s financial situation.

What it does

SEMA is a game-based speech therapy support app that helps children practice pronunciation at home in a fun, adaptive, and privacy-first way.

A child sees a word, hears or reads how it is used, and then speaks it out loud. The app analyzes pronunciation in real time, identifies specific sounds or syllables that need improvement, and provides friendly, encouraging feedback rather than labeling responses as “wrong.” For example, with words like “butterfly,” the app highlights which sounds need more practice while celebrating effort.

Speech audio is processed briefly and never stored, protecting child privacy. Based on performance, the system adapts difficulty and recommends targeted practice. The app is designed to support both children, parents and caregivers, with progress insights that can also assist therapists.

How we built it

We built the app by splitting it into the front end and back end. The front-end included the design of the app and splitting it into different views that depict the game.

In the front end, we used Swift to develop an app that works on an iOS system. The decision to use Swift was motivated by the fact that if we were to role out this app, we would start in the USA and the greater part of the population uses iOS over Android systems. To build this, we split the different tabs shown in our app into different Swift views and added additional swift views for other animated concepts. The animated Concepts builds into a stronger UI that appeals to children and gives them the game vibe that we were going for.

We built SEMA’s backend as a privacy-first, AI-powered system designed specifically for children. The iOS app connects to a FastAPI (Python) REST API that analyzes speech and returns real-time, supportive pronunciation feedback.

Speech audio is processed using PocketSphinx, a free, offline speech recognition engine. This keeps costs at $0/month and ensures audio is never stored or sent to external servers. The backend breaks speech into phonemes, assigns confidence-based scores, and compares them to expected sounds while respecting dialect and accent variations.

Feedback is generated based on confidence: high-confidence results receive helpful articulation tips, while lower-confidence cases return only positive encouragement to avoid misleading or discouraging feedback.

An explicit ethics layer (ethics.py) ensures all feedback is supportive, child-safe, and never harmful. We store only abstract progress metrics—never raw audio or personal data—and all audio is deleted immediately after processing. This makes SEMA fast, affordable, inclusive, and safe for children.

Challenges we ran into

Our challenges were both technical and non-technical. On the technical side, combining the front end and backend was a challenge since we had different developers work on each section and then try and combine it. We ran into a lot of compatibility issues.

On the non-technical side, we had project scheduling issue. The project had to be done over the break and we all had different travel timelines and vacation dates which made it difficult to effectively work on our ideas together. Although we delegated tasks effectively and everyone worked on their piece, the timing of our project and that coordination was really hard

Accomplishments that we're proud of

What we learned

Strong collaboration and communication skills

Effective task delegation across a distributed team

Technical skills in Swift, Python, and backend–frontend integration

The importance of designing AI systems with care, especially for children

What's next for SEMA

Next, we plan to expand development through user testing and feedback from therapists and caregivers, improve adaptive learning features, and explore broader platform support. Our long-term vision is to make SEMA a trusted, affordable companion to speech therapy helping every child find their voice

Built With

Share this project:

Updates