MathSpeech

Inspiration

As CASA tutoring center tutors, we noticed that sometimes there aren’t enough tutors to support every student—often one tutor has to assist three or four students at once. We also lack the resources to cover every course, and many students in introductory classes struggle due to weak foundations, making sessions even more time-consuming.

That’s why we built MathSpeech — an AI-powered tutor that can explain, solve, teach, and patiently walk students through problems just like a real tutor.

What it does

Solves handwritten math problems.
Explains written math questions from images.
Generates quizzes, practice tests, and offers a conversational AI that walks students step-by-step with grading based on their answers.
Reads PDFs, Docs, and text files, storing context for RAG retrieval to ensure personalized and relevant help based on students’ materials.

How we built it

MongoDB stores document embeddings generated by Gemini embedding models for vector search.
Gemini 2.5 Flash handles content generation, quizzes, practice tests, and conversational tutoring, integrated with Mathpix OCR API for better image comprehension.
Mathpix PDF Processing API converts PDFs and Docs into LaTeX for better parsing and embedding.
ElevenLabs powers the text-to-speech component for a natural, conversational tutoring experience.

Challenges we ran into

Compatibility issues with Gemini models when mixing multiple input types such as images, LaTeX, and audio.
Parsing complex math PDFs without Mathpix used a large amount of context due to heavy LaTeX formatting.

Accomplishments that we're proud of

In just 24 hours, we built a fully functional prototype with all the core features working smoothly — something we can realistically implement in our tutoring center.

What we learned

We learned how to effectively delegate tasks within a short sprint for a relatively complex project. We explored new technologies like React, RAG pipelines, Mathpix API, 11 Labs, and Gemini — discovering how they can work together to create a powerful educational AI tool.

What's next for MathSpeech

Despite being a hackathon project, we are actively working to implement MathSpeech in our tutoring center and scale it up beyond just math.
Our next steps include:

Expanding subject coverage to areas like physics, chemistry, and computer science.
Adding real-time comprehension tracking to analyze students’ handwritten or typed responses for adaptive feedback.
Optimizing system performance to significantly boost response speed and reduce latency.
Continuing to refine the conversational AI for a smoother, more human-like tutoring experience.

Made with ❤️ by CASA Tutors.