MathMatrixMovies

Workflow Screenshot
UX Screenshot

Inspiration

We were inspired by the challenge of making math education accessible, engaging, and personalized for students and educators alike. We wanted to create a tool that could break down complex mathematical concepts into easy-to-understand videos, tailored to different age groups and learning levels.

What it does

MathMatrixMovies generates engaging, animated math explainer videos from simple text prompts. Users input a prompt, select the target age range (3-18, undergraduate, or graduate), and choose the language (English, Hindi, Tamil, Spanish, etc.). The tool then creates a high-quality, personalized math video using Google's Gemini Pro 1.5 and Manim.

How we built it

We leveraged Google's Gemini Pro 1.5 to generate object-oriented Manim code from text prompts. The code is debugged until it compiles successfully. The rendered video is split into 1-second frames, which are processed by Gemini Pro 1.5 for a second pass to refine the content. The frontend platform was built using Streamlit and hosted on an Azure Virtual Machine. We also used Meta's Llama3 on Groq for generating compelling titles, descriptions, tags, categories, and metadata for YouTube uploads. We are also curating a dataset of input prompts and output Manim code for published Manim videos so we can train an open weight or open source model such as Llama3 to generate the Manim code in the future.

Challenges we ran into

One of the main challenges was ensuring the generated Manim code was syntactically correct and produced the desired video output. We overcame this by implementing a robust debugging system that iteratively refined the code until it compiled successfully. Another challenge was working around the rate limits of Google Gemini Pro 1.5 and managing the usage costs. As mentioned before, we intend to train an Open Weight or Open Source model like Llama3 to perform the video codegen task and are curating a dataset for this purpose.

Accomplishments that we're proud of

We successfully created a tool that generates high-quality, personalized math explainer videos from simple text prompts. The tool has the potential to revolutionize math education by making complex concepts accessible and engaging for learners of all ages and skill levels.

What we learned

We learned the importance of user testing and feedback in shaping the platform's development. We also gained valuable experience in working with large language models like Google's Gemini Pro 1.5 and Meta's Llama3, as well as the Manim animation library.

What's next for MathMatrixMovies

We're actively collecting prompts and interim code to build a fine-tuning dataset that will further enhance the system's capabilities and make it more open by running on an open weight model like Llama3. We're making our codebase, methodology, and finetuning dataset publicly available to foster transparency and collaboration within the AI and education communities.

You can find our project here: https://math.auto.movie Blog Post: https://medium.com/@aditya-advani/deep-dive-creating-accurate-math-videos-using-llms-02db22ba1d2d Github: https://github.com/LilySu/MathMatrixMovies