Mindcraft

Inspiration

After watching the Minecraft movie last week and spending all our money on Minecraft Happy Meals, Minecraft was all we could think about. During our brainstorming session, we realized many students struggle with mental health, often feeling ashamed or embarrassed to seek help due to stigma and fear of judgment. To address this, we combined our passion with this important issue to create Mindcraft, an app designed to reduce stigma by allowing users to have supportive video calls with family-friendly, lovable Minecraft characters.

What it does

Mindcraft immerses users in engaging, heartfelt conversations with their choice of friendly Minecraft pets. Each pet character embodies a unique role—such as a compassionate therapist, thoughtful philosopher, or uplifting motivator—to best match the user's emotional needs. During video calls, we leverage Hume AI to analyze users' voices, enabling the pets to deliver personalized, emotion-sensitive responses. After each session, users can generate an insightful summary of their interaction with the help of Gemini AI.

How we built it

The frontend of Mindcraft is built using Next.js with TypeScript for a smooth and scalable user experience. We integrated Hume AI with WebSockets to enable real-time emotion detection and voice-based interactions, while Gemini is used to generate summaries of each conversation. For visuals, we designed 3D models in Blender and rendered them in-browser using Three.js. To support remote development and access, we used ngrok for secure tunneling.

Challenges we ran into

Our biggest challenge was learning Blender and Three.js to render our 3D models, as none of us had prior experience with either tool. We also ran into difficulties crafting effective prompts to generate the character models we envisioned.

Accomplishments that we're proud of

We're very proud that we were able to create working models that you can talk to!

What we learned

We gained a ton of knowledge working with Blender, Three.js, and Hume AI throughout this project.

What's next for Mindcraft

Next for Mindcraft, we plan to expand the range of pet personalities and roles, giving users even more ways to connect based on their emotional needs. We also want to improve the realism of interactions by incorporating facial emotion recognition and more advanced voice synthesis. Eventually, we hope to bring Mindcraft to mobile platforms and make it accessible to schools and mental health organizations to help reduce stigma and encourage open conversations around mental well-being.