DinoMind

Students are often swamped with work, and stressed each week. It's not easy to let those feelings out to others, but what about a cute Dino, or 3 of them? Meet DinoMind, your new audio journal buddy!

Comment

Inspiration

As students, we understand the stress that builds up in our lives. Furthermore, we know how important it is to reflect on the day and plan how to improve for tomorrow. It might be daunting to look for help from others, but wouldn't an adorable dino be the perfect creature to talk to your day about? Cuteness has been scientifically proven to increase happiness, and our cute dino will always be there to cheer you up! We want students to have a way to talk about their day, get cheered up by a cute little dino buddy, and have suggestions on what to focus on tomorrow to increase their productivity! DinoMind is your mental health dino companion to improve your mental wellbeing!

What it does

DinoMind uses the power of ML models, LLMs, and of course, cute dinos (courtesy of DeltaHacks of course <3)!! Begin your evening by opening DinoMind and clicking the Record button, and tell your little dino friend all about your day! A speech-to-text model will transcribe your words, and save the journal entry in the "History" tab. We then use an LLM to summarize your day for you in easy to digest bullet points, allowing you to reflect on what you accomplished. The LLM then creates action items for tomorrow, allowing you to plan ahead and have some sweet AI-aided productivity! Finally, your dino friend gives you an encouraging message if they notice you're feeling a bit down thanks to our sentiment analysis model!

How we built it

Cloudflare was our go-to for AI/ML models. These model types used were:

  1. Text generation
  2. Speech-to-text
  3. Text classification (in our case, it was effectively used for sentiment analysis) We used their AI Rest API, and luckily the free plan allowed for lots of requests!

Expo was the framework we used for our front-end, since we wanted some extra oomph to our react native application.

Challenges we ran into

A small challenge was that we really really wanted to use the Deltahacks dino mascots for this year in our application (they're just so cute!!). But there wasn't anything with each one individually online, so we realized we could take photos of the shirts and extra images of the dinos from that!!

As for the biggest challenges, that was integrating our Cloudflare requests with the front-end. We had our Cloudflare models working fully with test cases too! But once we used the recording capabilities of react native and tried sending that to our speech-to-text model, everything broke. We spent far too long adding console.log statements everywhere, checking the types of the variables, the data inside, hoping somewhere we'd see what the difference was in the input from our test cases and the recorded data. That was easily our biggest bottleneck, because once we moved past it, we had the string data from what the user said and were able to send it to all of our Cloudflare models.

Accomplishments that we're proud of

We are extremely proud of our brainstorming process, as this was easily one of the most enjoyable parts of the hackathon. We were able to bring our ideas from 10+ to 3, and then developed these 3 ideas until we decided that the mental health focused journaling app seemed the most impactful, especially when mental health is so important.

We are also proud of our ability to integrate multiple AI/ML models into our application, giving each one a unique and impactful purpose that leads to the betterment of the user's productivity and mental wellbeing. Furthermore, majority of the team had never used AI/ML models in an application before, so seeing their capabilities and integrating them into a final product was extremely exciting!

Finally, our perseverance and dedication to the project carried us through all the hard times, debugging, and sleepless night (singular, because luckily for our sleep deprived bodies, this wasn't a 2 night hackathon). We are so proud to present the fruits of our labour and dedication to improving the mental health of students just like us.

What we learned

We learned that even though every experience we've had shows us how hard integrating the back-end with the front-end can be, nothing ever makes it easier. However, your attitude towards the challenge can make dealing with it infinitely easier, and enables you to create the best product possible.

We also learned a lot about integrating different frameworks and the conflicts than can arise. For example, did you know that using expo (and by extension, react native), you make it impossible to use certain packages?? We wanted to use the fs package for our file systems, but it was impossible! Instead, we needed to use the FileSystem from expo-file-system :sob:

Finally, we learned about Cloudflare and Expo since we'd never used those technologies before!

What's next for DinoMind

One of the biggest user-friendly additions to any LLM response is streaming, and DinoMind is no different. Even ChatGPT isn't always that fast at responding, but it looks a lot faster when you see each word as it's produced! Integrating streaming into our responses would make it a more seamless experience for users as they are able to immediately see a response and read along as it is generated.

DinoMind also needs a lot of work in finding mental health resources from professionals in the field that we didn't have access to during the hackathon weekend. With mental wellbeing at the forefront of our design, we need to ensure we have professional advice to deliver the best product possible!

Share this project:

Updates