Inspiration
Have you ever been sitting at your desk doing your work, when you get an email, surprise! Your deadline got moved to tomorrow! All of a sudden, your heartrate spikes, your breathes get shallow, and you are very very stressed. In these stressful situations, many people completely lose focus of themselves and stay stressed and working for excessive periods of time. This is unhealthy, and actually damages your ability to be productive in the long run.
So naturally, we made a Wellness Cowboy coach that runs on an Amazon Echo dot to help you destress and stay productive and healthy while you work!
(We were also inspired by the cowboy theme )
What it does
Wellness Cowboy is an app which integrates with your Amazon Echo Dot to keep you destressed as you work! First, Wellness Cowboy uses computer vision through the webcam to determine a 'stress index.' This stress index is then sent over to the Echo Dot, where it is compared to a threshold to determine if the individual is stressed or just working hard! If the threshold is reached, the Echo Dot begins Coach mode. In Coach mode, the Echo Dot takes over your computer, forcibly removing you from the stressful environment, and chooses a calming exercise for you to use to destress! The Echo Dot displays a tutorial of the exercise on your screen, narrates instructions, and awaits for you to finish your destressing! Once the exercise is complete, the Dot waits to be called again by a high stress index.
And of course, it talks like a Cowboy!
How we built it
The Backend was built in two main components. The computer vision system operates by taking a scan of the user's face every 10 seconds, sending it to an AI model to produce a stress index, and producing a voice simulation to trigger the Amazon echo dot to activate.
Once the Echo Dot activates, it is fed the stress index of the user through the backend, and determines whether to activate Cowboy Coach. If the index is at normal levels, the Echo says "Nevermind!" and returns to sleep, but otherwise, Cowboy Coach assigns you your first exercise! This functions through coding a custom infrastructure which can be ran on the Echo Dot hardware called a custom skill. Our custom skill receives the stress index, activates some Cowboy dialogue, sends an API call to GPT4o alongside the stress level and a retrieval augmented generation architecture for storing our list of exercises. The model then returns instructions for the most effective exercise choice, takes over the users computer to display the tutorial, and coaches you as you destress!
The Frontend was first designed with Figma Dev, and then slowly converted into raw HTML CSS and Javascript.
Challenges we ran into
We ran into many challenges throughout this journey, but as all good cowboys say, "There's only enough room in this town for one of us.." and the millions of bugs and errors had no place in our midwestern town of choice. Our biggest challenges were finding a way to accurately measure the stress index, and creating functional software that the Echo Dot graciously permitted us to run. Following THREE HOURS of attempting to pair the Echo Dot to our phones, we finally were able to start coding only to realize that the Alexa documentation really is the wild west. There's nothing there. So after coding over 400 lines of code attempting to brute force our way through the syntax, we finally ended up with a functional product.... sometimes. Leading us to realize that if we covered a 0.5mm hole on the Echo Dot it would just stop functioning.
Yep There's two more hours we'll never get back.....
Accomplishments that we're proud of
WE OVERCAME THE CHALLENGES YEEEEEEHAWWWWWWWW Despite facing every bug in the books, we persevered through the storm and produced a fully functional product that makes full use of the Echo Dot! We also scoured ALL of github for a good way to measure stress, only to realize that maybe the real stress index is the one we make along the way. WE BUILT IT OURSELVES YEEEEEHAWWWWWWWW
We are also very proud that our project is not only an MVP, but a largely functional product that could be posted to the Echo Dot skill store in its current state. We hope that our project might help keep our fellow coders, office workers, and students working hard, working well, and working healthy!
YEEHAW
What we learned
We learned a lot. We learned about facial tracking algorithms, microexpression analysis, emotion classification, Alexa firmware, custom Echo models, and the hypothetical speed an Echo Dot would be flying at from the fifth floor of JMSB.
We also learned that cowboy hats are an absolute FIT
What's next for Wellness Cowboy
We hope to push the Wellness Cowboy skill to the Alexa skill store to allow the public easy access to stress relief during their everyday working schedules!
Log in or sign up for Devpost to join the conversation.