Daisy Helps

Empowering the non-technical in an era of ever advancing technology.

Comment

Inspiration

Competing in the Accessible, Equitable, Inclusive Tech track, we aimed to promote tech literacy within our communities. The technology available in the market assumes a baseline level of tech literacy and understanding; it assumes you know what certain icons mean and where certain features are located. But that's not always true, and that could be for a plethora of reasons. You might be a senior citizen getting your first phone, someone with a visual disability, or could live in a community without access to modern tech training. According to a study published by Frontiers in Psychology, "most of [the older adults] were eager to adopt new technology,... but voiced apprehension about lack of, or lack of clarity in, instructions and support." One of our members used to work at Target, where he saw many people, especially senior citizens, come in for tech help. These tech questions were usually simple- how do I set up my email? How do I add my grandson's contact to my phone? Their requests could easily be fixed with the help of a friend or family member, but sometimes, life feels too busy to answer all seventeen of your grandmother's phone calls about how to set up her computer. Enter Daisy Helps, a chatbot made to encourage tech fluency.

What it does

Daisy is a desktop app that can be installed on your loved one's computer; when they run into a tech issue, they can simply activate the chat using their voice and ask a question. "Hey Daisy, how do I join this Zoom call?" "Let me help you with that!" Daisy uses a screenshot of the user's device to guide them through the process. It shows them where to click, what the screen is prompting them to do, and how to operate common software. By showing the user what to do in an accessible, intuitive way, Daisy doesn't fix their problems for them; rather, it guides them so they gradually learn to use their tech on their own. Daisy aims not only to fix, but to empower.

How we built it

Through our process, we used multiple programs and software in order to create the right interface and program for us. We leveraged Claude Code for much of the project coding, Render for the user's initial landing page, ElevenLabs for the voice prompt, the Grok API for the audio input, Claude Sonnet for the image analysis, and Claude Haiku for the final user output. As the user speaks to Daisy, it takes a screenshot, then analyzes it to guide the user, and a spotlight cursor shows the user exactly what needs to be done. Daisy lives in the corner of the user's screen as an easy-to-access overlay icon, always ready to help. For users who are hard of hearing, Daisy also provides chat transcripts.

Challenges we ran into

We ran into multiple challenges with the voice detection. Daisy would often cut off the user or start speaking too early because its mic was overly sensitive, leading to a confusing user experience. We also had a couple of UI/UX bugs throughout the process. When we were creating an animation with Daisy's petals, they would often end up blocking important text.

Accomplishments that we're proud of

We are proud to have taken advantage of the available software and programs to create this, even if it wasn't easy. By using Claude Code, Haiku, Sonnet, ElevenLabs, Grok, and Render, we combined features and techniques to create a cohesive platform. It took us a long time to master the voice controls and make them properly responsive to the user, as it was one of the integral parts of our design. We also pride ourselves on being able to create such a wide scope of use; the platform can be used not only by senior citizens, but by anyone who may have trouble using tech or learning a specific software. We feel as though when it comes to accessibility in technology, the wider the scope, the better. We want to get as many people into technology because, in today's digital society, being able to use tech independently is a form of empowerment.

What we learned

We've realized that accessibility in tech is such an underrepresented sector, especially since corporations tend not to consider senior citizens as a very profitable or marketable audience. It was only as we began researching the accessibility challenges facing our communities that we realized our tech served only a small part of our society. Most tech and websites are not visually or audibly accessible, and many are cluttered with ads, icons, and confusing text. As we conduct more research, it becomes evident that there's a significant gap in the tech sector, with larger tech companies prioritizing profit over social good.

What's next for Daisy Helps

While we are very happy with what we've built, we also acknowledge that there are many more features we need to implement. With our short timeframe, we were able to nail down the primary demo, but many other features are necessary in order to fully serve our community. We want to expand to mobile, as our current model is only a desktop app. We would also like to implement additional toggles to customize the webpage's *visual accessibility: larger text options, high-contrast color schemes, dyslexia-friendly typefaces, volume controls, etc. In order to benefit communities that may struggle with technology due to language barriers, we would also like to implement multiple other languages, as our current options are limited to English and Spanish.

Built With

  • claude
  • elevenlabs
  • grok
  • render
Share this project:

Updates