MedMate

A centralized app for caregivers to organize medication, care time, and more.

Comment 1

See our commercial for the app here: Commercial!

Inspiration

Our inspiration comes from the millions of Americans, including close family, who have had to take care of a loved one in old age or through sickness. Families such as my (Andy) own, who had a mother who went through surgery and had a week-long recovery, administering medications and providing physical and emotional support for them. Or, Krish, who had to help his family take care of his grandfather at his bedside. At-home care is a job of love; and it's a job that we aim to make comfortable and less stressful.

Every family, at some point, will experience this kind of time. Our desire to help a community that is often overlooked, forgotten, and minimally supported by our government. Enough is enough.

We present an app that aims to make at-home care extremely convenient, simple, and organized, to both avoid medication errors and help families feel closer together during difficult times, without having to stress over daily tasks such as remembering to administer medication or find interactions through a mishmash of different apps and sources.

What it does

We have centralized the logistics of at-home care, with patient safety and caretaker mindfulness as our core.

Our app is a one stop shop for all logistics in home care, compiling FDA data with prescription inputs into the page, and setting reminders and taking care of vital sign data collection. Our app has several amazing features:

  1. Simply take a photo of your prescription bottle label, and the app does the rest! It will find FDA drug interaction warnings, set up a to-do list with times of when to administer medications, and keep the dosage instructions, and send notifications when it's time to administer medication. No more worrying about setting a timer and checking drug interactions over and over!
  2. Enter our data entry tab, where you can take vital signs of your loved one, including weight, temperature, heart rate, and blood pressure. We will graph it for you, and determine whether the signs are in a safe range. We'll warn you if you need to seek medical help.
  3. No more worrying about refills! Simply scan your prescription, and our app will ensure you won't run out of doses by reminding you when to get a refill of a prescription, keeping your mind at ease.
  4. Security-minded: All data is anonymized when looking to AI for warnings, insights, and more. We have Propelauth authentication securing your account.

How we built it

Frontend - React to build the website/mobile app, and pull in information from the backend. It was used to build the calendar, the 911 alert button, and other interfaces to present to the user.

Backend - Flask for all backend calculations and API calls with Python. Python was used to build the pytesseract image capture system for the prescription labels, then in the same backend, called TuneStudio and GPT-4o to decode the text, and present the final json in a usable manner. We also used the openFDA API to get medication warnings, and then used python to build the final json to be used by frontend. Flask was used for all the request handling and the data pipelines for the visualizations.

Challenges we ran into

Finalizing the idea was difficult, in addition to the features that we would have in the end. This is because there were many tracks that we were interested in, but had to balance with what we could feasibly implement in the span of 36 hours.

Pytesseract was extremely finicky, and we tried to avoid using AI to fix the text, but much of the data that was pulled from Pytesseract was garbled due to the formatting of the prescription labels and the lack of standardization.

Further, we were unable to get the correct prompt to make TuneStudio AI output the proper json that we desired, using a lot of prompt engineering to do so. Frontend and backend integration was difficult as errors kept popping up, and then deciding on the frontend and backend integration, switching between different database backgrounds.

Finally, there was a lot of bugs with CORS conflicts, and lots of merge conflicts in GitHub since each of us was assigned to one task over another with a skill matched to what we knew already and was willing to learn.

Accomplishments that we're proud of

We're proud of overcoming the challenges and coming up with not just a mock up of our final project, but a truly working model of our product that we hope to continue. In addition, we're proud of developing an idea that was not our strengths entirely, and instead, was built on the basis of learning new frameworks and new ideas. We're also proud of learning different APIs, from openFDA to TuneStudio to make our product come together in the end.

What we learned

We learned a lot of different technical debugging skills and how to prompt engineer with TuneStudio to decipher the raw text from pytesseract. We learned the development of software from beginning to finish, how to choose an idea, and plan it out entirely. We learned how to work together with different skill sets, different backgrounds in frontend/backend and different platforms, and how to still make them work together relatively efficiently.

What's next for MedMate

We hope to continue to develop this product, and add more features to integrate it with hospital systems, such as United Healthcare's MyChart system, so that users have easier access to their records instead of manual entry, and in addition, have easier access to their doctor. Bigger datasets and more drug interaction features, in addition to missed ideas such as calling a doctor automatically, getting automatic prescription refills and delivery to home. We hope to make this a dedicated mobile app as well, and make an impact in the community by helping the at home caretakers both here in our country, and internationally as well, fostering a community of care.

Built With

Share this project:

Updates