Healthyz

Inspiration

Our team was inspired to take on the challenge of telemedicine in rural India. Inadequacies in the public health infrastructure in India often mean that even routine tasks, such as asking basic check-in questions, can consume significant labor and compromise the quality of treatment. Insufficient information on patients forces telemedicine doctors to waste precious time asking basic questions and can be detrimental to diagnostic accuracy. We were very interested in the technical challenge of applying AI to benefit patients and improve public services on a large scale.

What it does

In place of a nurse at a PHC or a doctor, pre-screening questions required for the appointment are asked by Healthyz, our AI-powered patient analysis and database software. It begins with standard general questioning on medical history and common conditions. Patients are then prompted to record a voice memo of the symptoms they’re experiencing and the reason for their visit. As we’ll demonstrate, the AI will internally translate this entry from any language into English, summarize the patient’s account, and begin a more specific line of questioning that readjusts with every answer. When this concludes, patients may download a PDF of the interaction to keep with them. Healthyz will assemble a report for the doctor to review before meeting the patient along with a list of likely diagnoses – one that is not visible to the patient.

How we built it

On the frontend, we leveraged React to develop a simple and user-friendly interface. Our objective was to focusing on accessibility and ease of use, particularly when it came to patient's symptoms and medical history. Emphasizing an easy-to-use application, we also provided a way to speak to an AI - GPT-4 Mini - that serves as a mini engine to provide patients with Q&A, shortening the diagnostic period. GPT-4 Mini is paired with AI, Whisper, which provides commonly used languages and dialects in rural India.

Challenges we ran into

We faced several challenges in our project, particularly the need to deploy AI in areas with limited 2G/3G coverage, which we addressed by utilizing lightweight models like GPT-4 Mini. However, other aspects of the project required admittedly larger models like Whisper. The second challenge we faced was integrating AI into our project. From programming the speech-to-text to outputting the conclusion of the conversation, the AI models were quite tricky and time-consuming. Lastly, combining everyone’s different coding styles and frameworks into a single HTML project led to occasional integration issues. There were a few instances of structural mismatches, and we worked as a team to amalgamate them into a smooth, cohesive project.

Accomplishments that we're proud of

Effective communication and teamwork were key to our success. In our team of three, we ensured tasks were evenly spread and each member diligently worked on their assigned responsibilities. Once tasks were completed, everyone was willing to assist others or move on to the next task, demonstrating our collective grit and commitment. Muhammad gets a special shoutout for his work with the speech-to-text translation AI, which is an aspect of our project we’re especially proud of. David and Maddie learned React and frontend design on the fly, which was a difficult but fun, collaborative challenge that resulted in the streamlined UI we had initially envisioned. All of the above required a tremendous amount of grit and determination.

What we learned

The team learned to understand not only technology but the cultural applications. Creating solutions for India’s rural areas showed how a sense of understanding is a must to create an effective solution. Realizing the limitations placed in this track caused the team to not fully understand the culture and infrastructure, which forced us to start understanding what others needed. On the technical side, we learned the importance of communication, patience, and an open mind. A proper team thrives on clear communication, which gains patience, and creates an environment that welcomes creativity. Most importantly, we learned to have fun even through the challenges given!

What's next for Healthyz

Looking ahead, we'd love to continue collaborating with eSanjeevani and Intelehealth if given the opportunity. As artificial intelligence grows in the healthcare industry daily, we’re enthusiastic to support their efforts to bring quality telemedicine to those in need. Outside of HopHacks, we have a few ideas we’d like to implement to further develop our software. After a discussion surrounding how to conduct patient logins, we believe facial recognition technology strikes an excellent balance between privacy concerns and accessibility. Additionally, we’d like to develop our patient dashboard to include prescriptions and doctors’ notes in the patient dashboard.