CuraKeep

Inspiration

The inspiration for CuraKeep comes from observing the daily realities of public healthcare infrastructure. Millions of people queue for hours at public clinics, only to leave with handwritten clinic cards or paper prescriptions. These documents are notoriously difficult to read, filled with complex medical shorthand like "BD" or "TDS". When patients or caregivers cannot decipher these scribbles, it leads to critical dosage errors, missed follow-up appointments, and avoidable health risks. I wanted to build a bridge between traditional paper-based tracking and modern digital health wallets, turning a fragile piece of paper into a smart, accessible lifeline.

What it does

CuraKeep is a multimodal digital health companion that helps families manage handwritten medical records in one secure place. A user simply takes a photo of their clinic card or prescription note. CuraKeep uses AI to instantly transcribe the handwriting, decode complex clinical shorthand, and convert the unstructured text into a clean digital profile. Crucially, it translates medical jargon into plain, everyday language and automatically sets up SMS or WhatsApp notifications for dosage times and upcoming clinic appointments, ensuring patients never miss a step in their care.

How we built it

I engineered CuraKeep with a Python backend powered by FastAPI to manage file uploads and data structures securely. The core intelligence relies on the Google Gemini 1.5 Pro API, which I configured with specialized clinical reasoning system prompts and a deterministic temperature setting to handle advanced Optical Character Recognition (OCR). I utilized Gemini's structured output capability (response_mime_type="application/json") to map messy handwriting directly into a strict JSON schema. The frontend is built as a lightweight, mobile-responsive web dashboard that allows seamless image capturing and displays the extracted medical data as scannable, intuitive digital health cards.

Challenges we ran into

The biggest challenge I faced was handling the sheer variance in handwritten clinical notes—ranging from faded ink and cursive script to structured abbreviations unique to localized medical systems. Standard OCR tools failed entirely. I also had to ensure the model never hallucinated critical safety information, such as drug names or dosages. I overcame this by refining my prompt engineering in Google AI Studio, using strict XML-style constraints that force Gemini to return null values instead of guessing, and leveraging Gemini 1.5 Pro's massive context window to cross-reference ambiguous handwriting with known clinical terms.

Accomplishments that we're proud of

As a solo developer, I am incredibly proud of building a functional prototype that handles the entire end-to-end flow—from a messy image upload to a beautifully structured, human-readable health dashboard—in just a few days. Successfully forcing the AI to perfectly translate complex Latin medical shorthand (like Nocte or PRN) into clear, everyday instructions without compromising medical accuracy felt like a massive technical and personal victory.

What we learned

I learned that the true power of multimodal LLMs isn't just their ability to see or read, but their ability to reason over what they see. This hackathon taught me how to design robust guardrails around AI models when dealing with high-stakes data like healthcare. I also deepened my understanding of structured JSON outputs, API performance tuning, and how to design UI/UX that keeps accessible design at the forefront for everyday users.

What's next for CuraKeep

Moving forward, I want to integrate multilingual support directly into the translation layer so that patient instructions can be generated in local languages like isiZulu, Sesotho, or Afrikaans. I also plan to integrate secure database storage like Supabase to keep long-term historical records, build out the automated WhatsApp notification engine using the Twilio API, and implement an offline caching mechanism so users can still access their vital health cards even without a stable internet connection.

Built With

• docker
• fastapi
• google-ai-studio
• google-gemini-1.5-pro-api
• google-genai-sdk
• next.js
• pillow
• pydantic
• python
• render
• tailwind-css
• typescript
• vercel