MedLeaf

Inspiration

As international students living in the US, navigating the American healthcare system felt like learning a completely different language. Without a guide, we had to figure out everything ourselves. We quickly realized two major pain points:

Medical Jargon: Doctor's notes and lab results are incredibly hard to decipher for everyday patients.

Insurance Blackbox: The details of health insurance—deductibles, copays, out-of-pocket maximums—are buried in dense, confusing Explanation of Benefits (EOB) documents. We never knew exactly what we were covered for or what we were supposed to pay.

We built this app because we wished we had a personal, intelligent healthcare companion. We wanted a tool that could instantly translate doctor's notes into plain English, explain our insurance benefits, and track our medical history effortlessly.

What it does

MedLeaf is a privacy-first, AI-powered digital healthcare companion. It takes the guesswork and anxiety out of the US medical system by acting as your personal medical translator and insurance navigator.

Users simply snap a photo or upload a PDF of their complex doctor’s notes, lab results, or dense insurance Explanation of Benefits (EOBs). MedLeaf processes these documents instantly, translating intimidating medical jargon into plain English. It automatically builds a chronological, easy-to-read medical history records and a visual dashboard that tracks exactly how much of your insurance deductible or copay you have used, so you never get hit with a surprise medical bill.

How we built it

We adopted a "vibe coding" approach, rapidly prototyping with AI assistance while strictly prioritizing user privacy.

Frontend & Framework: We used React Native and Expo to build a cross-platform mobile experience with a calming, health-focused UI (light blues and greens).

Privacy-First Storage: Since we are handling highly sensitive medical data, we completely avoided cloud databases. All user profiles, timelines, and insurance logs are stored strictly on the device using SQLite (expo-sqlite).

The AI Brain: We leveraged the Google Gemini API (Multimodal) and fine-tuned our prompts using Google AI Studio. Instead of using clunky OCR tools, we send images of insurance cards and PDF doctor notes directly to Gemini. The model parses the visual data, translates the jargon, and returns beautifully structured JSON that populates our local database.

Challenges we ran into

Consistent AI Outputs: Getting an LLM to consistently return valid, perfectly structured JSON from messy, unstructured medical documents was a challenge. We spent a lot of time in Google AI Studio engineering the exact System Instructions and JSON Schema to ensure the app wouldn't crash from malformed data.

The Local-First Tradeoff: Designing an architecture that relies entirely on local storage meant we couldn't easily use cloud-based AI agents. We had to carefully manage the data flow—sending a document temporarily to the Gemini API, getting the analysis, and instantly saving the result back to the local SQLite database without leaving a footprint.

Multimodal Messiness: Users upload documents in all shapes and sizes—crooked photos of insurance cards, multi-page PDFs, and blurry lab results. Optimizing the image payload for Gemini while maintaining fast load times required significant tweaking.

Accomplishments that we're proud of

100% Local, Cloud-Free Architecture: Handling sensitive health data is risky. We are incredibly proud that we successfully built an architecture where all your personal medical records and insurance data live strictly on your physical device using SQLite. No cloud databases, no tracking, just complete patient control.

Ditching OCR for Multimodal AI: Traditional text-extraction (OCR) tools are terrible at reading messy medical documents. We bypassed them entirely by passing raw images directly to the Google Gemini API. Seeing the AI perfectly understand the context of a crooked photo of an insurance card and output clean JSON data was a massive "wow" moment for us.

What we learned

The Power of Multimodal AI: We learned that modern LLMs like Gemini 2.5 can completely replace traditional OCR pipelines. The ability to "read" an image and instantly understand the context (e.g., distinguishing a copay from a deductible) is game-changing.

Privacy is a Feature: We realized that for healthcare apps, telling users "we don't have a database" is actually a massive selling point.

Prompt Engineering is the New Backend: Structuring prompts in AI Studio to act as our backend data parsers taught us how to bridge the gap between unstructured real-world chaos and strict frontend UI requirements.

What's next for MedLeaf

Smart Pre-Visit Summaries: We plan to build an AI feature that analyzes your recent symptom logs and generate a concise, professional summary to hand to your doctor.

Proactive Benefit Alerts: We want to expand our insurance tracker to send local push notifications when you are close to hitting your out-of-pocket maximum, or when you have expiring benefits (like annual dental cleanings or FSA funds) before the year ends.

Native Language Translation: Staying true to our roots of helping international students, we want to add a feature that instantly translates doctor's notes and insurance terms from English into the user's native language for ultimate clarity.