Inspiration

In today’s healthcare system, patient support is fragmented. Chatbots usually give one-off answers, and valuable insights from doctor visits are rarely captured or followed up on. We were inspired to reimagine the patient journey — from preparing for appointments to post-visit follow-through — by transforming isolated interactions into a proactive, intelligent support loop. Our key idea was: What if multiple specialized AI agents could collaborate like a care team, instead of acting alone?

What it does

Protect Health is a multi-agent web app that supports patients before, during, and after their doctor visits. • Before visits: Provides lifestyle guidance and question checklists. • During visits: Converts spoken conversations into structured tasks and analyzes skin images for triage. • After visits: Compares medical history to generate personalized reports and ensure follow-ups.

Core agents include: • Task Intake Agent – extracts tasks from voice transcripts. • Pre-Visit Coach Agent – prepares patients with tailored advice. • Skin Triage Vision Agent – analyzes skin images and shares findings. • Report Generator Agent – creates medical history comparison summaries. Together, they form a smart assistant team that ensures continuity of care.

How we built it

• Frontend (React + Vercel): chat UI, voice recording, image uploads, task/report components. • Backend (FastAPI + Google ADK): orchestrates agents using Loop, Sequential, and Parallel execution patterns; exposes REST APIs for transcripts, coaching, skin analysis, and reporting. • Google Cloud: o Vertex AI Gemini: multimodal LLMs for text, vision, and voice. o Firestore: stores tasks and reports. o Cloud Storage: saves images and audio. o Pub/Sub: handles asynchronous events. • Deployed on Vercel (frontend) and Cloud Run (backend).

Challenges we ran into

• Orchestrating multiple agents in real time without conflicts. • Balancing latency with accurate inference across vision, voice, and text. • Designing effective agent-to-agent (A2A) communication. • Maintaining context persistence across multiple visits. • Coordinating a distributed hackathon team under tight deadlines.

Accomplishments that we're proud of

• Built a fully functional multi-agent system using Google ADK and A2A. • Enabled real-time task extraction from live conversations. • Achieved cross-modal collaboration between vision and text agents. • Delivered a smooth frontend UX with chat, voice, and image support. • Designed a scalable, event-driven backend ready for future agents.

What we learned

• How to design and orchestrate multi-agent AI systems with Google ADK. • Best practices for coordinating agents with A2A protocols. • Applying Gemini effectively across voice, text, and vision inputs. • Building event-based architectures for asynchronous workflows. • The importance of communication, clarity, and role alignment in hackathons.

What’s next for Protect-Health

• Adding new agents for medication reminders, mental health check-ins, and insurance navigation. • Integrating with EHR systems for real-world clinical use. • Strengthening HIPAA-compliant data handling and security. • Launching a patient beta to validate usability in real-world care journeys.