Skip to content

Hjhirp/PatientHero

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

11 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

PatientHero - Complete AI Healthcare Assistant

A comprehensive healthcare consultation platform that combines a modern ChatGPT-style interface with intelligent AI agents for medical information collection, symptom analysis, hospital search, appointment processing, and patient support throughout their healthcare journey.

๐ŸŒŸ Features

Core Functionality

  • ๐Ÿค– Multi-Agent AI System: Three specialized CrewAI agents for different phases of patient care
  • ๐Ÿ’ฌ Intelligent Chat Interface: Natural conversation flow with context-aware responses
  • ๐Ÿฅ Hospital Discovery: Automated search for nearby medical institutions using Exa.ai
  • ๐Ÿ“… Appointment Processing: Parallel processing of hospital websites to find available slots
  • ๐Ÿ›ฃ๏ธ Journey Support: Comfort guidance and reassurance during travel to hospital
  • ๐Ÿ“Š Data Extraction: Structured JSON output for seamless integration with other systems
  • ๐Ÿ”„ Background Processing: Non-blocking appointment scraping and data processing

Technical Features

  • ๐ŸŽจ Modern UI: Clean, responsive design with dark/light theme support
  • ๏ฟฝ Mobile Responsive: Works perfectly on all device sizes
  • โšก Real-time Processing: Live updates and background task management
  • ๐Ÿ”’ Session Management: Persistent sessions with conversation history
  • ๐Ÿ“ˆ Monitoring: Comprehensive logging with W&B Weave integration
  • ๐Ÿ›ก๏ธ Error Handling: Robust error handling and graceful fallbacks

๐Ÿ—๏ธ Architecture

AI Agent System (CrewAI)

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚   Chat Agent    โ”‚    โ”‚ Reasoning Agent  โ”‚    โ”‚ Extraction Agent โ”‚
โ”‚ (Info Collect)  โ”‚ -> โ”‚ (Symptom Analys) โ”‚ -> โ”‚ (Data Structure) โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
         โ”‚                        โ”‚                        โ”‚
         โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                  โ”‚
                    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
                    โ”‚     Gemini 2.5 Flash     โ”‚
                    โ”‚    (Google AI API)       โ”‚
                    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Data Flow

User Input -> Chat Agent -> Basic Info Complete? 
                              โ”‚
                              โ”œโ”€ No: Continue Collection
                              โ”‚
                              โ””โ”€ Yes: Hospital Search (Exa.ai)
                                      โ”‚
                                      โ”œโ”€ Background: Appointment Processing
                                      โ”‚
                                      โ””โ”€ Reasoning Agent -> Comfort Guidance

Backend Services

  • FastAPI Server: REST API with session management
  • CrewAI Agents: Specialized AI agents for different tasks
  • Playwright: Browser automation for appointment scraping
  • Exa.ai Integration: Hospital and clinic discovery
  • W&B Weave: AI interaction monitoring and tracing

๐Ÿš€ Quick Start

Prerequisites

  • Node.js 18+
  • Python 3.11+
  • Google Gemini API Key
  • W&B API Key (for monitoring)
  • Exa.ai API Key (for hospital search)

1. Clone and Setup

git clone <repository-url>
cd PatientHero

# Install frontend dependencies
npm install

# Setup Python environment
cd crewai_agents
chmod +x setup.sh
./setup.sh

# Install Playwright browsers
conda activate patienthero-crewai
playwright install

2. Environment Configuration

Create .env files with your API keys:

Root .env:

GEMINI_API_KEY=your_gemini_api_key
WANDB_API_KEY=your_wandb_api_key
WANDB_PROJECT=patienthero-crewai
WANDB_ENTITY=your_wandb_entity
EXA_API_KEY=your_exa_api_key

crewai_agents/.env:

GEMINI_API_KEY=your_gemini_api_key
WANDB_API_KEY=your_wandb_api_key
WANDB_PROJECT=patienthero-crewai
WANDB_ENTITY=your_wandb_entity
EXA_API_KEY=your_exa_api_key

3. Start the Services

# Terminal 1: Start the API server
python api_server.py

# Terminal 2: Start the frontend
npm run dev

4. Access the Application

๐Ÿ“‹ Complete User Flow

Phase 1: Information Collection

  1. User starts conversation with greeting
  2. Chat Agent activates to collect:
    • Medical condition/symptoms
    • ZIP code for location
    • Phone number for contact
    • Insurance information

Phase 2: Hospital Discovery & Appointment Processing

  1. Automatic hospital search using Exa.ai
  2. Background appointment processing with Playwright
  3. Data structure extraction with confidence scoring

Phase 3: Medical Reasoning & Support

  1. Reasoning Agent analysis of symptoms and conditions
  2. Comfort guidance provision (2 rounds of support)
  3. Journey assistance with practical tips

๐Ÿ› ๏ธ Tech Stack

Frontend

  • Framework: Next.js 14 with App Router
  • Language: TypeScript
  • Styling: Tailwind CSS
  • UI Components: Radix UI primitives
  • Icons: Lucide React
  • State Management: React hooks and context
  • Theme: next-themes for dark/light mode

Backend

  • API Framework: FastAPI
  • AI Framework: CrewAI with custom Gemini LLM
  • Language Model: Google Gemini 2.5 Flash
  • Browser Automation: Playwright
  • Search Engine: Exa.ai API
  • Monitoring: W&B Weave
  • Session Storage: In-memory (production: Redis/DB)

Development Tools

  • Package Management: npm, conda/pip
  • Code Quality: ESLint, Prettier
  • Type Checking: TypeScript
  • API Documentation: FastAPI auto-docs

๐Ÿ“ Project Structure

PatientHero/
โ”œโ”€โ”€ app/                    # Next.js 14 app directory
โ”‚   โ”œโ”€โ”€ globals.css        # Global styles and themes
โ”‚   โ”œโ”€โ”€ layout.tsx         # Root layout with providers
โ”‚   โ””โ”€โ”€ page.tsx           # Main chat interface
โ”œโ”€โ”€ components/            # React components
โ”‚   โ”œโ”€โ”€ ui/               # Reusable UI primitives
โ”‚   โ”œโ”€โ”€ providers/        # Context providers
โ”‚   โ”œโ”€โ”€ medical-consultation.tsx  # Main chat component
โ”‚   โ”œโ”€โ”€ appointment-flow.tsx      # Appointment processing UI
โ”‚   โ”œโ”€โ”€ sidebar.tsx       # Navigation sidebar
โ”‚   โ””โ”€โ”€ theme-toggle.tsx  # Dark/light mode toggle
โ”œโ”€โ”€ lib/                  # Utility functions and services
โ”‚   โ”œโ”€โ”€ ai-service.ts     # API communication layer
โ”‚   โ”œโ”€โ”€ utils.ts          # Helper functions
โ”‚   โ””โ”€โ”€ hooks.ts          # Custom React hooks
โ”œโ”€โ”€ crewai_agents/        # AI agent system
โ”‚   โ”œโ”€โ”€ main.py           # Core CrewAI implementation
โ”‚   โ”œโ”€โ”€ exa_helper.py     # Hospital search integration
โ”‚   โ”œโ”€โ”€ requirements.txt  # Python dependencies
โ”‚   โ”œโ”€โ”€ environment.yml   # Conda environment
โ”‚   โ””โ”€โ”€ setup.sh          # Automated setup script
โ”œโ”€โ”€ api_server.py         # FastAPI backend server
โ”œโ”€โ”€ process_clinics_parallel.py  # Appointment processing
โ”œโ”€โ”€ docs/                 # Documentation
โ”‚   โ”œโ”€โ”€ DEPLOYMENT_GUIDE.md
โ”‚   โ”œโ”€โ”€ PROMPT_SYSTEM.md
โ”‚   โ””โ”€โ”€ WANDB_SUMMARY.md
โ””โ”€โ”€ scripts/              # Utility scripts

๐Ÿ”ง API Endpoints

Core Chat API

  • POST /api/chat - Process user input through AI agents
  • GET /api/status/{session_id} - Get session status and patient data

Appointment System

  • POST /api/complete-flow/{session_id} - Full medical flow processing
  • GET /api/appointments/{session_id} - Get processed appointment data
  • POST /api/appointments/process - Manual appointment processing

Patient Support

  • GET /api/comfort-guidance/{session_id} - Comfort guidance rounds
  • POST /api/sessions/{session_id}/end - End patient session

Monitoring

  • GET /api/sessions - List active sessions (debug)
  • GET / - API health check

๐Ÿ’พ Data Models

Patient Data Structure

interface PatientData {
  session_id: string;
  timestamp: string;
  medical_condition?: string;
  zip_code?: string;
  phone_number?: string;
  insurance?: string;
  symptoms?: string[];
  reasoning_analysis?: string;
  conversation_history: ConversationEntry[];
}

Appointment Data Structure

interface AppointmentData {
  hospital_name: string;
  website: string;
  institution_type: string;
  accepts_user_insurance: string;
  appointment_availability: {
    available_slots: AppointmentSlot[];
    total_slots_found: number;
    next_available: string;
    booking_method: string;
  };
}

๐Ÿ”„ Agent Workflows

1. Chat Agent (Information Collector)

  • Role: Friendly medical intake specialist
  • Goal: Collect essential patient information naturally
  • Behavior: One question at a time, empathetic responses
  • Transition: Switches to Reasoning Agent when basic info complete

2. Reasoning Agent (Medical Analysis)

  • Role: Medical reasoning specialist
  • Goal: Analyze symptoms and provide preliminary guidance
  • Behavior: Evidence-based analysis, follow-up questions
  • Output: Detailed symptom analysis and medical reasoning

3. Extraction Agent (Data Structuring)

  • Role: Data extraction specialist
  • Goal: Structure conversation data into JSON format
  • Behavior: Intelligent parsing, confidence scoring
  • Output: Structured patient data with metadata

๐Ÿฅ Hospital Processing

Exa.ai Integration

  • Search scope: .org and .gov domains only
  • Filters: Location-based, insurance compatibility
  • Output: Structured hospital data with contact information

Appointment Processing

  • Method: Parallel Playwright browser automation
  • Target: Hospital booking pages and forms
  • Extraction: Available time slots, booking methods
  • Performance: Concurrent processing for speed

Data Cleaning

  • Standardization: Consistent appointment slot formats
  • Validation: Remove invalid or malformed data
  • Enhancement: Add booking methods and availability status

๐Ÿ“Š Monitoring & Analytics

W&B Weave Integration

  • Agent Interactions: Every AI model call traced
  • Session Tracking: Complete conversation flows
  • Performance Metrics: Response times, success rates
  • Error Tracking: Detailed error logs and context

Key Metrics

  • Information Collection Rate: Percentage of complete patient profiles
  • Agent Transition Success: Chat -> Reasoning -> Extraction flow
  • Appointment Processing: Success rate of hospital data extraction
  • User Engagement: Session duration and interaction quality

๐Ÿ”’ Security & Privacy

Data Protection

  • Session Isolation: Each session independent and secure
  • Temporary Storage: Patient data not permanently stored
  • API Security: CORS protection and request validation
  • Privacy: No data sharing with external services (except AI APIs)

Safety Measures

  • Content Filtering: Gemini safety settings configured
  • Error Boundaries: Graceful handling of API failures
  • Input Validation: Sanitization of user inputs
  • Rate Limiting: Protection against abuse

๐Ÿš€ Deployment Options

Development (Local)

# Start both services locally
python api_server.py &
npm run dev

Production (Recommended)

  • Frontend: Vercel, Netlify, or similar
  • Backend: Railway, Render, or VPS
  • Database: PostgreSQL or MongoDB for persistence
  • Monitoring: W&B dashboard for production monitoring

Docker Deployment

# Build and run with Docker
docker build -t patienthero .
docker run -p 3000:3000 -p 8000:8000 patienthero

๐Ÿงช Testing

Frontend Testing

npm run test          # Run test suite
npm run test:watch    # Watch mode
npm run lint          # Code quality check

Backend Testing

cd crewai_agents
python test_complete_flow.py  # Test complete AI flow
python -m pytest tests/       # Run unit tests

API Testing

# Test core endpoints
curl -X POST http://localhost:8000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"user_input": "Hi, I have a fever", "session_id": "test-123"}'

curl -X POST http://localhost:8000/api/complete-flow/test-123

๐Ÿค Contributing

Development Setup

  1. Fork the repository
  2. Create a feature branch
  3. Set up development environment
  4. Make your changes
  5. Add tests if applicable
  6. Submit a pull request

Code Style

  • Frontend: ESLint + Prettier configuration
  • Backend: PEP 8 Python style guide
  • Commits: Conventional commit messages
  • Documentation: Update README for significant changes

๐Ÿ“ž Healthcare Disclaimer

Important: This application is designed for educational and demonstration purposes. It should not be used as a substitute for professional medical advice, diagnosis, or treatment. Always consult with qualified healthcare providers for medical concerns.

๐Ÿ”ฎ Future Enhancements

Planned Features

  • User Authentication: Persistent user accounts
  • Database Integration: Long-term data storage
  • Real Appointment Booking: Direct integration with hospital systems
  • Telemedicine: Video consultation capabilities
  • Multi-language: Support for multiple languages
  • Mobile App: Native iOS/Android applications

AI Improvements

  • Memory Enhancement: Long-term conversation memory
  • Specialized Models: Domain-specific medical AI models
  • Voice Interface: Speech-to-text and text-to-speech
  • Predictive Analytics: Early symptom detection
  • Personalization: Adaptive responses based on user history

Integration Opportunities

  • EHR Systems: Electronic health record integration
  • Insurance APIs: Real-time coverage verification
  • Pharmacy Systems: Prescription management
  • Wearable Devices: Health data integration
  • Emergency Services: Direct 911/emergency routing

๐Ÿ“„ License

This project is licensed under the MIT License - see the LICENSE file for details.

๐Ÿ™ Acknowledgments

  • CrewAI: Multi-agent AI framework
  • Google Gemini: Advanced language model capabilities
  • Exa.ai: Intelligent web search for healthcare
  • W&B Weave: AI application monitoring and tracing
  • Radix UI: Accessible component primitives
  • Playwright: Reliable browser automation

๐Ÿ“š Additional Documentation

๐Ÿ†˜ Support

For technical support, bug reports, or feature requests:

  1. Check existing Issues
  2. Create a new issue with detailed description
  3. Include environment details and error logs
  4. Tag with appropriate labels

Built with โค๏ธ for better healthcare accessibility โ”‚ โ”œโ”€โ”€ ui/ # Reusable UI components โ”‚ โ”œโ”€โ”€ providers/ # Context providers โ”‚ โ”œโ”€โ”€ sidebar.tsx # Chat sidebar โ”‚ โ””โ”€โ”€ theme-toggle.tsx # Theme switcher โ”œโ”€โ”€ lib/ # Utility functions โ”‚ โ””โ”€โ”€ utils.ts # Helper functions โ”œโ”€โ”€ public/ # Static assets โ”‚ โ””โ”€โ”€ manifest.json # PWA manifest โ””โ”€โ”€ types/ # TypeScript type definitions


## Key Components

### Chat Interface
- **Message Display**: Shows user and AI messages with timestamps
- **Real-time Input**: Send messages with Enter key or button click
- **Typing Indicators**: Visual feedback when AI is responding
- **Message History**: Persistent chat history across sessions

### Sidebar Navigation
- **Chat List**: Organized by date (Today, Yesterday, Previous 7 days, Older)
- **New Chat**: Create new conversations
- **Search**: Filter chats by title
- **Responsive**: Collapsible on mobile devices

### Theme System
- **Dark/Light Mode**: Toggle between themes
- **System Preference**: Respects user's system theme
- **Persistent**: Theme preference saved locally

## Customization

### Styling
The app uses Tailwind CSS with a custom color system. You can modify the theme in:
- `tailwind.config.js` - Tailwind configuration
- `app/globals.css` - CSS custom properties for colors

### AI Integration
To connect to actual AI services, modify the `sendMessage` function in `app/page.tsx`:

```typescript
// Replace the simulation with actual AI API calls
const sendMessage = async () => {
  // Your AI API integration here
}

Components

All UI components are in the components/ui/ directory and are built with Radix UI primitives for accessibility and customization.

Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Add tests if applicable
  5. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Healthcare Disclaimer

This application is designed for educational and demonstration purposes. It should not be used as a substitute for professional medical advice, diagnosis, or treatment. Always consult with qualified healthcare providers for medical concerns.

Implementation Status

โœ… Completed Features

  • Modern Next.js 14 + TypeScript setup
  • Responsive chat interface with sidebar
  • Dark/light theme toggle
  • Message input with auto-resize textarea
  • Chat history management
  • Loading states and animations
  • Settings dialog
  • PWA configuration with manifest
  • Modern UI components (Button, Input, Card, Avatar, etc.)
  • Proper favicon and app icons
  • TypeScript type safety
  • Tailwind CSS styling
  • Radix UI component primitives
  • Med42-8B Integration: Production-ready medical AI on Fly.io
  • AI Service Layer: OpenAI-compatible API for seamless integration
  • Medical Specialization: Healthcare-focused prompts and responses

๐Ÿšง In Progress / Next Steps

  • Deploy Med42-8B: Run cd fly-llm-server && ./deploy.sh for cloud deployment
  • User authentication
  • Database integration for chat persistence
  • Message export functionality
  • File upload support
  • Voice input/output
  • Medical data integration
  • Advanced settings (model parameters)
  • Chat sharing and collaboration
  • Analytics and usage tracking

๐Ÿ’ก Future Enhancements

  • Multi-language support
  • Plugin system for medical tools
  • Integration with EHR systems
  • Appointment scheduling
  • Medication reminders
  • Health data visualization
  • Telemedicine integration

Current Status

The application is production-ready with complete AI integration!

๐Ÿค– AI Integration Status

  • โœ… Med42-8B Integration: Production-ready medical AI on Fly.io
  • โœ… OpenAI-Compatible API: Seamless integration with standard formats
  • โœ… Medical Specialization: Healthcare-focused system prompts
  • โœ… Scalable Deployment: Auto-scaling GPU infrastructure on Fly.io

๐Ÿš€ Quick Start Options

Production Deployment (Recommended)

cd fly-llm-server && ./deploy.sh  # Deploy Med42-8B to Fly.io
npm run dev                       # Start the frontend

๐Ÿ“Š Deployment Comparison

Feature Med42-8B on Fly.io Local Setup
Cost $50-150/month (scales to $0) Free
Setup Time 15 minutes N/A
Performance Very Fast (A10 GPU) N/A
Privacy Secure Cloud N/A
Maintenance Managed N/A

Next Steps

  1. Deploy Med42-8B: Follow the deployment guide in fly-llm-server/DEPLOYMENT.md
  2. Configure environment: Update .env.local with your Fly.io URL
  3. Test integration: Ensure the frontend connects to your deployed backend
  4. Add authentication: For user management and chat persistence

The UI is production-ready and follows modern design patterns. All core components are implemented and the application is responsive across all device sizes.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors