AI-Powered Universal Product Scanner & Health Information App

MedPal is a comprehensive React Native application that uses cutting-edge AI technology to analyze any product through camera scanning. From prescription medications to supplements, consumables, and everyday products - MedPal instantly provides structured health information, usage guidance, benefits, and potential effects for anything you scan.

• Instant OCR: Powered by Google Cloud Vision API
• Live Camera Integration: Real-time product label scanning
• High Accuracy: Advanced text recognition for any product type
• Immediate Navigation: Seamless camera-to-results flow with animated loading

• Universal Product Intelligence: Analyzes medications, supplements, consumables, and everyday products
• Structured Information: Name, doses/servings, frequency, usage instructions
• Health Benefits: Treatment purposes and beneficial effects for any product
• Realistic Safety Info: Actual side effects and risks based on product type
• Contextual Responses: Different information depth based on product category

• Gesture-Free Navigation: No accidental swipe-backs, button-only navigation
• Toggle Information Display: Switch between structured summary and detailed description
• Real Image Display: Shows your actual captured photo in results
• Loading Animation: Custom GIF with fallback for smooth transitions
• State Management: Fresh data for each scan, no cached results

• Material Design: React Native Paper components with custom theming
• Structured Data Cards: Clean grey boxes for medication information with sub-instructions
• Responsive Layout: Optimized for all screen sizes
• Error Handling: 30-second timeouts with graceful fallbacks

• Secure User Authentication: Powered by Clerk with email verification
• Protected Routes: Authentication-based access control
• Session Management: Secure token caching and automatic session handling
• User Profiles: Personal accounts with medication tracking
• Sign-out Protection: Confirmation modals and secure session clearing

• Local Processing: Images processed in real-time, not stored permanently
• Secure APIs: Encrypted communication with Google AI services
• No Gesture Interference: Prevents accidental navigation to sensitive states
• Personal Data Protection: User medications stored securely per account

• Node.js 18+ installed
• Expo CLI installed globally: npm install -g @expo/cli
• iOS Simulator or Android Emulator (or physical device with Expo Go)

# Clone the repository
git clone https://github.com/AnamGTR99/medpal_ai.git
cd medpal_ai

# Install dependencies
npm install

# Set up environment variables
cp .env.example .env
# Add your API keys and Clerk configuration to .env (see Setup Guide below)

# Start the development server
npm start

• 📱 iOS: Press i to open in iOS Simulator
• 🤖 Android: Press a to open in Android Emulator
• 🌐 Web: Press w to open in web browser
• 📲 Physical Device: Scan QR code with Expo Go app

1. Go to Clerk.com and create an account
2. Create a new application in your Clerk dashboard
3. Get your Publishable Key from the API Keys section
4. Configure email/password authentication in your Clerk settings

MedPal requires two Google API keys for full functionality:

1. Go to Google Cloud Console
2. Create a new project or select existing one
3. Enable Cloud Vision API
4. Create an API key with Vision API access
5. Important: Enable billing (required even for free tier)

1. Visit Google AI Studio
2. Create a new API key
3. No billing required for Gemini

# Copy the example environment file
cp .env.example .env

Edit .env and add your keys:

# Clerk Authentication
EXPO_PUBLIC_CLERK_PUBLISHABLE_KEY=your_clerk_publishable_key_here

# Google AI Services
GOOGLE_VISION_API_KEY=your_vision_api_key_here
GOOGLE_GEMINI_API_KEY=your_gemini_api_key_here

• Xcode 14+ required for iOS development
• iOS Simulator available through Xcode

• Android Studio with SDK Platform-Tools
• Android Emulator or physical device with USB debugging

MedPal/
├── app/                      # Expo Router file-based navigation
│   ├── _layout.tsx          # Root layout with Clerk authentication
│   ├── index.tsx            # Landing page with sign-in/sign-up options
│   ├── sign-in.tsx          # User sign-in screen with validation
│   ├── sign-up.tsx          # User registration with email verification
│   ├── (tabs)/              # Protected tab navigation screens
│   │   ├── _layout.tsx      # Tab navigation layout
│   │   ├── index.tsx        # Home screen with medication list
│   │   ├── scan.tsx         # Scan tab (opens camera modal)
│   │   └── settings.tsx     # User settings and profile management
│   ├── scan-modal.tsx       # Full-screen camera interface
│   ├── processing.tsx       # Loading screen with AI processing
│   └── confirm-scan.tsx     # Structured results display
├── components/              # Reusable UI components
│   ├── ProtectedRoute.tsx   # Authentication wrapper component
│   ├── ConfirmModal.tsx     # Sign-out confirmation modal
│   ├── ErrorModal.tsx       # Error display modal
│   ├── DateTracker.tsx      # Scrollable date selector
│   ├── MedicationCard.tsx   # Individual medication display
│   ├── PlaceholderIcons.tsx # Icon system
│   └── ScanConfirmModal.tsx # AI result confirmation
├── utils/                   # Utility functions
│   ├── tokenCache.ts        # Clerk secure token storage
│   └── validation.ts        # Form validation helpers
├── services/                # Business logic
│   └── AIService.ts         # Google AI integration
├── state/                   # Global state management
│   └── MedicationContext.tsx # Medication data context
├── theme/                   # Design system
│   └── theme.ts             # Colors, typography, spacing
├── types/                   # TypeScript definitions
│   └── index.ts             # App-wide type definitions
└── test-ai-pipeline/        # Comprehensive testing suite
    └── README.md            # Detailed testing documentation

• Frontend: React Native with Expo
• Language: TypeScript for type safety
• Authentication: Clerk (email verification, session management)
• Navigation: Expo Router (file-based) with protected routes
• UI Library: React Native Paper (Material Design)
• State Management: React Context API
• AI Services: Google Cloud Vision + Gemini AI
• Camera: Expo Camera with live preview
• Styling: StyleSheet with custom theme system

MedPal includes a comprehensive AI pipeline testing framework located in test-ai-pipeline/.

cd test-ai-pipeline

# Test individual components
node test-vision-api.js sample-images/prescription-test.JPG
node test-enhanced-gemini.js --file sample-ocr-text.txt

# Test complete pipeline
node test-enhanced-full-pipeline.js sample-images/prescription-test.JPG

# Batch test multiple images
node test-enhanced-full-pipeline.js --batch sample-images/

• Individual API Testing: Test Vision and Gemini APIs separately
• Full Pipeline Testing: End-to-end image → medication data flow
• Enhanced Analysis: Comprehensive medication information with safety data
• Batch Testing: Test multiple prescription images
• Performance Benchmarking: Timing and accuracy metrics
• Detailed Logging: All results saved to timestamped JSON files

📋 Complete Testing Documentation

graph LR
    A[📷 Camera] --> B[🔍 Vision API]
    B --> C[📝 Text Extraction]
    C --> D[🧠 Gemini AI]
    D --> E[📋 Medication Data]
    E --> F[👤 User Interface]

1. 📷 Image Capture: User photographs any product label
2. 🔍 OCR Processing: Google Vision extracts text with high accuracy
3. 🧠 AI Analysis: Gemini AI analyzes and categorizes product information
4. 📋 Data Structuring: Organized into structured, actionable format
5. 👤 Display: Clean, toggle-able information presented to user

• 💊 Basic Details: Name, doses/servings, frequency, instructions
• 🎯 Treatment/Benefits: What the product helps with or is used for
• ⚠️ Side Effects/Risks: Realistic effects and warnings based on product type
• 📋 Structured Display: Toggle between summary cards and detailed description
• 📸 Visual Context: Shows the actual captured image alongside data
• 🔄 State Management: Fresh analysis for each scan without cached results

• 💊 Prescription Medications: Full medical analysis with dosage and safety info
• 💉 Supplements & Vitamins: Health benefits, dosage recommendations, precautions
• 🚬 Consumables (like nicotine products): Effects, usage, and health considerations
• 🥤 Everyday Products: Basic benefits (like hydration for water) and safety info

Today's major update introduced a complete authentication system powered by Clerk:

• Complete User Authentication: Secure sign-up, sign-in, and session management
• Email Verification: Two-step verification process for new accounts
• Protected Application: All medication features now require authentication
• Custom UI Design: Authentication screens match MedPal's design language
• Enhanced UX: Touch-to-dismiss keyboards, loading states, and error handling
• Security Improvements: Confirmation modals for sign-out with logo branding

• Integrated @clerk/clerk-expo with secure token caching
• Created custom authentication screens (sign-in.tsx, sign-up.tsx)
• Implemented ProtectedRoute component for route protection
• Added form validation with user-friendly error messages
• Built custom modal components (ConfirmModal.tsx, ErrorModal.tsx)
• Updated app navigation structure with authentication flow

• Replaced text with MedPal logo on landing page
• Custom branded authentication screens with logo integration
• Improved modal designs with proper spacing and styling
• Touch-to-dismiss keyboard functionality on auth screens
• Better error handling with contextual messages

• Complete React Native app with Expo Router
• Custom tab navigation with elevated scan button
• Live camera integration with permissions
• Google Cloud Vision API integration (OCR)
• Google Gemini AI integration (analysis)
• TypeScript throughout for type safety
• Material Design UI components with custom theming
• Environment-based configuration with proper Expo integration

• Clerk Authentication Integration: Complete user management system
• Email Verification Flow: Secure account creation with verification codes
• Protected Routes: Authentication-based access control throughout app
• Session Management: Secure token caching with automatic refresh
• Custom Auth Screens: Branded sign-in/sign-up with MedPal design
• Form Validation: Client-side validation with user-friendly error messages
• Sign-out Protection: Confirmation modals with logo and secure session clearing
• Navigation Guards: Prevents unauthorized access to protected screens
• Keyboard Dismissal: Touch-to-dismiss keyboard functionality on auth screens

• Universal Product Intelligence: Analyzes any product type
• Structured Results Display: Toggle between summary and detailed view
• Real Image Integration: Shows captured photos in results
• Gesture-Free Navigation: Button-only flow, no accidental swipe-backs
• Enhanced State Management: Fresh data for each scan
• Professional UI: Grey info boxes with sub-instructions display
• Loading Animation: Custom GIF with fallback support
• 30-Second Timeouts: Robust error handling and recovery
• Contextual Health Info: Realistic benefits/risks for all product types

• Medication reminders and notifications
• Personal medication history tracking
• Multi-language support
• Voice-guided instructions
• Dark mode theme support

• Pill identification by visual recognition
• Integration with health apps (Apple Health, Google Fit)
• Pharmacy integration for refills
• Family account management
• Medication interaction warnings
• Export medication data to PDF reports

We welcome contributions to MedPal! Here's how you can help:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Make your changes and test thoroughly
4. Run the test suite: cd test-ai-pipeline && npm test
5. Commit with clear message: git commit -m 'feat: add amazing feature'
6. Push to your fork: git push origin feature/amazing-feature
7. Open a Pull Request

• 🎨 UI/UX Improvements: Enhance user experience and accessibility
• 🧪 Testing: Add test cases for edge cases and new features
• 🌐 Internationalization: Add support for multiple languages
• 📱 Platform Features: iOS/Android specific optimizations
• 🤖 AI Enhancements: Improve prompt engineering and accuracy
• 📚 Documentation: Improve guides and add tutorials

⚠️ Medical Information Disclaimer

• MedPal is a medication information tool, not a replacement for professional medical advice
• Always consult with healthcare professionals before making medication decisions
• The AI-generated information is for educational purposes only
• In case of medical emergencies, contact emergency services immediately

🔒 Privacy & Data

• Prescription images are processed in real-time and not permanently stored
• API communications are encrypted and secure
• No personal health information is shared with third parties
• Users are responsible for protecting their own medication information

• 📚 Documentation: Comprehensive guides in /test-ai-pipeline/README.md
• 🐛 Issues: Report bugs via GitHub Issues
• 💬 Discussions: Ask questions in GitHub Discussions
• 📧 Contact: For security issues, email the maintainers directly

• Expo Documentation
• React Native Guide
• Google Cloud Vision API
• Google Gemini AI

This project is licensed under the MIT License - see the LICENSE file for details.

• Google AI Platform for providing powerful Vision and Gemini APIs
• Expo Team for the excellent React Native development platform
• React Native Community for comprehensive libraries and tools
• Open Source Contributors who make projects like this possible