FreshCam - AI-Powered Food Waste Reduction 🍎🤖

Inspiration 💡

Every year, 1.3 billion tons of food is wasted globally, with a significant portion being fresh produce that spoils before consumption. We noticed that people often struggle to determine if their fruits are still good to eat, leading to premature disposal of perfectly edible food or, conversely, consuming spoiled produce.

FreshCam was born from a simple question: "What if your phone could tell you if your fruit is still fresh?"

We wanted to create an accessible, AI-powered solution that empowers everyday consumers to make informed decisions about their produce, reduce food waste, and save money—all while contributing to a more sustainable future.

What It Does 🔍

FreshCam is a mobile application that uses advanced AI to analyze fruit freshness in real-time:

📸 Instant Freshness Detection: Scan any fruit with your camera to get immediate ripeness analysis (unripe, ripe, or overripe)
🍳 Smart Recipe Suggestions: Receive 3 personalized recipes optimized for the fruit's current ripeness level
✅ Safety Assessment: Get clear guidance on whether the fruit is safe to eat
📅 Shelf Life Estimation: Know exactly how many days you have before the fruit spoils
🥗 Nutritional Information: View detailed nutrition facts including calories, vitamins, and minerals
🌍 Environmental Impact: See the carbon footprint and sustainability metrics for each fruit
💾 Storage Tips: Learn the best storage methods to maximize freshness

How We Built It 🛠️

Architecture Overview

FreshCam employs a hybrid AI approach combining computer vision and generative AI:

Backend (Python + FastAPI)

Primary Detection: Roboflow custom-trained CV model for ripeness classification
Fallback Intelligence: Google Gemini 2.0 Flash Experimental for vision analysis when CV model has low confidence
Fruit Identification: Gemini Vision API for accurate fruit name detection
Content Generation: Gemini for recipe creation, safety assessment, and nutritional analysis

Frontend (React Native + Expo)

Cross-platform mobile app with native camera integration
Clean, intuitive UI with color-coded ripeness indicators
Real-time image upload and analysis
Responsive design with ScrollView for extensive content

Infrastructure

ngrok tunneling for secure backend-mobile communication
SQLite database for future pantry management features
RESTful API design with /predict and /recipes endpoints

Technical Implementation

API Integration Strategy

We integrated two complementary AI systems:

Roboflow CV Model: Fast, specialized ripeness detection with $O(1)$ inference time
Gemini 2.0 Flash: Contextual understanding, recipe generation, and safety analysis

This hybrid approach achieves the best of both worlds: speed and accuracy from CV, with intelligent fallback and rich content generation from Gemini.

Challenges We Faced 🚧

1. API Communication & CORS Issues

Initially, our mobile app couldn't communicate with the backend due to network restrictions.

Solution: Implemented ngrok tunneling to create a secure public endpoint, allowing the React Native app to communicate with our local FastAPI server.

2. JSON Parsing Errors

The frontend was receiving HTML error pages instead of JSON responses, causing SyntaxError: JSON Parse error: Unexpected character: <.

Solution: Added proper error checking with res.ok validation before attempting to parse JSON, and removed manual Content-Type headers from FormData uploads.

3. Gemini API Safety Settings

Initial attempts to send food images to Gemini resulted in KeyError exceptions due to incorrect safety settings configuration.

Solution: Updated to use proper enum-based safety settings.

4. Model Reliability & Confidence

The CV model sometimes failed to detect fruits or returned low-confidence predictions.

Solution: Implemented a smart fallback system where Gemini takes over when CV confidence is below threshold, ensuring users always get a result.

5. Port Conflicts & Server Management

Multiple backend instances running simultaneously caused [Errno 10048] port binding errors.

Solution: Implemented proper process management to kill existing servers before starting new instances.

What We Learned 📚

Hybrid AI Systems: Combining specialized models with general-purpose AI creates more robust applications
Mobile-Backend Communication: Understanding tunneling, CORS, and API security in mobile development
Prompt Engineering: Crafting effective prompts for Gemini to generate accurate, actionable content
Error Handling: The importance of graceful degradation and informative error messages
User Experience: Balancing feature richness with simplicity—removed incomplete features (pantry) to maintain focus

What's Next for FreshCam 🚀

Multi-Fruit Detection: Analyze entire fruit bowls in a single scan
Pantry Management: Track all your fruits with expiration reminders
Personalized Recommendations: Learn user preferences for better recipe suggestions
Grocery Shopping Assistant: Scan fruits at the store to pick the perfect ripeness
Community Features: Share recipes and tips with other users
Expanded Food Database: Support for vegetables, dairy, and other perishables

FreshCam isn't just an app—it's a step toward a more sustainable future, one scan at a time. 🌱