Onboarding
Sign Up screen
Login
Home screen
scan screen
Image preview
Result screen
Comparison screen
profile page
Comparison screen

The Story of Labelly: Health Analysis from Ingredients Labels

Inspiration

Labelly was inspired by the everyday challenge of understanding what’s really inside the products we buy—be it food, cosmetics, or supplements. We wanted to empower users to make healthier choices, instantly, by simply scanning an ingredient label. Our goal was to combine the simplicity of a photo-based interface with the intelligence of modern AI, making health analysis accessible to everyone.

What It Does

Labelly is a mobile app that lets users analyze the healthiness of any packaged product by taking a photo of its ingredient label. The app uses advanced AI to provide:

A clear health score (e.g., Healthy, Moderate, Unhealthy)
Bullet-pointed insights about the product’s ingredients
Warnings about additives or allergens

All of this is delivered in a clean, minimal interface—no manual text entry or ingredient research required.

How We Built It

App Flow

Onboarding & Authentication
- New users are greeted with a splash and onboarding screen, introducing Labelly’s mission: “Know your product.”
- Users must sign up or log in using email/password authentication (powered by Firebase). Anonymous use is not supported, ensuring a personalized and secure experience.
Main Navigation
- After login, users land on the main app interface, organized with tab navigation for easy access to core features.
Scanning Ingredients
- The heart of the app is the “Scan Ingredients” button, always accessible from the main screen.
- Tapping this launches the camera interface, where users can snap a photo of an ingredient label or pick an image from their gallery.
- The camera experience is smooth, with controls for flash, retake, and gallery access.
AI-Powered Analysis
- Once an image is captured, it’s sent to the backend, which forwards it to Perplexity’s Sonar API for analysis.
- The app displays a loading indicator while the image is being processed.
Results & Insights
- The analysis result is parsed and presented in a user-friendly summary:
  - A health score bar visualizes the proportion of safe, low-risk, not-great, and dangerous ingredients.
  - Bullet-pointed insights and warnings are shown.
  - Users can explore ingredient categories in detail, compare products, or view suggested alternatives.
History & Personalization (Planned)
- Future versions will allow users to view their scan history, mark favorites, and share results.

Technical Stack

Frontend: React Native with Expo CLI for rapid, cross-platform development. UI is built with Tamagui for consistent styling and smooth animations.
Backend: Flask (Python) acts as the bridge between the app and third-party services.
AI Integration: Perplexity Sonar API for image-based ingredient analysis.
Authentication & Storage: Firebase Authentication for secure login, Firestore for scan logs and preferences, and optional Cloud Storage for image archiving.
Testing & CI/CD: Jest for unit testing, GitHub Actions for automated linting, type checking, and build artifact generation.

Challenges We Ran Into

Seamless Camera Experience: Ensuring a smooth, reliable camera and gallery picker flow across devices required careful handling of permissions, errors, and UI states.
AI API Integration: Working with a third-party AI model meant handling variable response times and formats, as well as robust error and retry logic.
Authentication Flow: Creating a secure, user-friendly authentication experience with Firebase, including edge cases for sign-up, login, and error handling.
Result Presentation: Designing a results screen that is both informative and easy to understand, translating complex AI output into actionable insights.

Accomplishments That We're Proud Of

End-to-End User Flow: Users can go from onboarding to actionable health insights in just a few taps, with a frictionless experience.
Modern, Minimal UI: The app’s interface is clean, intuitive, and visually appealing, inspired by the best in health tech.
Robust Architecture: The use of modern frameworks and cloud services ensures scalability, security, and maintainability.
Hackathon-Ready MVP: We delivered a fully functional MVP within the hackathon timeframe, with a clear roadmap for future enhancements.

What We Learned

The Power of AI APIs: Leveraging advanced models like Sonar can dramatically simplify complex tasks, such as ingredient analysis, without building custom ML pipelines.
Importance of Error Handling: Real-world APIs are unpredictable; robust error handling and user feedback are essential for a smooth experience.
Value of Clean Design: A minimal, focused UI not only looks better but also makes the app more usable and approachable.
Rapid Prototyping with Expo: Expo CLI enabled us to iterate quickly and test on real devices with ease.

What's Next for Labelly

OCR Integration: To reduce costs and improve flexibility, we plan to add OCR-based text extraction as an option.
Scan History & Favorites: Users will soon be able to save and revisit past scans, and mark favorite products.
Sharing & Social Features: We aim to let users share analysis results with friends and family.
iOS & Web Support: Expanding beyond Android, we’ll bring Labelly to iOS and evaluate web support post-MVP.
Enhanced Analytics & Personalization: Future versions will offer more personalized insights and recommendations based on user preferences and scan history.

Labelly is just getting started. Our mission is to make healthy choices easier for everyone, one label at a time. Thank you for following our journey!