SnapFarm: Instant AI Plant Health Diagnostics

About the Project

SnapFarm is a client-side AI tool that helps hobbyist gardeners, indoor farmers, and educational labs monitor plant health. It combines image analysis with environmental data like humidity, temperature, light, pH, and plant age to diagnose diseases and nutrient deficiencies. SnapFarm provides actionable recommendations and tracks sustainability metrics such as water saved and pesticide avoided.

Inspiration

The project was inspired by the challenges small-scale growers face. Late detection of plant diseases often leads to unnecessary water, fertilizer, and pesticide use. We wanted to create a lightweight, offline solution that is accessible, fast, and responsible, empowering growers to make informed decisions without relying on expensive hardware or cloud services.

What We Learned

How to build multi-input machine learning models in TensorFlow.js that combine image and metadata.
Techniques for automatic leaf cropping, including HSV-based green masks and object detection fallbacks.
How to compute real-world sustainability metrics based on model predictions.
Handling offline-first, client-side architecture, including model loading, inference, and local storage tracking.
Integrating TypeScript, React, and TailwindCSS for a seamless and responsive user experience.

How We Built It

Multi-Input Model: Fine-tuned MobileNetV3 on the PlantVillage tomato dataset for the image branch and a small MLP for environmental metadata.
Automatic Cropping: Implemented HSV-based green masks to detect the main leaf. Added COCO-SSD as a fallback when the mask failed.
Inference Pipeline: Preprocessed both images and metadata, ran predictions in-browser, and extracted top-3 disease/nutrient predictions.
Business Metrics: Computed water saved and pesticide avoided using confidence-weighted formulas and stored results in local storage. Displayed tracking charts via react-chartjs-2.
UI & UX: Built a sleek, mobile-first React interface using TailwindCSS, allowing users to upload images, see predictions, and track sustainability over time.

Challenges

Leaf Detection: Creating a reliable crop method that works for varied lighting and plant sizes. Solved using a hybrid approach of green masks and fallback object detection.
Client-Side Performance: Ensuring the model runs fast in-browser. Optimized with TensorFlow.js WebGL backend and strategic await tf.nextFrame() calls.
Offline Data Tracking: Designing local storage schemas for both predictions and chat messages while keeping the data persistent and type-safe.
Model Integration: Combining image and metadata inputs in a single TF.js model and ensuring predictions are interpretable and actionable.

Key Takeaways

SnapFarm demonstrates how client-side AI can make plant health diagnostics accessible and sustainable. By combining images, environmental factors, and actionable recommendations, the project bridges the gap between technology and practical gardening solutions. It highlights the potential of lightweight, offline-first AI applications that deliver measurable impact.