vision_ai_animals #

On-device animal detection, species/breed classification, and pure-Dart analytics for Flutter — built on the vision_ai engine (camera + native ML pipeline). Android + iOS, fully on-device, no cloud.

Features #

• Detect & locate animals — bounding box + COCO class label + score per animal, via the MediaPipe Object Detector (EfficientDet-Lite0), LIVE_STREAM mode.
• Breed / species (optional) — crops each classifiable detection and runs a MediaPipe Image Classifier (EfficientNet-Lite, ImageNet — ~120 dog breeds and several cat types).
• Pure-Dart analytics (zero model weight) — multi-animal tracking IDs, counting, enter/exit events, motion + activity level, proximity, and zone alerts.
• Overlay widgets — AnimalCameraView, boxes, labels, and zone painting.
• Bring your own model — swap in any compatible TFLite detector/classifier.

Installation #

dependencies:
  vision_ai_animals: ^0.1.0

vision_ai_animals pulls in vision_ai (the engine) and bundles the animal models — an animal-only app needs nothing else. (For hand/face detection too, also add vision_ai_models.)

Android release builds: MediaPipe crashes with R8 shrinking — disable it in android/app/build.gradle.kts:

android { buildTypes { release { isMinifyEnabled = false; isShrinkResources = false } } }

Add camera permission (Android AndroidManifest.xml / iOS Info.plist) — see the vision_ai README.

Quick start #

import 'package:vision_ai_animals/vision_ai_animals.dart';

final vision = await AnimalVision.create(
  animal: const AnimalConfig(detectBreed: true),
);

final textureId = await vision.start();
vision.results.listen((result) {
  for (final a in result.animals) {
    print('${a.label} ${(a.confidence * 100).round()}% ${a.breed ?? ''}');
  }
});

Render the preview + overlays with AnimalCameraView:

AnimalCameraView(
  results: vision.results,
  textureId: textureId,
  mirrored: false, // true for the front camera
)

Configuration (AnimalConfig) #

Update the non-start-time-only fields at runtime with vision.updateAnimalConfig(...) (the model, breed-classifier options, locale, and rotation are fixed at start).

Each AnimalResult also carries categoryIndex and a localized displayName alongside label.

Result (AnimalResult) #

label, confidence, boundingBox (image pixels), breed / breedConfidence / breedTopK (null unless breed ran), and trackingId (assigned by AnimalTracker).

Analytics #

Pure-Dart, zero model weight. Run AnimalTracker first each frame so the others receive stable IDs.

final tracker  = AnimalTracker();
final counter  = AnimalCounter();
final presence = AnimalPresenceDetector();
final motion   = AnimalMotionTracker();
final zones    = ZoneAlertDetector(zones: [Rect.fromLTWH(0, 0, 320, 480)]);

vision.results.listen((r) {
  tracker.update(r.animals);                    // assigns trackingId in place
  final counts = counter.count(r.animals);      // {dog: 2, cat: 1}, total: 3
  for (final e in presence.update(r.animals)) print(e); // entered / left
  final m = motion.update(r.animals, r.timestampMs, r.imageSize); // velocity + activity
  for (final z in zones.update(r.animals)) print(z);    // enteredZone / leftZone
});

Overlays #

AnimalCameraView stacks a Texture preview with an AnimalBoxPainter (boxes, class/breed labels, confidence bar, optional zones) and a floating AnimalLabel for the primary animal. Style everything via AnimalOverlayStyle. Use the overlayBuilder escape hatch for custom UI.

Bring your own model #

Swap in your own TFLite object detector and/or classifier:

final vision = await AnimalVision.create(
  animal: const AnimalConfig(
    detectBreed: true,
    // A null allowlist applies the COCO animals-only preset, which filters out
    // non-COCO classes. For a custom detector, allow every class it emits:
    allowedCategories: allowAllCategories, // or your own {'fox', 'deer', ...}
    // Sharpen a general (e.g. ImageNet) breed model: drop low-confidence guesses
    // and constrain its output to the labels you care about.
    breedScoreThreshold: 0.3,
    breedAllowedCategories: {'beagle', 'pug', 'Persian cat'},
    displayNamesLocale: 'en',
  ),
  customDetectorPath: '/path/to/your_detector.tflite',
  customClassifierPath: '/path/to/your_breed.tflite',
);

When customDetectorPath / customClassifierPath are supplied, the matching bundled asset is not copied — your path is passed straight to the engine.

Model contract (both platforms load via MediaPipe Tasks):

• Format: a .tflite model with embedded TFLite metadata, including the label map. Result labels come from that metadata — there is no separate labels file.
• Detector: a MediaPipe-compatible object detector (EfficientDet-Lite or SSD-MobileNetV2 family).
• Classifier: a MediaPipe-compatible image classifier (e.g. EfficientNet-Lite).
• Input: MediaPipe resizes/normalizes to the model's input tensor internally.

Analytics for custom classes: map your labels with AnimalGrouping:

const grouping = AnimalGrouping(petLabels: {'beagle'}, wildlifeLabels: {'fox'});
final kind = grouping.kindOf(animal.label);

Performance & concurrency #

• Animal detection runs in LIVE_STREAM (async) — vision_ai self-throttles via the camera backpressure strategy, so slow inference drops frames instead of queueing.
• Animal detection is best used as its own mode. Enabling it alongside hand/face runs three models on one analysis thread; it works but costs FPS, so treat it as a deliberate choice.
• Breed classification reuses the detector's frame (region-of-interest crop) — no extra per-frame bitmap allocation.

Footprint #

Bundles the animal models only; it does not pull in the hand/face models. The bundled models are the float32 MediaPipe variants (~14 MB detector, ~19 MB classifier). To shrink the app, supply the int8 variants — or your own models — via the bring-your-own-model path and skip the bundled assets.

Platform support #

License #

Apache License 2.0. Bundled models are distributed by Google under the Apache 2.0 license.