Aeris is a real-time environmental intelligence system built for HackAugie, where it won Best Data Insight. It detects visible waste, normalizes local environmental context, and returns a practical sustainability recommendation that still works when LLM providers are unavailable.

🏆 Winner: HackAugie Best Data Insight

Aeris is not production recycling infrastructure. The portfolio signal is the system design: a clear boundary between vision detections, environmental context, LLM-backed advice, deterministic fallback behavior, and cache-aware backend orchestration.

• Hackaton Submission: Devpost
• Demo video: Youtube
• Architecture: docs/architecture.md
• Local dev setup: docs/local-dev-setup.md
• License: MIT

Aeris answers one live question:

What is this object, what local context matters, and what should someone do right now?

The current project-ready flow is:

Camera or uploaded clip
  -> Streamlit WebRTC / upload processor
  -> local YOLO waste detector
  -> latest structured detection
  -> FastAPI environmental context layer
  -> Gemini or Anthropic advice
  -> deterministic fallback and cache
  -> sustainability recommendation in the UI

The custom detector focuses on:

• can
• paper
• bottle

The environmental context layer combines:

• CASTNET processed readings
• Open-Meteo weather
• Open-Meteo air quality
• weather.gov alerts
• derived risk flags

• Deterministic: environmental context normalization, risk flags, API contracts, cache behavior, and fallback recommendation policy.
• Model-based: YOLO waste detection for cans, paper, and bottles.
• AI-assisted: Gemini or Anthropic recommendation wording when provider keys are configured.
• Bounded: the LLM does not own object detection, environmental measurements, risk flags, or fallback behavior.

Aeris is intentionally split into three boundaries:

The important engineering choice is that raw video is not sent to the reasoning layer. The recommendation path receives structured detections and environmental context, which keeps the pipeline debuggable and avoids calling an LLM on every frame.

backend/
  app/main.py                         FastAPI routes
  app/context/                        CASTNET/weather/air-quality context
  app/sustainability/adviser.py       Gemini/Anthropic/fallback advice cache
  app/cv/yolo_service.py              Image scan service path
  streamlit_app.py                    Primary live scanner and upload UI
  scripts/                            YOLO training, realtime, and dataset utilities

ui/
  src/pages/Index.tsx                 Optional React shell / Streamlit embed
  src/components/aeris/               Portfolio UI components
  src/vision/                         Browser/backend YOLO experiments

data/
  castnet/processed/                  Demo CASTNET profiles/readings
  sample_inputs/                      Demo scene fixtures

tests/                                Backend API and policy tests
scripts/                              Root-level backend smoke and data utilities
assets/                               Demo/reference assets

docs/
  architecture.md                     Current system design
  trash-model.md                      Custom model training record
  system-audit.md                     Portfolio-readiness audit

archive/legacy-ui/
  frontend/                           Earlier Vite/React experiment
  aeirs-ui/                           Earlier Next.js experiment
  aeris-ui-scratch/                   Scratch workspace notes only

Earlier UI experiments are archived under archive/legacy-ui/ for provenance. The current maintained UI paths are backend/streamlit_app.py and, optionally, ui/.

Install backend/UI dependencies once using docs/local-dev-setup.md, then start the local demo from the repo root:

powershell -ExecutionPolicy Bypass -File .\scripts\dev.ps1

This starts FastAPI at http://127.0.0.1:8000/docs and Streamlit at http://127.0.0.1:8507. Add -WithUi to also start the optional React shell.

Use the root .env as the main local configuration file:

Copy-Item .env.example .env

Aeris works without LLM keys because deterministic fallback advice remains available. The strongest documented checkpoint is backend/models/trash-quick-v4-best.pt; it is intentionally gitignored because model artifacts can be large.

Backend policy smoke tests:

powershell -ExecutionPolicy Bypass -File .\scripts\smoke.ps1

This includes offline API contract tests for the FastAPI boundary and backend policy tests for event/fallback behavior.

UI tests and build:

powershell -ExecutionPolicy Bypass -File .\scripts\smoke.ps1 -WithUi

• Streamlit remains the primary live scanner because it was the fastest reliable way to connect camera input, YOLO inference, and backend recommendations during the hackathon.
• Model checkpoints are local artifacts and are intentionally gitignored; the documented path is backend/models/trash-quick-v4-best.pt.
• Legacy UI experiments are kept under archive/legacy-ui/ for provenance, but the maintained demo paths are backend/streamlit_app.py and ui/.
• The recommendation layer favors a small, inspectable fallback path over broad autonomous-agent behavior.

• @postigodev: FastAPI layer, environmental context integration, recommendation pipeline, LLM/fallback/cache orchestration, and glue between Streamlit/YOLO detections and the rest of the system.
• @shuja-waraich-03: AI integration, real-time synchronization between vision and reasoning, Gemini prompt structure, and live detection validation.
• @kacytran1122: Frontend work, React camera detection, landing page, responsive interaction design, and user experience polish.
• @GALGALLOR: Computer vision pipeline, dataset preparation, YOLO fine-tuning, real-time inference performance, and Streamlit integration.