Making cities accessible for everyone through the power of AI and geospatial intelligence
Built for UofTHacks 13
Accessibility isn't just about disabilities—it affects everyone. Parents pushing strollers, elderly individuals with walkers, travelers with luggage, people using canes, wheelchair users, and even children learning to walk all face the same urban obstacles. Broken sidewalks, missing ramps, and blocked pathways create barriers that impact millions of people daily. Yet municipal teams struggle to identify and prioritize these repairs across vast urban landscapes.
Communify transforms accessibility reporting through intelligent automation. Citizens snap a photo or video of a barrier, and our AI instantly analyzes, categorizes, and routes the report to the right municipal team—complete with repair cost estimates and severity assessments.
|
Multi-Modal AI Intelligence at Scale We leverage Gemini 2.0 Flash across three distinct use cases: 1. Vision Analysis Pipeline
2. Intelligent Search Agent (LangGraph)
3. Semantic Embeddings
# Multi-agent architecture with Gemini LLM
Intent Analyst → Search Specialist → Supervisor
↓ ↓ ↓
(Gemini) (Tools/FAISS) (Aggregation) |
Geospatial Intelligence & Real-Time Data Our entire data architecture is built on MongoDB Atlas with advanced geospatial capabilities: GeoJSON-Powered Reports {
location: {
type: 'Point',
coordinates: [-79.3832, 43.6532] // Toronto
},
aiDraft: { /* Gemini analysis */ },
content: { /* User-verified data */ },
routing: { /* Auto-assigned area */ }
}Administrative Area Matching
Key Capabilities:
Collections:
|
Optimized Media Pipeline with Intelligent Transformations
Our media infrastructure leverages Cloudinary for production-grade image handling:
| Feature | Implementation |
|---|---|
| Direct Upload | Client-side signed uploads with progress tracking |
| Auto-Optimization | q_auto,f_auto delivers WebP/AVIF based on browser |
| Responsive Images | Thumbnails (400px), Previews (800px), Full (1600px) |
| Format Conversion | HEIC → JPEG conversion for iOS compatibility |
| Graceful Fallback | Base64 encoding when Cloudinary unavailable |
// Cloudinary transformation presets
const TRANSFORMS = {
thumbnail: 'w_400,q_auto,f_auto', // Fast feed rendering
preview: 'w_800,q_auto,f_auto', // Modal previews
full: 'w_1600,q_auto,f_auto' // Full detail view
};
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
│ COMMUNIFY │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────────┐ │
│ │ Frontend │───▶│ Next.js API │───▶│ Google Gemini API │ │
│ │ Next.js │ │ Routes │ │ ┌────────────────────────┐ │ │
│ │ React 19 │ └──────────────┘ │ │ gemini-2.0-flash │ │ │
│ │ Mapbox GL │ │ │ │ (Vision + LLM) │ │ │
│ └──────────────┘ │ │ ├────────────────────────┤ │ │
│ │ │ │ │ text-embedding-004 │ │ │
│ │ ▼ │ │ (Semantic Search) │ │ │
│ │ ┌──────────────┐ │ └────────────────────────┘ │ │
│ │ │ MongoDB │ └──────────────────────────────┘ │
│ │ │ Atlas │ │
│ │ │ ┌──────────┐ │ ┌──────────────────────────────┐ │
│ │ │ │ reports │ │ │ Cloudinary │ │
│ │ │ │ (GeoJSON │ │ │ ┌────────────────────────┐ │ │
│ └────────▶│ │ Points) │ │◀─────▶│ │ Image Optimization │ │ │
│ │ ├──────────┤ │ │ │ Auto-format (WebP) │ │ │
│ │ │ areas │ │ │ │ Responsive transforms │ │ │
│ │ │(Polygons)│ │ │ └────────────────────────┘ │ │
│ │ └──────────┘ │ └──────────────────────────────┘ │
│ └──────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
| Layer | Technology |
|---|---|
| AI/ML | Google Gemini 2.0 Flash (Vision + LLM), Gemini Embeddings, FAISS Vector Store |
| Database | MongoDB Atlas with Geospatial Indexing |
| Media | Cloudinary (CDN + Transformations) |
| Frontend | Next.js 16, React 19, TypeScript, Tailwind CSS |
| Maps | Mapbox GL JS + Mapbox Draw |
| Backend | Next.js API Routes, FastAPI (Python agents) |
| Agents | LangGraph Multi-Agent Orchestration, LangChain |
| Endpoint | Method | Description |
|---|---|---|
/api/reports |
GET | List all reports with geospatial data |
/api/reports |
POST | Create report with auto area-routing |
/api/areas |
GET/POST | Manage administrative polygons |
/api/analyze |
POST | Gemini Vision analysis |
/api/cloudinary/signature |
GET | Signed upload params |
/api/db/init |
POST | Initialize geospatial indexes |
uofthacks-13/
├── frontend/
│ ├── src/
│ │ ├── app/
│ │ │ ├── page.tsx # Main map interface
│ │ │ ├── admin/page.tsx # Admin dashboard
│ │ │ └── api/ # Next.js API routes
│ │ │ ├── analyze/ # Gemini Vision endpoint
│ │ │ ├── reports/ # MongoDB CRUD
│ │ │ └── cloudinary/ # Media upload
│ │ ├── components/ # React components
│ │ ├── lib/
│ │ │ ├── ai/gemini.ts # Gemini API client
│ │ │ ├── mongodb.ts # Database connection
│ │ │ └── cloudinary.ts # Media utilities
│ │ └── hooks/ # Custom React hooks
│ └── package.json
├── agent_backend/
│ ├── agent.py # LangGraph orchestration
│ ├── agents/intent_analyst.py # Gemini LLM agent
│ ├── embeddings.py # Gemini embeddings + FAISS
│ ├── tools.py # Search tools
│ └── db.py # MongoDB connection
└── README.md
Our Gemini-powered analysis classifies barriers into 19 specialized categories:
|
|
|