An intelligent travel companion that uses AI to understand and document cultural experiences through image analysis, location-aware conversations, and automatic journaling.
Frontend: React Native (Expo), TypeScript, Firebase, Axios
Backend: FastAPI, Python, Gemini 2.5 Flash
AI: Google Gemini Vision API
Storage: Firebase Firestore + Firebase Storage
Location: OpenStreetMap (free reverse geocoding) + Wikipedia geosearch (free)
- User uploads photo → Backend analyzes with Gemini Vision
- Location detected → OpenStreetMap reverse geocoding + Wikipedia nearby search
- Context built → Combines image analysis + location + landmarks
- AI response → Location-aware cultural insights with recommendations
- Storage → Saves to Firestore, creates journal entries
Frontend (React Native) → Backend (FastAPI) ├─→ Gemini AI (Vision + Chat) ├─→ OpenStreetMap (Location) ├─→ Wikipedia (Nearby Attractions) └─→ Firebase (Storage + Database)
Image Processing Pipeline: Photo Upload → Perception Analysis (Gemini Vision) ↓ Geo Detection (OpenStreetMap + Wikipedia) ↓ Context Building (Merge data) ↓ AI Response (Location-aware insights) ↓ Storage (Firestore + Journal)
- Location-Aware: Detects city/country, fetches nearby Wikipedia attractions
- Translation Priority: Translates foreign text in images (signs, menus, etc.)
- Smart Responses: Initial photos get detailed analysis (4 sections), follow-ups are ultra-concise (20 words max)
- Auto Journaling: Conversations automatically saved as travel journals
# Backend
cd backend && python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
uvicorn main:app --host 0.0.0.0 --port 8000 --reload
# Frontend
cd frontend && npm install
npx expo startRequired .env variables:
GEMINI_API_KEY=your_key
FIREBASE_PROJECT_ID=your_project
FIREBASE_API_KEY=your_api_key
FIREBASE_AUTH_DOMAIN=your_domain
FIREBASE_STORAGE_BUCKET=your_bucket
FIREBASE_MESSAGING_SENDER_ID=your_id
FIREBASE_APP_ID=your_idFrontend: app/screens/HomeScreen.tsx - Photo upload + chat interface
Backend:
main.py- FastAPI endpointsutils/perception_utils.py- Image analysisutils/geo_api_utils.py- Location detectionutils/response_utils.py- AI response generationservices/db_service.py- Firestore operations
POST /api/image/process- Upload image, get AI analysisPOST /api/chat- Send chat messageGET /api/journal- Get user journals
Docs: http://localhost:8000/docs
Contributors:
Sujan Katari: Multi-Agent Orchestration + Backend
Nick Gonzalez: Database + Fullstack
Vignesh Sundararajan: Fullstack + Voice Functionality
Ayaan Khan: Image Processing + Frontend (Camera Functionality)