An intelligent AI-powered learning platform that provides real-time feedback on student work through voice assistance, PDF annotation, and conversational AI tutoring.

• 📄 Smart PDF Workspace - Upload PDFs and annotate them with an integrated drawing canvas
• 🎨 Real-time Drawing - Draw, highlight, and mark up documents with multi-page support
• 🔊 Voice Interaction - Ask questions using voice input and receive audio feedback
• 🤖 AI-Powered Analysis - Automatic OCR processing and error detection using Mathpix & Google Gemini
• 💬 Conversational AI Tutor - Context-aware tutoring with conversation memory
• 🎵 Natural Voice Feedback - Text-to-speech responses using ElevenLabs
• 📤 Export Functionality - Save annotated PDFs with your work
• 🎯 Auto-capture - Automatic screenshot capture 5 seconds after drawing stops

• Next.js 15.5 - React framework with Turbopack
• React 19 - UI library
• Tailwind CSS 4 - Styling
• React PDF - PDF rendering
• Fabric.js - Canvas drawing
• React Dropzone - File uploads

• FastAPI - Python web framework
• Google Gemini AI - Conversational AI and analysis
• Mathpix OCR - Handwriting and math formula recognition
• ElevenLabs - Text-to-speech synthesis
• Uvicorn - ASGI server

• Node.js 18.x or higher
• Python 3.8 or higher
• npm or yarn
• API Keys (see Configuration section)

1. Clone the repository

   git clone https://github.com/your-username/FUJI-hackathon.git
   cd FUJI-hackathon

2. Set up the Frontend

   cd tutor
   npm install

3. Set up the Backend

   cd ../backend
   pip install -r requirements.txt

Create a .env file in the backend/ directory:

# Required API Keys
GEMINI_API_KEY=your_google_gemini_api_key_here
MATHPIX_APP_ID=your_mathpix_app_id_here
MATHPIX_APP_KEY=your_mathpix_app_key_here
ELEVENLABS_API_KEY=your_elevenlabs_api_key_here

# Optional: ElevenLabs Voice ID (default provided in code)
ELEVENLABS_VOICE_ID=your_preferred_voice_id

Get your API keys:

• Google Gemini: https://makersuite.google.com/app/apikey
• Mathpix: https://mathpix.com/
• ElevenLabs: https://elevenlabs.io/

Create a .env.local file in the tutor/ directory:

NEXT_PUBLIC_API_URL=http://localhost:8000

cd backend
uvicorn main:app --reload --port 8000

The API will be available at http://localhost:8000

In a new terminal:

cd tutor
npm run dev

The application will be available at http://localhost:3000

1. Navigate to the Workspace page
2. Upload a PDF document using drag-and-drop or the "Add PDF" button
3. Enable Drawing Mode to annotate the document
4. Your work will be automatically analyzed after 5 seconds of inactivity

1. Click the "Talk" button to start voice recognition
2. Ask your question or describe your problem
3. The system will:
   • Capture the current PDF page
   • Process handwriting with OCR
   • Analyze your work with AI
   • Provide audio and text feedback

1. Navigate to the Tutor page
2. Type your questions in the chat interface
3. Receive instant AI-powered responses
4. Context is maintained throughout the conversation

1. Click the "Export" button in the workspace
2. Choose to export:
   • Current page only
   • All pages with annotations
3. Download the annotated PDF

Primary Colors:

• Indigo: #4F46E5 (indigo-600)
• Purple: #9333EA (purple-600)
• Pink: #EC4899 (pink-600)

Gradients:

• Primary: from-indigo-600 to-purple-600
• Extended: from-indigo-600 via-purple-600 to-pink-600

FUJI-hackathon/
├── backend/                 # FastAPI backend
│   ├── main.py             # Main API routes and logic
│   ├── requirements.txt    # Python dependencies
│   └── .env                # Environment variables (create this)
│
├── tutor/                  # Next.js frontend
│   ├── src/
│   │   ├── app/           # Next.js app directory
│   │   │   ├── page.js           # Home page
│   │   │   ├── work/page.js      # PDF workspace
│   │   │   ├── tutor/page.js     # AI chat tutor
│   │   │   └── login/page.js     # Login page
│   │   └── components/    # React components
│   │       ├── Navbar.js         # Navigation bar
│   │       └── DrawingCanvas.js  # Canvas component
│   ├── public/            # Static assets
│   └── .env.local         # Environment variables (create this)
│
└── README.md              # This file

• POST /api/ocr - Process image with Mathpix OCR
• POST /api/analyze - Analyze student work with Gemini AI
• GET /api/voice - Get voice settings
• POST /api/voice - Update voice settings
• POST /api/chat - Chat with AI tutor

1. Student uploads PDF → Renders in workspace
2. Student draws/annotates → Auto-capture after 5 seconds
3. Screenshot sent to Mathpix → OCR extracts LaTeX/text
4. Data sent to Gemini AI → Analyzes work and context
5. AI generates feedback → Text response created
6. ElevenLabs TTS → Converts to natural speech
7. Student receives → Audio playback + visual feedback panel

Port already in use:

uvicorn main:app --reload --port 8001

API keys not working:

• Double-check .env file is in the backend/ directory
• Ensure no extra spaces in API keys
• Verify API keys are active and have credits

PDF not rendering:

• Check that pdf.worker.min.js is in the public/ folder
• Clear browser cache and reload

Voice not working:

• Use Chrome or Safari (required for Web Speech API)
• Grant microphone permissions
• Check if backend is running

This project was created for the FUJI Hackathon.

• Usmaan Sayed
• Iqbal Ghanci
• Furqan Ahcom
• Jonathan Robin

• Google Gemini AI for intelligent tutoring
• Mathpix for OCR capabilities
• ElevenLabs for natural voice synthesis
• Next.js and FastAPI communities

Made with ❤️ for students everywhere