Snapshot of our Website

Draw-And-Solve

An interactive AI-powered web application that allows users to draw mathematical and physics equations on a canvas, upload the image, and get step-by-step solutions using Google Gemini AI. Users can also chat with AI to clarify doubts.

Inspiration

While solving complex math and physics problems, we realized that typing equations manually is time-consuming and frustrating. Many existing math solvers require specific input formats, making it difficult for students working with handwritten equations and diagrams.

Our goal was to bridge this gap by creating an AI-powered canvas-based tool where users can draw equations naturally, making problem-solving more intuitive and interactive.

What It Does

Canvas-Based Input – Users can draw math/physics equations instead of typing.
AI-Powered Problem Solving – Google Gemini AI extracts, solves, and explains equations step by step.
Interactive Chat – Users can ask follow-up questions and get detailed explanations.
User-Friendly Interface – Simple React + Tailwind CSS design for a smooth experience.
Flask Backend – Handles image processing, AI integration, and chat interactions.

How We Built It

Frontend (React + Tailwind CSS)

Canvas Drawing – Users can sketch equations using the React Canvas API.
Image Upload System – Converts the drawing into an image file and sends it to the backend.
Chat System – Allows users to interact with AI after receiving solutions.
State Management – Uses useContext, useEffect, and useState for dynamic UI updates.

Backend (Flask + Python)

Image Processing – Uses PIL (Pillow) and BytesIO to process and encode images.
AI Integration – Sends the image to Google Gemini AI for equation extraction and solving.
Flask API Endpoints – Handles requests from the frontend and returns AI-generated solutions.
Chat Functionality – Allows users to send follow-up questions and receive AI responses.

Challenges We Ran Into

1. Extracting Equations from Images

Problem: Ensuring AI could accurately recognize handwritten equations.
Solution: We optimized image preprocessing using grayscale conversion and adaptive thresholding before sending it to AI.

2. Handling Large Image Files Efficiently

Problem: Large image files caused slow API response times.
Solution: We used image compression and Base64 encoding to reduce file sizes.

3. Ensuring a Smooth Chat Experience

Problem: Making the chat system interactive while keeping AI responses meaningful.
Solution: We structured API calls properly and implemented state management for real-time updates.

Accomplishments That We're Proud Of

Successfully integrated Google Gemini AI to analyze handwritten equations.
Developed a canvas-based drawing system for users to input problems naturally.
Implemented a chat feature that allows users to ask follow-up questions and clarify doubts.
Optimized image processing and API calls to ensure fast response times.
Built a seamless frontend-backend communication system using React and Flask.

What We Learned

AI API Integration – How to process images and send them to Google Gemini AI for meaningful outputs.
React State Management – Using useContext, useEffect, and useState for smooth UI updates.
Image Processing Techniques – Base64 encoding, compression, and grayscale conversion for efficient handling.
Building a Dynamic Chat System – Implementing an AI-powered interactive chat for better user engagement.
Backend Optimization – Efficient handling of API requests and responses using Flask.

What's Next for Draw-And-Solve?

Real-time Collaboration – Allow multiple users to solve equations together in real time.
Multi-Language Support – Enable AI to provide solutions in multiple languages.
Handwriting Recognition AI – Improve handwritten equation detection for better accuracy.
Voice-Based Input – Allow users to speak their equations, making it even more intuitive.
Mobile-Friendly Version – Optimize the UI for better accessibility on mobile devices.