SketchFlow

Inspiration

We were inspired by the need for a seamless, intelligent, and intuitive digital whiteboard, something that could merge manual sketching, AI assistance, and visual organization all in one place. Tools like Miro and Google Jamboard fall short for users who want the flexibility to draw, the precision of object manipulation, and the magic of natural language input. We envisioned SketchFlow as a solution, a canvas where you can draw freely, describe what you want, and see it come to life instantly.

What it does

SketchFlow is a smart whiteboard tool where users can:

Draw shapes, lines, and text freely using intuitive tools
Multi-select and group objects, move them in sync, and rearrange them easily
Use natural language prompts to generate diagrams via AI (e.g. "Draw a binary search tree"
Right-click anywhere to bring up a custom prompt modal to interact with the AI
Visualize SDG concepts like climate action or sustainable cities through sketch prompts

How we built it

Frontend: Built using React and Konva for interactive canvas rendering
Backend: FastAPI for parsing prompts and returning structured shape objects
AI Integration: Google Gemini 1.5 Flash for interpreting user prompts and converting them to JSON
Canvas Logic: Custom logic for multi-select, bounding box detection, stage transforms, layering, and group-based movement

Challenges we ran into

Making multi-select and group movement work correctly after camera panning
Preventing render glitches and state mismatches when switching tools quickly
Guiding Gemini to generate consistent and valid JSON output for shape rendering
Ensuring layering was preserved so that selections worked predictably
Building a modal that didn't interfere with clicks, inputs, or other tool states
Avoiding ghost-selection after the canvas was moved (stage offset bugs)

Accomplishments that we're proud of

Creating an AI-enhanced drawing environment that can interpret prompts and turn them into functional shapes
Implementing robust multi-shape selection, grouping, and movement logic
Solving nuanced bugs like cursor-offset mismatch after zoom/pan, and persistent state issues
Designing an interface that looks cohesive and matches the sketch aesthetic
Supporting SDG storytelling with custom AI prompt interpretations

What we learned

Advanced use of Konva layers, groups, and coordinate transformations
Building a modular backend parser that mixes rule-based and AI-based approaches
How to coax LLMs to return deterministic JSON output with reliable schemas
Managing React state across complex interaction patterns with multiple tools active
How to make UX feel "natural" across pen drawing, object selection, and modal interaction