SigmaGuide

F*** SCREENSHOTS! SigmaGuide does it all!

Comment

Inspiration

We've all been there — stuck in unfamiliar software, Googling "how to freeze row in Excel," scrolling through outdated screenshot tutorials that don't match our version. It's 2026, and we're still learning software the same way we did in 2010.

We asked: What if your computer could just show you where to click?

What it does

SigmaGuide is an AI-powered screen assistant that watches your screen in real-time and provides step-by-step visual guidance for ANY software. Instead of reading tutorials, you see a glowing green circle pointing exactly where to click next.

Ask "How do I merge cells in Excel?" and SigmaGuide:

  1. Captures your screen
  2. Analyzes the UI with computer vision
  3. Shows you exactly where to click with a visual overlay
  4. Automatically detects when you complete each step
  5. Moves to the next instruction

Say goodbye to screenshot tutorials forever.

How we built it

  • Electron + React + TypeScript for the cross-platform desktop app
  • OpenAI GPT-4o Vision for understanding screenshots and generating instructions
  • Custom visual overlay system using transparent always-on-top windows
  • Real-time screen capture via Electron's desktopCapturer API
  • Zustand for lightweight state management
  • TailwindCSS for the sleek dark-mode UI

The magic is in the orchestration — our AI pipeline captures screens, analyzes UI elements, determines the next action, and renders a click target overlay in under 2 seconds.

Challenges we ran into

  • Transparent overlay windows — Getting a click-through overlay that floats above all apps (including fullscreen) required diving deep into Electron's window APIs
  • Coordinate mapping — Translating AI-detected coordinates to actual screen positions across different display scales
  • Real-time performance — Balancing screenshot frequency with API latency to feel responsive

Accomplishments that we're proud of

  • The "green circle" overlay that points exactly where to click — it just feels magical
  • Works with ANY software, not just specific apps
  • Auto-detects step completion — no manual "next" clicking

What we learned

  • Electron's window management is powerful but quirky
  • Vision AI models are incredibly capable at understanding UI
  • The best UX is invisible — users shouldn't think about the tool, just the task

What's next for SigmaGuide

  • Voice commands ("Hey Sigma, how do I...")
  • Multi-step task memory (remember complex workflows)
  • Community-shared guides for common tasks
  • Mobile companion for cross-device guidance

Built With

Share this project:

Updates