Cl1ppy

Clippy crawled out of the recycle bin, stole an AI brain, and now lives in your browser whispering truth, chaos, and motivation. Cl1ppy, your computer just grew feelings.

Comment

Inspiration

We were inspired by the nostalgic charm of Clippy, Microsoft's iconic office assistant, and wanted to revive him for the modern web with AI superpowers. The idea was to combine the beloved (and sometimes annoying) paperclip character with cutting-edge AI technology to create a self-aware digital companion that provides helpful and hilariously sarcastic commentary about your browsing habits.

What it does

Cl1ppy is an AI-powered Chrome extension that brings back everyone's favorite office assistant with a modern twist. It:

  • Analyzes your browsing: using Google Gemini's vision AI to understand what you're doing
  • Provides witty commentary: with four distinct personality modes (Funny, Encouraging, Productivity-focused, Demotivating)
  • Speaks to you: using ElevenLabs voice synthesis with different voice styles for each mood
  • Manages your todos and contextually reminds you about them based on what you're browsing
  • Periodically checks in: with configurable frequency (1-30 minutes)
  • Appears on every webpage: as a draggable 3D character that can be moved around
  • Stays self-aware: with personality-driven prompts that make Clippy feel genuinely sarcastic or supportive

How we built it

We built Cl1ppy as a Chrome Extension using Manifest V3, which required:

  • Service Worker Architecture - Implemented background processing for API calls, periodic analysis with Chrome Alarms API, and screenshot capture
  • AI Integration - Connected Google Gemini AI for screenshot analysis with custom personality-driven prompts, and ElevenLabs for multi-voice TTS
  • Content Script Injection - Created a draggable Clippy UI using Shadow DOM for isolation, with speech bubbles and 3D model integration via Sketchfab iframes
  • Personality System - Built a modular personality framework with four distinct moods, each with unique prompting strategies and voice mappings
  • Secure Key Management - Implemented encrypted storage for API keys with fallback to build-time environment variables
  • Offscreen Document - Used Chrome's Offscreen API to enable audio playback in Manifest V3 (required workaround)
  • Build System - Created Node.js build scripts to inject environment variables, copy assets, and package the extension

Challenges we ran into

  • Manifest V3 Audio Playback - MV3 service workers can't directly play audio, requiring us to implement an offscreen document as a workaround
  • Ad Blocker Conflicts - The Sketchfab iframe was flagged by ad blockers as third-party content, requiring careful z-index and injection strategies
  • API Rate Limits - Balancing analysis frequency with free tier rate limits for both Gemini and ElevenLabs
  • Context Persistence - Service workers can shut down, so we had to carefully manage state persistence in Chrome Storage
  • Screenshot Permissions - Some pages (chrome://, extension pages) can't be captured, requiring graceful error handling
  • Personality Prompting - Tuning prompts to get consistent, concise responses (1-2 sentences) that matched each personality's tone

Accomplishments that we're proud of

  • Successfully integrated two major AI APIs (Gemini and ElevenLabs) into a cohesive Chrome extension
  • Created a genuinely funny personality system with distinct, recognizable voices and commentary styles
  • Solved the Manifest V3 audio challenge with an elegant offscreen document solution
  • Built a beautiful, draggable UI with smooth animations
  • Made Cl1ppy self-aware - the AI-generated commentary genuinely feels like Clippy has opinions about your browsing
  • Created a complete todo integration that contextually reminds users based on what they're browsing
  • Achieved ad blocker resilience with respectful injection strategies

What we learned

  • Manifest V3 architecture and the shift from background pages to service workers
  • Vision AI capabilities - Gemini's screenshot analysis is remarkably good at understanding web content
  • Voice synthesis integration - How to map different voice styles to personality types for consistency
  • Prompt engineering - Crafting system prompts that generate concise, personality-consistent responses
  • Chrome Extension security model and permission management
  • Balancing humor with helpfulness - Making Cl1ppy funny without being genuinely annoying

What's next for Cl1ppy

  • Integrate with calendars and reminders to proactively assist users
  • Cl1ppy remembers habits and past interactions to personalize advice
  • Gamify productivity with “Cl1ppy approval points” for tasks and creativity
  • Expand ElevenLabs voices with emotional range, accents, and character personalities.

Built With

  • archiver
  • chrome-alarms-api
  • chrome-extension-manifest-v3
  • chrome-offscreen-api
  • chrome-runtime-api
  • chrome-storage-api
  • chrome-tabs-api
  • elevenlabs-api
  • fs-extra
  • google-gemini-ai-api-(gemini-2.0-flash)
  • html5/css3
  • javascript-(es6+)
  • model-viewer-library
  • node.js
  • service-workers
  • shadow-dom
  • sharp
  • sketchfab-api
Share this project:

Updates