Companion Bot

Inspiration

Most AI tools live in a tab you have to consciously open. You context-switch, you type, you wait, you go back. The friction is small but constant. I wanted to create a bit of whimsy with a character I liked, ambient, always watching, always present. A companion that notices what you're doing and has something to say about it without you having to ask.

Kasane Teto felt like the right character for it. She's expressive, she has personality, and she has opinions. The idea of her sitting in the corner of your screen reacting to your life in real time felt both genuinely whimsical and fun.

What it does

Teto is a transparent desktop companion that lives on top of your screen. She watches what you're doing and reacts to it unprompted with spoken commentary, animated expressions, and genuine personality. You can also talk to her directly by typing or holding Ctrl+Space to use your mic.

She has a full emotion system with Gemini Ai generated sprites: happy, smug, annoyed, furious, curious, mortified, cozy, and more. She gets sleepy during long idle stretches and yawns if you leave her alone too long. Enable game mode and she pays closer attention while you play, reacting to wins, sympathizing with losses. Ask her to look something up and she searches the web and comes back with an answer. You can also put a hat on her.

How we built it

The shell is Electron with a frameless, transparent, always-on-top window. The renderer is React + Vite. Claude Sonnet handles direct chat and Claude Haiku handles fast screen reaction analysis, both get a detailed personality prompt that shapes how Teto speaks and what emotions she picks. Fish Audio S2 synthesizes her voice with inline delivery tags ([laughing], [sighing], [flat]) that shape her cadence. OpenAI Whisper handles voice transcription. Tavily powers web search, injected as context before Claude responds.

Screen awareness works by capturing frames via Electron's desktopCapturer, diffing them pixel-by-pixel, and only sending to Claude when enough has changed, avoiding both silence and spam.

Challenges I ran into

Animation sync was a notable problem. Hat accessories are PNG overlays on top of her sprite if they animate independently, they drift out of phase the moment you mount one, since the hat starts its animation cycle from zero while the sprite has been running for minutes. The fix was wrapping both in a single div and animating the wrapper, so both children move as one rigid unit.

Tuning the screen reaction loop took significant iteration. Too sensitive and she never stops talking. Too slow and she feels disconnected from what's happening. Getting the frame diff threshold, reaction cooldown, and API timeout to cooperate without making her annoying or laggy was a balancing act.

Windows Smart App Control blocking unsigned executables was a late-breaking packaging problem that had no clean solution in the hackathon timeframe.

Accomplishments that we're proud of

The emotion system feels genuinely alive. Watching Teto go from smug to mortified to concerned based on what's happening on screen without any manual trigger is the moment the project clicked. The voice delivery tags that shape her cadence ([laughing], [flat], [short pause]) make her feel like a character rather than a text-to-speech demo. The hat gallery was a last-minute addition and it's my favorite feature.

What we learned

Prompting for personality is a distinct skill from prompting for capability. Getting Teto to feel warm without being saccharine, honest without being mean, and reactive without being annoying required careful iteration on the system prompt, not just the model or the API calls. The character is mostly prompt engineering.

Electron transparent windows are more fragile than they look. Compositing, click-through regions, and always-on-top behavior all interact in ways that aren't well documented and vary across Windows versions.

What's next for Companion Bot

Custom voice cloning so users can give Teto any voice they want. A memory system so she actually remembers things about you across sessions. More accessories. Better packaging and code signing so the Windows install experience isn't a fight. And eventually, letting users build their own companion characters with different sprites and personalities on top of the same engine.

Built With

• anthropic-claude
• electron
• fish-audio
• javascript
• openai
• react
• vite