NIXY | Devpost

Inspiration Every day, someone manually copies purchase orders from Walmart or Soriana's portal into Arca Continental's internal system. It doesn't scale, it causes errors, and EDI integration takes months per client. We wanted to eliminate that manual work — starting today. What it does Nixy is an AI mascot that lives in your browser. It watches you perform a process between two web systems just once, learns the field mapping dynamically using Gemini Vision, and then replicates it autonomously with Playwright — no hardcoded rules, no manual work. How we built it

Chrome Extension (Manifest V3) captures clicks, navigation, forms, and inputs without interfering with the page Gemini API analyzes the HTML of the source system and infers the field mapping dynamically FastAPI on Vultr orchestrates the pipeline and triggers the Playwright bot MongoDB stores the learned mappings and execution history ElevenLabs gives Nixy her voice — alerts and confirmations spoken out loud Playwright replays the learned process autonomously on the destination system

Challenges we ran into

Getting Nixy to persist across page navigations without re-injecting Ensuring zero hardcoding in the field mapping — everything inferred by AI

Accomplishments that we're proud of

A fully working end-to-end demo: observe once in automationexercise, execute autonomously in the destination system Zero hardcoded rules — Gemini infers everything from raw HTML A mascot that changes states visually

What we learned How to build a Chrome extension that observes behavior non-intrusively, how to prompt Gemini Vision for structured field mapping, and how to ship a full AI automation pipeline in under 24 hours.

What's next for Nixy Support for more complex workflows with conditional logic Multi-language support for international retailers A dashboard showing all learned processes and their execution history Native mobile app for warehouse floor operations