Me-Agent

Me-Agent is a secure, identity-bound AI that shops for you while respecting human-defined limits. It makes AI assistance powerful, transparent, and safe by design.

Comment

Inspiration

As AI agents become more capable, the biggest open question is not just intelligence, it’s trust. Who is the agent acting for? What is it allowed to do? And how can a user verify that nothing happened without their consent?

Consumer choices are one of the most visible ways people express identity. What you buy reflects who you are, who you want to be, and how you want to be perceived. We saw shopping assistants not just as recommendation engines, but as identity-shaping tools. Our AI acts as a personalized extension of you; it is a delegated version of your preferences, values, and boundaries.

However, most shopping assistants today either make passive suggestions or act autonomously with little transparency. As for Me-Agent, we explored a novel idea: identity wielded as consent. The AI can act like you, but only within explicitly defined boundaries that you can always audit.

UofTHacks’ theme, Identity, pushed us to see identity not as a static profile or login, but as something that can evolve with AI while also being protected from it. Our system allows identity to change as users do, while protecting it through clear permissions, auditability, and user-owned decision boundaries. This is how we let AI assist without replacing autonomy.

What it does

Me-Agent is a consent-bound AI shopping agent that helps users assemble shopping bundles while staying fully transparent, controllable, and secure.

  • Users authenticate with passkeys (WebAuthn), no passwords.
  • The agent generates shopping bundles based on intent, preferences, and history.
  • A server-side authority layer enforces user-defined rules (budget, categories, confirmation).
  • Every action is explained in natural voice (ElevenLabs) and logged with policy snapshots.
  • If the agent exceeds limits, it is blocked, and the reason is shown immediately.
  • The result is an AI agent that feels helpful without ever taking control away from the user.

How we built it

We designed Me-Agent around a three-layer architecture:

  1. Memory Layer - Stores user preferences, feedback, and shopping history using MongoDB Atlas.
  2. Reasoning Layer - Uses Backboard.io to orchestrate LLMs that:
    • Select items for a bundle
    • Generate human-readable explanations
    • Model routing allows us to separate decision-making from explanation.
  3. Authority Layer - A FastAPI backend that:
    • Enforces policies server-side
    • Requires passkey authentication for sensitive actions
    • Logs every decision with a policy snapshot

The frontend is built with React + TypeScript (Vite) and presents:

  • An intent form
  • A live authority panel
  • A full audit log
  • A voice-driven explanation using ElevenLabs
  • Optional "bitmoji" avatar generation using Google Gemini
  • The app is deployed at www.me-agent.tech

Challenges we ran into

Balancing security and UX: Too many security prompts feel restrictive; too few feel unsafe. We iterated heavily on when passkeys are required. Making AI failure visible: We wanted the agent to fail gracefully. Designing meaningful “blocked” states was harder than success paths. Keeping claims honest: It was tempting to over-promise on autonomy or immutability, but we focused on what we could truly demonstrate. Coordinating multiple AI services: Managing Backboard, ElevenLabs, Gemini, and WebAuthn in a short time required careful separation of concerns.

Accomplishments that we're proud of

  • A fully working passkey-first authentication flow
  • A server-side policy engine that cannot be bypassed by client code
  • A human-sounding, accessible voice explanation for every AI decision
  • A transparent audit trail that users can inspect at any time
  • Demonstrating AI being blocked by design, not hidden

What we learned

  • Identity is more than authentication; it’s about what actions are allowed
  • Users trust AI more when limitations are explicit
  • Explaining why an AI acted matters as much as what it did
  • Security features can improve UX when they’re visible and understandable
  • AI systems need authority boundaries, not just better prompts

What's next for Me-Agent

  • Merchant-side integrations for Shopify storefronts
  • Cryptographically tamper-evident audit logs
  • Fine-grained, per-action consent scopes
  • Multi-agent identity delegation (e.g., family or team policies)
  • Broader applications beyond shopping: scheduling, subscriptions, and finance

Built With

  • backboard
  • bun
  • cbor
  • cors
  • elevenlabs
  • fastapi
  • gemini
  • mongodb
  • openapi
  • pydantic
  • python
  • radix-ui
  • react
  • react-router
  • render
  • shopify-storefront-api
  • swagger
  • tailwind
  • typescript
  • uvicorn
  • vercel
  • vite
  • vitest
Share this project:

Updates