AgentPay — Your AI Bill-Pay Butler

Inspiration

Managing bills is frustrating. Utility invoices, credit card statements, surprise medical charges, these are all charges arriving in different formats and at different times. Even with auto-pay, you must trust each provider and manually handle exceptions.

We asked ourselves: What universal channel do all billers already use? The answer: email.

That’s how AgentPay was born. An AI agent with its own inbox that reads incoming bills and either pays them or negotiates on your behalf. No new integrations for providers, no tedious reminders for users. Just forward the bill and let AgentPay handle it.

What We Learned

• Integrating AgentMail: creating programmatic inboxes, receiving webhook events, fetching raw email content, and sending replies.
• Using Method Financial: simulating payments to thousands of institutions and showing how an embedded API can power a consumer-facing agent.
• Parsing unstructured emails reliably using Gemini LLM, to facilitate JSON extraction for fields like
  [ \text{amount_cents} = \lfloor \text{Amount Due} \times 100 \rfloor ] and
  [ \text{due_date_iso} = \text{YYYY-MM-DD}. ]
• Designing a decision engine to autopay smaller bills while requesting a payment plan for larger ones.
• Building a FastAPI backend with idempotent webhook handling, a live dashboard, and a self-seeding test mechanism.

How We Built It

1. FastAPI Backend

   • Receives message.received webhooks from AgentMail at /webhook/agentmail.
     
   • Fetches raw message text/HTML and extracts {payee, amount_cents, due_date_iso} with LLM + regex fallback.
     
   • Stores bills + actions in SQLite for auditing and demo stability.

2. Decision Engine

   • If (\text{amount_cents} \le 15000): schedule a simulated Method payment + reply “Payment scheduled.”
     
   • Else: reply requesting an extension/plan and notify the user by email.

3. AgentMail Outbound

   • Replies in-thread to vendors (showcasing autonomous agents).
     
   • Sends summary updates to the user’s real email.

4. Dashboard

   • A lightweight front-end AgentPay dashboard, made with React, TypeScript, showing “Bills,” “Payments,” and “Events” in real time.
     

Challenges We Faced

• Email variability: Bill templates vary widely; we had to combine regex heuristics with deterministic LLM parsing to get consistent structured data.
• Webhook retries: Duplicate processing risk; solved with unique message_id constraints.
• Security: We kept API keys server-side and stored only structured fields, not entire emails.
• Financial Data: Working with potential financial data requires a deep understanding of security, and authentication making sure potential future applications adhere to standards.
• Time pressure: With a time constraint, we focused on text-based emails first and left PDFs/OCR for a future iteration.

Takeaways

AgentPay shows how combining AgentMail (communication) and Method (payments) can transform the inbox into a hands-free financial workflow hub. Unlike reminders or budgeting apps, our agent actually acts on your behalf.

With U.S. households averaging 13 bills per month, AgentPay demonstrates a path to reducing cognitive load, preventing late fees, and making bill management almost invisible.

Built With

• agentmail
• fast-api
• method
• python
• react
• sqlite
• supabase
• typescript