-
-
Welcome to the Tax Pilot
-
View your Tax Profile
-
Upload your Tax Documents
-
Chat directly with your Tax files
-
Call when in doubt!
-
Get detailed analytics of your Tax Returns
💡 Inspiration
Tax preparation is overwhelming, and trying to decode past tax documents feels like reading a foreign language for most people. We wanted to build a tool that replaces static, confusing forms with a highly interactive, multimodal AI assistant with conversational features making tax organization and optimization as easy as chatting with a knowledgeable friend.
⚙️ What it does
Tax Pilot is an AI powered educational assistant that ingests prior year tax documents (like W-2s and 1099s), automatically extracts key financial data, and flags potentially missed deductions. Users can ask questions about their documents via a web-based chat or jump on a live phone call with our AI voice agent. The agent interviews them about recent life changes and income updates to generate a structured, personalized filing-readiness report. It suggests any anomalies in the filings and lets the user know if they can save taxes anywhere with a personalized tax strategy.
🛠️ How we built it
Powered by context-aware AI, our system transforms how users interact with their tax data via phone and chat. Users upload previous tax returns, which are vectorized using Actian into accessible data chunks. This allows our ElevenLabs voice agent and Gemini chat model to analyze the forms and communicate directly with the user. The system uses fine-tuned models to generate tailored insights, dynamic visualizations, and recommendations, while an automated anomaly detection feature cross-references fields to flag inconsistencies and highlight irregularities.
🚧 Challenges we ran into
One of our biggest challenges was connecting the vectorized dataset to our ElevenLabs and Gemini agents to achieve true context awareness. Because tax documents are dense and interrelated, engineering the system to accurately parse and remember information across multiple forms at once took considerable time and troubleshooting.
🏆 Accomplishments that we're proud of
We are incredibly proud of engineering a live, context-aware AI phone agent. By integrating Twilio with a complex backend pipeline, we successfully enabled real-time data retrieval and dynamic context passing during active calls.
🧠 What we learned
Building Tax Pilot taught us how to manage complex LLM context windows and orchestrate real-time, low-latency audio streams for AI voice agents. We also gained invaluable experience architecting a robust RAG pipeline by seamlessly connecting Actian vector embeddings with a Snowflake data warehouse.
🚀 What's next for Tax Pilot
We plan to expand our parsing engine to handle complex documents like Schedule Cs and integrate APIs like Plaid to automate deductible expense tracking. Additionally, we aim to build a secure "CPA Handoff Portal" so users can instantly export their AI-generated readiness reports directly to certified tax professionals.
Log in or sign up for Devpost to join the conversation.