auraforming.ai

Inspiration

Filling out complex forms is stressful, slow, and often confusing, especially when there's governmental jargon involved. If you don't speak the language well or you're an immigrant, it can be a massive headache just to get basic paperwork done. We wanted to turn that stressful chore into a calm, guided voice experience in the user's native language that actually feels human. We also wanted to help out small businesses so they don't have to waste their employees' time doing manual intake over the phone or in person.

What it does

auraforming.ai lets a business upload any blank fillable PDF. We auto-create a custom AI “agent” for that specific form and generate a shareable link. Clients open the link and fill out the form by talking to the AI. They can ask for clarification on weird terms whenever they need it, and at the end, they get a cleanly filled-out PDF. On the flip side, businesses can review and manage all these intake results right from their admin portal.

How we built it

Frontend: React + Vite
Backend: Python Flask + modular API routes
Database: SQLite to handle agent, session, and intake states
PDF processing: Custom parsing to extract fields and map the data so we can write it back into the PDF
AI reasoning: Gemini (using strict JSON-structured prompting to evaluate the conversation turn-by-turn)
Voice: ElevenLabs STT (Speech-to-Text) and TTS (Text-to-Speech) for a natural conversational flow

Challenges we ran into

PDF field naming is a mess (human-readable labels almost never match the actual hidden widget names).
Figuring out how to map checkbox and dropdown values reliably across completely different forms was tricky.
Orchestrating real-time voice sessions gracefully took a lot of work.
Wrestling with API keys and config issues while tying the AI and voice services together in real-time.

Accomplishments that we're proud of

Getting a full end-to-end flow working in hackathon time: upload a PDF → get an agent link → do a voice interview → download the filled PDF.
Building dynamic schema handling that adapts to whatever PDF form the agent is processing.
Putting together a really clean UX for both clients and businesses to intake and review data.
Creating a push-to-talk voice interface that actually feels natural and patient to use.

What we learned

Prompt design and strict JSON constraints are absolutely make-or-break for getting the AI to behave reliably.
Automating forms is mostly about dealing with edge cases and normalizing weird data.
A great UX is just as important (if not more) than the backend logic if you want users to actually trust the AI.
You absolutely need good logging and observability to debug real-time multi-modal AI pipelines.

What's next for auraforming.ai

Building smarter and more versatile validation for different types of form fields.
Better multilingual support and giving the agent more domain-specific knowledge to explain complex terms.
Adding auth, org-level permissions, and hardening the app for production.
Giving users the ability to easily retry, jump around the form, or fix things on the fly if the agent mishears a spelling.