FairVio: Legal Assistance System for Migrant Workers

The RAG model returns answers based on legal documents in the database and utilizes Misteral AI.
Prompting the LLM model to generate responses.

1. Inspiration

Immigrant workers in North America often face complex barriers when trying to access legal support—particularly in areas such as labor law, wage theft, and workplace mistreatment. Many lack fluency in English, legal literacy, or reliable internet access, and they often fear retaliation if they report violations. These challenges result in silence and inaction, even in situations where help is most urgently needed.

Our team was inspired by real stories from immigrant communities, where individuals endured mistreatment simply because they didn’t know their rights or couldn’t afford legal assistance. We set out to build a multi-modal legal assistant that marginalized workers can easily access through phone calls, chat, or by submitting a simple report. Depending on the user's environment, the assistant may not require any data access at all.

2. What does our solution do?

Our solution is a multilingual, multimodal legal support platform designed to empower immigrant workers and vulnerable populations who are often left behind by traditional legal systems. At its core, the platform enables users to access reliable legal information, report labor law violations, and connect with legal aid — all without requiring English fluency, technical literacy, or expensive legal consultation.

What makes our approach different is that it’s not just a chatbot or a call center — it's an integrated ecosystem offering three seamless access channels:

Phone-based voice assistant
Web-based multilingual chatbot
Direct legal report form

All three are powered by a shared AI backend that ensures consistent, document-grounded responses, and offers an end-to-end support pipeline that goes beyond simple Q&A.

📞 Voice Assistant (Call-in Access) Through a Twilio-powered phone line, users can call into our system and navigate a multilingual voice menu in English, Spanish, French, Chinese, or Korean. Speech recognition transcribes user input in real time, and responses are generated via a Retrieval-Augmented Generation (RAG) pipeline. The AI doesn’t simply guess — it searches a database of real legal documents using FAISS vector search, retrieves the most relevant sections, and then uses Mistral-7B (via Hugging Face) to generate grounded, legally sound responses.

The answer is read aloud using text-to-speech, and an optional SMS summary is sent afterward. If a potential labor violation is detected, the system begins building a case report in the background.

💬 Multilingual Chatbot (Web Interface) For users who prefer a typed experience, we offer a chatbot interface accessible from mobile or desktop. Built with React and TypeScript, the chatbot supports the same languages and backend intelligence as the phone assistant. Users can ask questions, upload relevant documents or images, and receive AI-generated answers in a clear, user-friendly chat flow. This mode offers a faster and more interactive experience, particularly for tech-comfortable users.

📝 Direct Report Submission (Web Form) Sometimes, users already know something is wrong. Our Direct Report form allows them to bypass the Q&A flow and file a structured report directly. Through a guided form, users input who was involved, what happened, where and when it took place, and what type of violation occurred. The system then generates a PDF case summary and stores it securely.

🧠 AI Pipeline (Shared Intelligence Across All Modes) All three access channels converge on a shared RAG pipeline:

User input (speech, text, or form) is converted into an embedding.

FAISS vector search locates the most relevant sections of legal documentation.

Mistral-7B generates a response based only on this retrieved context.

The system may summarize, escalate, or route the case depending on the severity and clarity of the input.

This design ensures all users — whether they call in, chat online, or submit a report — receive equally accurate, document-grounded guidance.

⚖️ From Information to Legal Action: Case Routing and Escalation Information alone isn't enough. Once the system detects a likely labor violation, it initiates an automated legal support flow:

Lawyer Matching The assistant uses metadata from the case — including violation type, industry, and location — to match the user with vetted legal professionals. These matches are generated using similarity scoring based on previous case data and specialization profiles.

Auto-Generated Briefing Packet To reduce the burden on legal staff, the system creates a clean, standardized PDF case summary. This includes all key metadata, the user’s input, timestamps, and AI-detected red flags — formatted using the 5W1H principle (Who, What, When, Where, Why, How).

Case Escalation If the AI confirms the case is likely valid, and the user gives consent, the case packet is forwarded directly to a matched lawyer or legal aid clinic. The legal team can then review and, if appropriate, take further action such as filing an official complaint, contacting the employer, or reporting the violation to government authorities.

This system dramatically shortens the gap between incident and legal action — especially for people who might otherwise remain silent.

🌍 Broader Impact By combining voice, chat, and form-based reporting in one intelligent legal pipeline, our solution provides accessible, free, and high-quality legal guidance to people who are often digitally or linguistically excluded. It bridges the gap between knowing your rights and enforcing them — not only informing users, but empowering them to act.

Whether it’s through a simple phone call, a typed message, or a detailed report, our platform gives workers a voice — and ensures that voice is heard by the people who can help.

3. How we built it

To develop our solution, we took a modular and parallel approach that allowed different team members to work on independent components while maintaining tight integration across the entire system. Our architecture was built around the core principle of accessibility, ensuring that each interaction mode—voice, chat, or direct report—would plug into a shared intelligent backend. This enabled us to focus on both usability and technical reliability.

For the voice-based experience, we integrated Twilio to handle all incoming and outgoing phone calls. We designed a multilingual voice menu system that allowed users to select their preferred language and mode of interaction. Real-time speech-to-text transcription was enabled through Twilio’s built-in capabilities, which we configured dynamically based on language selection. For users uncomfortable with speaking, we added DTMF (keypad) support, allowing them to interact with the assistant by pressing numbers on their phone. Flask was used as our backend framework to manage Twilio webhooks and call routing through custom endpoints, including logic for language selection, input processing, fallback handling, and escalation flows.

The core of our legal intelligence is powered by a Retrieval-Augmented Generation (RAG) pipeline. First, we curated a set of legal documents, including labor law guides and immigration resources. These documents were parsed, cleaned, and divided into manageable chunks. We used the SentenceTransformers library to generate vector embeddings for each chunk, and stored them using FAISS, Facebook’s open-source vector similarity search engine. When a user submits a query—whether through speech or text—it is embedded in the same space, and the system retrieves the most relevant document segments. These are passed to Mistral-7B, a powerful open-source language model hosted via the Hugging Face Inference API, which generates legally grounded responses based only on the retrieved context. This method significantly reduces hallucinations and ensures accuracy.

To ensure that users receive meaningful outputs beyond spoken or written responses, we implemented a reporting and summarization system. After each interaction, the assistant condenses the conversation into a short, three-line summary suitable for SMS delivery. In cases where the AI detects a potential labor violation, it automatically compiles a structured PDF report. These reports include key metadata such as employer name, location, date, and type of violation, and are formatted clearly for use by legal aid organizations. We used ReportLab for dynamic PDF generation, ensuring proper layout, wrapping, and multilingual text support.

Throughout the development process, we relied on a modern and scalable tech stack. Our frontend is built using React, TypeScript, and TailwindCSS. The backend is powered by Python and Flask, exposed via RESTful APIs. Twilio handles all voice and SMS functionalities, while Supabase was used for rapid testing of serverless functions. LangChain serves as our orchestration layer, managing the logic that links together document retrieval, query generation, and output formatting. We used Vite for front-end tooling and ngrok to expose local endpoints during live testing with Twilio.

We placed strong emphasis on testing and validation to ensure the system was both robust and reliable. For the voice component, we tested speech-to-text recognition accuracy across five languages, using diverse speaker profiles to evaluate performance under different conditions. For our RAG pipeline, we benchmarked document retrieval performance using a set of standardized legal questions and validated that the top-k results aligned with human expectations.

We also conducted stress tests of our call flow logic, simulating edge cases such as timeouts, invalid input, and language switching mid-call. Finally, we manually reviewed LLM-generated responses to monitor for hallucinations, misinformation, and tone quality.

4. Challenges we faced

Building a real-world, user-facing legal assistant came with a range of technical and logistical challenges—especially given that we were not working in a lab or sandbox environment, but developing something intended for real users in production-like conditions. Many of the hurdles we faced came from the constraints of cost, infrastructure, experience, and the sensitive nature of our use case.

One of the first major challenges was finding a suitable large language model (LLM) that we could use within our resource limitations. Since we were not operating with access to enterprise-level cloud compute or paid APIs, we needed a free or open-access LLM that could still deliver reliable performance. Initially, we considered using OpenAI’s ChatGPT API or LLaMA models. However, both presented issues: ChatGPT required payment and was difficult to scale without cost, and LLaMA models exceeded the infrastructure limits we were working with. Eventually, we chose to use Mistral-7B via Hugging Face's inference API. It struck the right balance between performance, accessibility, and speed—and despite our constraints, it delivered strong, legally grounded answers when paired with a retrieval pipeline.

Another challenge was designing the backend system that supports retrieval-augmented generation. At first, we considered using traditional relational databases like MySQL, which we were more familiar with. But we soon realized that querying against thousands of legal text chunks in real time—especially after embedding PDF documents—would be too slow and inefficient with SQL. After researching alternatives, we discovered FAISS (Facebook AI Similarity Search), an open-source vector database optimized for similarity search at scale. By switching to FAISS and embedding our documents using SentenceTransformers, we were able to create a highly efficient retrieval system that responded well even on limited infrastructure.

A third key obstacle was our limited prior experience in both backend and frontend development. None of our team members came from a web development background, so we spent considerable time deciding what languages and frameworks to use. Eventually, we chose TypeScript and React for the frontend due to their developer-friendly ecosystem, strong documentation, and good compatibility with modern tools like Vite and TailwindCSS. We often relied on help from LLMs to debug and iterate our code, and this iterative learning approach allowed us to build a working prototype from scratch in just a few days.

That said, we still faced some technical limitations we were unable to fully resolve during the hackathon. For example, we lacked access to GPUs or accelerated compute, which made real-time chatbot interaction noticeably slow at times. While we were able to build out the backend logic for the chatbot and implement the frontend interface, we ran out of time when it came to fully integrating the backend algorithm with the live frontend component. As a result, the chat interface currently functions as a demo with simulated responses, rather than as a live API-connected experience.

We also encountered legal domain challenges. Since none of our team members were Canadian legal experts, we were initially unsure about which documents to include in our knowledge base. Due to file size and processing constraints, we ended up embedding only the Canadian Constitution. We believe this can be significantly improved in the future through collaboration with legal professionals and organizations who can help curate relevant labor law documents, and through access to better infrastructure that can support larger corpora.

In short, our team overcame many foundational challenges—finding a workable LLM, building a scalable RAG system, learning new technologies on the fly—but there were still technical gaps we couldn't fully bridge within the limited timeframe. Nevertheless, we believe these are solvable problems, and with access to legal advisors, GPU compute, and institutional infrastructure, the system could be expanded and productionized effectively.

5. Accomplishments/Learnings

One of the most valuable accomplishments from this project was the opportunity to explore technologies and methodologies that were entirely new to us—most notably, the Retrieval-Augmented Generation (RAG) model. Before this hackathon, none of our team members had worked with RAG or even knew much about its practical applications. Initially, we explored other ways to improve the accuracy of LLM responses, such as fine-tuning or developing custom models, but quickly realized these approaches would be too time-consuming and compute-intensive within our limited timeframe.

In the process of searching for a scalable yet efficient solution, we came across the 2020 paper “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” by Patrick Lewis et al. (https://arxiv.org/abs/2005.11401). This led us to implement RAG as our core architecture, which allowed us to deliver precise, context-grounded legal answers using external documents rather than relying solely on the language model’s internal memory. Learning and applying this architecture from scratch within the hackathon window was both challenging and incredibly rewarding.

Beyond that, we also significantly expanded our technical experience in areas we had never worked in before. While some of us had prior experience in natural language processing and text-based data analysis through coursework or internships, we had never worked with speech-based inputs. In building this system, we learned how to handle both speech-to-text and text-to-speech functionality, and integrate them smoothly into a multilingual, phone-based assistant. This was crucial because our primary user base includes migrant farm workers—many of whom may not have smartphones, reliable internet access, or comfort using apps. Prioritizing voice accessibility felt not only practical but necessary, and it gave us a strong motivation to dive deep into Twilio’s programmable voice API and connect it to our RAG pipeline.

Working with Twilio and integrating its capabilities into a language model-based backend was a significant technical leap for us. It was our first time designing a real-time call flow, implementing voice recognition, and returning dynamic AI-generated content over the phone. Connecting all of this to a retrieval pipeline and ensuring the end-to-end experience worked smoothly required both technical problem-solving and a strong understanding of user needs.

Another big learning point was frontend development. Most of us did not come from software engineering backgrounds—we’re students primarily focused on AI, machine learning, and statistics. So when it came to building the web application, chatbot interface, and forms, we had to quickly learn and adapt. We chose TypeScript and React for the frontend, which were completely new tools for many of us. With limited time and resources, we pulled together as a team, divided up responsibilities, and taught ourselves how to implement core UI components. By the end of the project, we had a working voice system, a structured reporting interface, and a partially integrated chatbot—all developed in just a few days.

Overall, we’re proud of what we were able to build—not only because the technical outcome was strong, but because of how much we learned along the way. This project gave us the rare chance to connect the dots between data science, machine learning, backend engineering, frontend development, and real-world social impact. We now feel much more confident building full-stack, AI-powered applications from scratch, and more motivated than ever to continue using tech for good.

6. What’s next?

While we’re proud of what we accomplished in a short timeframe, there are many improvements and extensions we would implement if given more time or with a stronger baseline of technical resources. Our current design is optimized primarily for vulnerable workers—particularly those with limited access to digital tools, legal knowledge, or English proficiency. However, as we look to expand the platform’s impact, we envision several short-term upgrades and long-term scalability strategies.

If we had more time and technical support, the first immediate improvement would be the completion and integration of our chatbot’s backend with the frontend interface. While the underlying logic, retrieval pipeline, and language model response system are fully operational, we were unable to fully connect them to our live chatbot UI during the hackathon. Fixing this connection would allow users to receive live, real-time answers via text, just as they currently can through the phone assistant. Additionally, due to limited compute resources, our LLM inference runs on a CPU backend, which introduces latency. With access to a GPU-enabled environment, we could significantly speed up the response time and improve the overall user experience.

Another important upgrade would be to implement role-based interfaces. Right now, the app is entirely user-focused—designed for workers to report violations and receive legal information. But in a more robust version, we would create a separate portal for lawyers and legal aid organizations. These users would go through a secure sign-up process, and access a dashboard specifically tailored to their workflow. This would include features like case intake management, real-time alerts for incoming violation reports, document viewing tools, and secure messaging with clients. Similarly, employers who partner with us (for example, through ethical labor initiatives) could have access to an educational portal or compliance checklist, helping promote transparency and accountability.

On a technical level, we would also add multi-device syncing and session persistence so users could start a report via voice call and follow up via web chat or text message. This would make the experience more seamless and user-friendly, especially for ongoing legal cases that require follow-up.

From a scalability perspective, our next big goal would be to take this platform global. To do that responsibly, we would need to expand our legal knowledge base beyond the Canadian Constitution and ensure our AI models are grounded in accurate, country-specific labor law documents. This would require partnerships with legal professionals in different countries, as well as access to multilingual, structured legal datasets. We would also seek validation from bar-certified lawyers and public interest legal clinics to review and approve our knowledge base, ensuring it remains both accurate and up-to-date.

Moreover, to support international users, we would need to enhance our language model’s localization capabilities. While our current assistant supports five languages, scaling globally would require adding many more languages and dialects—and ensuring cultural relevance, not just linguistic translation. This would also involve fine-tuning response tone, legal terminology, and case prioritization models for different jurisdictions.

To support all of this at scale, we would also need to strengthen our infrastructure. Our current system runs on lightweight cloud services suitable for prototyping, but a production-grade deployment would involve secure authentication layers, robust database management, user session encryption, and compliance with privacy laws such as GDPR or PIPEDA.

In terms of adoption, we envision a mixed sustainability model. While the platform would remain free for end users (especially vulnerable workers), we would monetize through B2B partnerships—offering subscription-based access or integration to law firms, labor unions, NGOs, and socially responsible employers. These institutional users would benefit from automation, smart intake pipelines, and ethically driven branding opportunities.

In summary, our next steps include technical refinements like backend optimization, chatbot integration, and GPU acceleration; structural improvements like multi-role portals and global legal expansion; and strategic moves toward long-term validation, legal collaboration, and financial sustainability. We believe these changes are entirely feasible with a few more weeks of focused development and the right partnerships in place. With continued iteration, our platform can evolve into a trusted, scalable legal support ecosystem serving both workers and legal professionals worldwide.