Schemes Reimagined

Application interface - search results and chat
Overall product flow diagram
Technical Architecture diagram
Application interface - home page

The inspiration behind Schemes arose from a common but profound concern: the difficulty of navigating social support systems. Our project was fueled by the desire to create a bridge between Singaporeans and the aid they need, especially during life's most trying times.

Imagine a scenario where life throws an unexpected curveball—maybe it’s a job loss, a sudden illness, or a family crisis. In those moments, the last thing anyone should face is confusion about where to find help. The Schemes provide quick and easy way for anyone to find relevant schemes in Singapore given their life situation and further chat with the application to find further details about schemes like services provided, contact details and further recommendations.

The technical journey to realize this vision has been nothing short of transformative. We began with an existing database of schemes, enriching it through meticulous manual curation and leveraging web scraping techniques to expand our dataset. This ensured that we have a robust and comprehensive list of schemes complete with relevant metadata.

We then harnessed the power of natural language processing, utilizing libraries like spacy and re for text preprocessing and lemmatization. Our choice of sentence-transformers, specifically the all-mpnet-base-v2 model, enabled us to generate meaningful embeddings that capture the essence of each scheme's intent. To efficiently retrieve relevant schemes, we utilized FAISS (Facebook AI Similarity Search) to add these embeddings to an index, allowing for lightning-fast searching capabilities. This indexing forms the backbone of our system, enabling users to find help with precision and speed.

After creating the embeddings an index, we leveraged Gemini 1.5 Pro to assess the quality of the search results using 500 random queries. For that we asked Gemini to assess relevance of the search results of our model versus previous implementations on random queries. This gave confidence that the new model is working better.

To bring the conversation to life, Google Gemini 1.5 Pro was pivotal. It helped us to dynamically generate conversation flows, tailoring interactions based on the user’s emotional cues and the context of their inquiries.

The whole application was containerised with Docker for Backend FastAPI returning search results and Frontend Streamlit providing UI. Frontend sends API request to backend to retrieve top schemes results, integrates with Gemini for the conversational chatbot experience and also saves all the user queries and chat history in BigQuery for future analysis.

The result is Schemes.sg reimagined—a solution where technology meets empathy. By using state-of-the-art AI tools and cloud services, we've created more than just a platform; we've crafted a supportive guide to help every Singaporean find the light during dark times. It's a testament to how innovative use of technology can have a tangible, positive impact on individual lives.