Sphinx Scholar

Note: to run demo, please use mock account Username: VertexAI Password: GoogleHackathon

Inspiration

Getting started with Vertex AI Agent Builder, we were impressed by the built-in RAG capability. It seemed so convenient to drag and drop files to ground LLM responses.
However, as we thought of use cases, it became apparent that the input data is set by the developer and is static.
We confirmed with a Google Cloud Engineer at a community event: that to do RAG on dynamic data that is input by the user — workarounds are needed
Realise this workaround solution through the means of a widely useful application — that is an all purpose quiz bot on user's own data

The best way to remember any information is by being quizzed on it. Making and evaluating quizzes is effortful. An LLM bot can act like a helpful friend that significantly reduces the effort.
Traditional methods of creating quizzes can be time-consuming, especially true for students having to juggle multiple subjects or modules at a time, and often lack the adaptability needed for effective learning.
As for why we chose this specific idea of doing a quiz AI agent, the inspiration came from the need to streamline and personalize the quiz preparation process for learners from all walks of life. Be it primary school learners needing help with additional practices before their science exams or students pursuing their masters in a very complex biomedical field, we wanted to cater to the whole spectrum in a very neat and elegant way.
Thus, by leveraging Google's native AI, we aimed to create an intelligent Quiz Bot that can dynamically generate, evaluate, and score quizzes based on user-provided documents or flashcards (in document format). This would be especially helpful given how we did not need to be domain experts in any one subject to be able to create the questions and answer pairs for the quiz.

A. Simple steps to ground agent response:

B. What needs to happen to ground agent to dynamic user data?

C. What happens by default?

After a data store is created from a bucket, adding new files to bucket does not update the data store
However, data store can be updated with ‘IMPORT’ option
Once data store is updated, re-index automatically, and reflected on app

Thus, main features of solution

Simple web page to allow user to upload their own documents which are saved on bucket
IMPORT with API call

Uploading to data store: this is achieved through using two functions exposed by the Google Cloud API.
- The first function uploads a file to a designated bucket.
- The second function imports this file to the datastore linked to our agent.
- This allows us to create a dynamic datastore that users can add documents to as they get more study material, circumventing the constraint of only allowing the agent manager to upload files for retrieval-augmented-generation.
Interacting with Vertex AI Agent: The code generated by the Dialogflow Messenger integration is used to create a component that enables users to talk to our agent after uploading their documents.

File Handling and Storage: Managing different file formats and ensuring they were loaded, stored, processed and read properly was a significant challenge. To solve this we implemented robust file handling mechanisms and utilized cloud storage solutions for efficient data management.
Scalability: We were facing issues with the bot handling multiple users at once and multiple documents at once. So we used the google cloud services and some optimized codes to handle the variable loads.
Accuracy and Relevance: Maintaining the accuracy and relevance of generated questions was critical. We learned to continuously refine our prompts and examples and integrate error-checking mechanisms in the instructions to improve the quality of questions and answers.
Multiple-agent workflow. We experimented with this, although a single agent was able to serve purpose with detailed instructions.