Inspiration
-Tableau / QlikView.
What it does
-Our app allows users to easily visualize any data from the CSV file or the whole CSV dataset from multiple files.
- Inspired by chatGPT we build "chat-assistant" with multiple different chats preserved in database.
- User can manage his conversations, delete them, create new, export them or import from others.
- User can prompt any question regarding his datasets (the model doesn't respond with answers not present in his datasets)
- After user asks a question our complex algorithm produces SQL query to fetch the relevant data from database, analyses them, based on the question produces and explains his answer, displays interactive charts analyzing fetched data and also visualizes insightful operations on the fetched data - from example growth rate or contribution to global sum.
- User can then ask following up question which will result in updating existing results or ask about a completely new topic
How we built it
- We created complex web scraper to get and preprocess, vectorize and store hundreds of datasets from internet.
- Our AI processes data from thousands of Qdrant vector databases we use to store and structure our data, to allow precise and fast querying.
- User can anytime broaden the database by uploading his own datasets.
- After users question using sqlagent, langchain and genAI multiple queries are generated and evaluated to finally produce optimal SQL query and fetch data from db.
- Data is sent to RAG model with custom vector retriever (implemented by us) to analyze data and produce their description and answer to users question. Advantage is, that RAG can work with context, so our solution enables user to always improve the results.
- Finally data from db and answers from RAD are sent to our app, which provides user with description, answer, data vizualizations and operation analytics.
Challenges we ran into
- Creating insightful data visualizations
- Storing large amount of data while not forgetting speed and efectivity - structuring data
- Implementing our solution for SQL query creation and improvement
- Implementing our own RAG solution
- Creating human language interface without making just "GPT wrapper"
- Making usable UI, we are all backend devs :(
Accomplishments that we're proud of
- Hopefully usable UI :)
- Our original and complex and scalable data storage solution
- Our custom RAG and querying solutions
- I have never made so many charts which actually look better then matplotlib :)
What we learned
- much about RAG, langchain, SQL querying & creating frontend
What's next for LoremIpsum-1
- AI :)
Log in or sign up for Devpost to join the conversation.