Inspiration
Inspired by JetBrains' suite of IDEs, CSVision bridges the gap between beginners learning to code and experienced data analysts looking to streamline and simplify their workflow by offering an easy-to-follow, highly connected, user-friendly environment for table management and plotting.
What it does
CSVision is a tool that allows users to efficiently work with CSV tables to perform essential database manipulation tasks by communicating with an AI language model to automatically generate code for managing and viewing database contents. With CSVision, the user can:
- import and read tables in CSV format
- chat with OpenAI's GPT4 language model to perform SQL queries or plot data using the CSV tables as inputs
How we built it
CSVision's front-end is built using Node.js and React Flow for creating a stable platform with a modern UI and seamless user experience.
The back-end is written in Python and contains various libraries for working with tabular data, such as pandas and pandasql for working with the tables, matplotlib to create and incorporate graphical representations of the table data, and networkx to store each individual table as a digraph node.
The database system works by converting the CSV files to Pandas' DataFrames to ensure fast, flexible and modular management of large-scale tables. Each database table and operation is stored as a node in a directed graph. As such, the result of each operation can be logged and traced back to allow pipelining of on-the-fly changes.
Communication between the front- and back-end occurs using WebSockets for near-instant feedback.
To run the application, install the necessary requirements, then run python3 src/backend/runServer.py to load the back-end, npm i to install the Node.js dependencies, and finally npm run dev to host the front-end locally.
Challenges we ran into
Production went surprisingly smoothly, with little to no conflicts (neither of interest, nor of git commits) and great teamwork all around. However, that does not mean that we didn't encounter our fair share of challenges, including (but not limited to):
- engineering the GPT prompts to maximize consistent, accurate, high-quality output while also keeping the efficiency high
- ensuring flawless communication between the front- and back-end while also updating events in real-time
- keeping the flow and idea of the project consistent, prioritizing certain features over others
- finding a good algorithm to plot data, since generating full-on Python code to be executed live is significantly harder than creating a SQL query with a pre-determined database
The bottom line is: the easier it is for you to use, the harder it was for us to make.
Accomplishments that we're proud of
We are incredibly proud to present an honest tool that programmers of all skill levels can use to improve their workflow, having created such a project in less than 36 hours completely from scratch.
What we learned
Developing CSVision taught us valuable lessons in the intersection of AI, database management, and user experience design. We gained insights into prompt engineering, front- and back-end communication, database systems and many more.
What's next for CSVision
CSVision is an ever-growing project. While certain ideas may have not made the cut due to the tight deadline, they still play an essential part in improving the overall user experience. These ideas are:
- pipelining (one small change in a table propagates to all further tables created using the original input)
- workflow splicing (creating and simultaneously managing multiple canvases)
- exporting newly created tables
- undo / redo
- custom queries
- overall improvements to the AI's responses (i.e. function calls)
- integration with other database / spreadsheet systems, such as Excel or Sheets.
Log in or sign up for Devpost to join the conversation.