Inspiration
Every year, ~24,000 reports of on-campus crimes are reported and investigators often face the daunting task of analyzing vast amounts of data on confiscated devices, such as phones, laptops, and tablets. Current methods are labor-intensive, requiring significant time and resources to manually search for evidence and we wanted to make something that can drastically reduce the time searching through files by combining AI and robust search functionality. This tool transforms a previously manual and time-consuming task into a streamlined, tech-driven process.
What it does
Sort My Stuff (SMS) allows the user to select a folder to recursively index information about. In addition to displaying metadata about each file/directory in the folder, SMS also generates a description of each file/directory based on the content and context of the file, effectively streamlining the user's file system analysis process. Additionally, SMS implements a smart searching feature to quickly and elegantly find a file based off of a human-friendly prompt as well as provide for common filters such as file extensions.
How we built it
We built both a GUI and a CLI frontend for our tool. For the CLI frontend, we used a Python script that imported our directory indexing algorithm. We built the GUI desktop app with Electron, HTML, CSS, and JavaScript.
For the backend segment of our project, we used MongoDB to efficiently store our directory indexing information. We then used a LLM (Llama/Gemini) to recursively generate descriptions of each file/directory, which included smart processing of images and video frames. After we collated descriptions of the files, we implemented a smart search through the files using LLMs.
Challenges we ran into
The biggest challenge we ran into was selecting which generative AI model to use. Whereas Llama didn't have an API quota and was run locally on our machines, other generative AI models processed prompts much quicker, leading to more efficient processing. We had to decide the most important features of our project to effectively decide which model to use.
Additionally, we also ran into challenges trying to connect our database to our desktop frontend. ICP was difficult to wrap our heads around, and it was sometimes difficult to determine what part of the code the error was coming from. However, we were able to work past that and develop a functioning app.
Accomplishments that we're proud of
We are proud of the fact that we were able to build the application that we envisioned, especially since Electron and MongoDB were new to the majority of us and we were learning it on the spot.
What we learned
We learned about how to develop desktop apps in Electron and MongoDB. We also learned that when we run into problems, we could ask someone else to look at our code because different people see the same thing in a different manner, resulting in a more efficient way of debugging.
What's next for Sort My Stuff (SMS)
A few things we want to implement in the future are speech-to-text searching, adding more advanced search, and making it both multi-modal and multilingual. We want our software to be applicable in a wider range of situations while also making it more accessible.
Log in or sign up for Devpost to join the conversation.