Inspiration
Our inspiration for this app stemmed from our frustrations with our Discrete Structures class. It was a large class with approximately five TAs per section, yet many students faced difficulties in finding the proper help outside of lectures. The professor and TAs were often overwhelmed with questions, making it challenging for everyone to get their concerns addressed. Moreover, we recognized that this tool could be invaluable not only for the general student population but also for those with learning disabilities. Students with learning disabilities often encounter additional hurdles in traditional learning environments, and our app aims to bridge those gaps by providing tailored support and accessible resources to meet their needs.
What it does
StudyBuzz takes an input of a media file, which can be either an audio file or video file. It then transcribes the full file, and our AI model processes the transcript of the lecture and generates a summary of the lecture, including key points and takeaways.
How we built it
To build StudyBuzz, we utilized Python, Javascript, HTML, and CSS. We built StudyBuzz utilizing multiple different technologies. We built the frontend utilizing React due to its quick and easy customizability. We connected our React frontend to a Python powered backend using Flask. On the backend side, to transcribe text, we utilized OpenAi's Whisper API. The transcription was then passed to an AI large language model. The model was trained on a dataset of books and their relative summaries to summarize any given text. In particular, the model was designed to surpass the limitations of normal language models, and create longer summaries for longer texts to accommodate the flexible length of lectures.
Challenges we ran into
Our main challenge, having never worked with full stack before, was connecting the frontend and backend with Flask. When we dedicated our time to learning how to integrate them together, we began to see results.
Accomplishments that we're proud of
One of our biggest accomplishments was developing our LLM for summarization. None of us had worked with AI or making LLMs, so it was a new experience for all of us. Being able to develop an algorithm that was able to summarize videos well was a huge accomplishment for all of us, and its only the beginning for our AI development potential
What we learned
One of our main learning experiences was with Flask, and learning how to connect the frontend with the backend. Our development plan was to start building everything simultaneously, with each team member working on a different portion and combining them together when we completed them. We realized when it came to combining the front and back end we would have difficulties because we didn't develop them together. Fixing that error and learning those design principles was definitely a huge learning and growth experience for us. Alongside the full stack development, developing the LLM was also a learning experience. We had initially started out by trying to train our data through Salesforce's XGen AI model. We quickly realized that it was definitely not the right move to try and run a massive corporation's model with billions of parameters on our local machines. We found a model with around 400 million parameters that would be able to run on our machines more reasonably, and trained it with the dataset of books to create our summarization algorithm. Overall, both experiences were big points of learning for us and definitely helped us become better developers and engineers.
What's next for StudyBuzz
We have identified various ways to grow StudyBuzz, stemming from our recognition of students' dissatisfaction with the current support provided by professors. Firstly, we plan to incorporate clips of essential lecture moments that correspond to key elements of the summarization. More significantly, our aim is to cultivate inclusive classrooms where students feel at ease engaging in discussions and collaborations with both their professors and peers. To achieve this, we intend to introduce a discussion feature that allows students to highlight specific sections of the summary, enabling them to ask questions and initiate discussions based on these highlights. Our vision is that this will create a unified classroom where students no longer feel unsupported by their professors.
Log in or sign up for Devpost to join the conversation.