Keynotes

Inspiration 🚀

Every student knows that feeling of utter panic when you're in your 8AM lecture, severely sleep deprived and blanking out...what did the prof just say? I guess we'll never know. Based on our highly researched data, we concluded quickly that students nowadays are some of the most easily distracted students to have existed in student kind (in fact, most students didn't even finish our survey). At Hack the North, inspired by some of the universal problems that students face, we wanted to develop a tool to revolutionize the way we learn and take notes.

Enter KeyNote, the personalized notes device leveraged by AI– made for students, by students.

What it does ⚙️

KeyNote is a powerful note-taking device paired with a web-app that leverages AI to expand the lesson beyond the lecture. Think, the best friend that'll always share their notes and answer the questions that you have from class. All users must do is record the lecture through our device, controlling it through the "Start/Stop" buttons. Then, KeyNote will process the transcript, and provide a summary of key concepts taken from the lecture, as well as the option to ask lecture-specific questions through our chat feature-- all previous transcripts being available through our curated calendar!

KeyNote allows students to not only review and ask about their lecture, but they can make their own annotations and edits on top of the key concepts, just as they would on paper. There are additional features allowing users to add, delete and modify the notes, giving students the power over their own learning.

How we built it 🛠️

KeyNote, our innovative web app, was constructed using TypeScript and Next.js for the front-end, while Python with FastAPI powered the backend. This blend ensured a reliable and scalable platform. Hardware integration was achieved through Python, facilitating seamless connections with our devices. Google Cloud Services played a pivotal role, enabling the retrieval and transmission of recordings. Assembly AI was leveraged for transcription tasks, and Cohere handled extensive data processing. This synergy of technologies has made KeyNote a game-changing solution for our users.

Challenges we ran into 🔥

While developing KeyNote, we ran into several challenges over the weekend. Given our idea, we were incredibly excited to integrate Cohere’s different services within our platform. In context of Cohere, most of the features we used were experimental services, which sometimes did not have the support to maintain the context in terms of larger sets of data. In turn, we had to do batch-wise processing in order to ensure that we had full data coverage.

Another challenge was integrating the front-end and backend. All of our components were programmed with extensibility in mind, meaning we created a dynamic interface that constantly interacts with the backend in order to pass in data- figuring out how to program our calendar feature was a challenge given the fact that we were not familiar with our tech stack. On a similar note, we had to learn how routing worked in Next.Js- a specific challenge being learning dynamic routing to create unique endpoints for each Note.

Accomplishments that we're proud of 💖

We’re incredibly proud that we were able to build a functioning device and platform that we most certainly find useful and relevant to our day-to-day life. Specifically, it was incredibly exciting exploring brand-new cutting-edge technologies that helped us make this idea a reality- what initially seemed like a wildly ambitious idea was made possible through meticulous planning, designing and learning on the go! Over this past weekend, we learnt how to create a software-hardware project, exploring how to integrate the front-end, backend, databases and hardware with each other was no-easy feat! We are incredibly excited to show you what we have in store.

What we learned 🌍

In reflecting over the past 36 hours, we may all agree that we’re taking away both technical learnings, but also best practices when working in a collaborative environment under time constraints! We found ourselves using Githubs multifaceted services and assigning ongoing tasks to each other in order to stay on track. In addition, working with many of Cohere’s services was both incredibly exciting, but also challenging. In hindsight, we really benefited from the variety of workshops and mentorship available throughout the event. As a team, arguably our biggest takeaway was the impact that modern technology has and will have on society– many of the world's solutions can be solved with the right technology, and a little bit of sleep deprivation!

What's next for Keynotes 💫

Looking ahead, some of the immediate additions we would like to make to KeyNotes would be adding an additional feature to allow users to jump to specific parts of the transcript based on their notes and questions. We would also love to explore some of Cohere’s other features such as Rerank, in order to make KeyNote a more personalized and efficient platform.