PauzzZ

Inspiration

As sleep-deprived students, all of our team members have agreed on having had trouble focusing during in-person lectures. But is watching recordings better? In most cases, recordings are even more challenging to maintain focus on, due to their static nature, and lack the real-time engagement. Given the high likelihood of students dozing off while watching recordings, our goal was to develop a Chrome Extension aimed at improving the remote learning experience for those dealing with sleep deprivation

What it does

PauzzZ makes sure the user doesn't miss any important lecture material by pausing the video when it detects that the user is likely asleep. In fact, if the user has their eyes closed for more than 5 seconds, the extension automatically pauses the video and rewinds the lecture by 5 seconds.

How we built it

Chrome Extension

Language: JavaScript

Manifest Version: Version 3, the latest version for improved security and performance optimizations.

Backend

Framework: Django (Python)

Pretrained Resnet inspired CNN model

Challenges we ran into

Since our extension is mainly designed for university students, we focused on getting our extension to work on McGill's MyCourses Page. We spent a significant amount of time trying to grab the video element and eventually manipulate it to pause and rewind depending on the eye status. We've identified that we had nested iframes preventing us from accessing the content of the inner iframe due to the same-origin policy. However, we found out that the third party who was hosting the content of the inner iframe was LRS(Lecture Recording System) by McGill and could access the video element without any iframes in the source code.
Although we ended up using the pre-trained model, we did put a lot of effort into training our own models using ANN and CNN. However, each model had some edge cases (ex: the user has glasses), for which the accuracy was significantly lower.

Accomplishments that we're proud of

It was our first time working on the Chrome extension, and we got it to work
Being able to connect the front-end and the back-end of both Django and Chrome extension

What we learned

How to make a Chrome Extension!

What's next for PauzzZ

Making our project more inclusive by getting our extension to pause the video when the user looks away. This could be ideal for people with disabilities.
Improving the accuracies for our own trained model, and eventually replacing our current pre-trained model with our own.