Breather

• 

  The stats window!

Inspiration

In the 21st century, practically everyone who has access to a computer is glued to their device. Many of us, including developers, are couch potatoes, and when work starts piling up we tend to sit in front of our screens for hours on end, leading to an unhealthy lifestyle filled with work and stress. It is especially easy to get fatigued while staring at a screen for a prolonged period of time, and even easier to forget to take a break when screens are designed to keep us invested. Thus, we decided to create Breather, a menubar app that detects user fatigue based on typing patterns and reminds its users to periodically take a break.

What it does

As we've previously mentioned, Breather detects user fatigue based on typing trends. It does so using a few metrics: changes in typing speed, error rate, average flight time (time between consecutive keys pressed), and average hold time (time a key is held down). If Breather detects that the user is fatigued based on these metrics, a popup notification is sent to the user, reminding them to take a break.

How we built it

We built Breather entirely in Python, using PySide6 (the Python API for the Qt framework) for the frontend. We also used multiple open-source packages to add functionality to the app, including pynput for keystroke monitoring and desktop-notifier for desktop notifications.

Breather had a significant research process to ensure sensible fatigue analysis. We first used ChatGPT to get a good idea of the current state of the art in keystroke fatigue detection, and then asked it for papers which we could read to get a better understanding of the topic. Notably, the following papers were very helpful in our research:

• Automated stress detection using keystroke and linguistic features: An exploratory study by Vizer, Zhou, and Sears
• Detection of Mental Fatigue in the General Population: Feasibility Study of Keystroke Dynamics as a Real-world Biomarker by Acien et al.
• Keystroke Dynamics: Concepts, Techniques, and Applications by Shadman et al.

Our formula uses hold time, flight time, error rate (as calculated from backspace/delete rate) and WPM to determine fatigue. Specifically, higher hold times, flight times, and error rates, and lower WPMs reflect possible fatigue, as compared to mean values. We compute the fatigue by summing the z-scores of the former three metrics and subtracting the latter (clamped to not cross 0). The "lifetime" means and variances for these metrics are calculated by starting from well-known population values (e.g. 0.110s is the average global hold time), and then slowly trending toward the user's average as they continue to use the app.

Challenges we ran into

Since our app is designed to run on all major operating systems, the differences in OS yielded difficulties in standardizing notification sending (macOS security settings in particular made it difficult to display notifications).

On the backend side, we encountered a lot of issues with getting our fatigue model right. We started with a complex multi-featured weighted model, which the literature supports, but due to the lack of real-world data, limited time to work on it, and difficulty with debugging we had to go back to the drawing board. Eventually, we decided on a simpler formula that reflected the features we found most indicative of fatigue: hold time, flight time, error rate and WPM.

Another challenge was implementing custom data structures to enable recency analysis of the user's keyboard activity, while maintaing a lifetime metric as well; we extended Python's deque class to allow for this in an efficient manner, while having a separate sub-structure to store approximate, easily-updatable lifetime data.

Finally, synchronizing the frontend and backend was a challenge, as we had to ensure that our app was properly threaded so it wouldn't block the main GUI thread, while also ensuring that there were no race conditions.

Accomplishments that we're proud of

For the majority of the team, this is our first hackathon and first experience using Python or even building CS projects. Within the short timeframe of 30 hours, we were able to create a menubar application that functions across various platforms. Throughout the project, there was a lot of "learning on the fly", using whatever resources we could get our hands on to gain familiarity with the language and its libraries. We're proud that we were able to get the app to output sensible data and look polished in the short time we're given, and that we were able to tackle a complex topic like fatigue detection.

What we learned

We learned to research effectively by thinking critically about our papers so that we choose only the most practical features of keystroke patterns to include in our model, and to constantly question how our data is being used so that we only use what we need to improve efficiency. For example, we started with a complex multi-feature model that we eventually chopped down to just 4 features.

Instead of using large lists to hold lifetime speed and timing data, we used running counts, sums and square sums to make means and standard deviations calculatable at any time over the course of the program.

Additionally, many of us have learned not only to use libraries such as Pynput and PySide6, but also how to hunt for the best libraries available to limit the weight of our application while allowing for the best functionality. While we would've had a much easier time with the analysis part of this project by importing NumPy or SciPy, we performed all computations independently, creating our own data structures that were relatively efficient and avoided the heavy startup overhead of the aforementioned packages.

What's next for Breather

In the future, Breather could be enhanced by incorporating additional user activity data to more accurately assess fatigue levels. For instance, monitoring mouse movement patterns—such as speed and idle times—can provide deeper insights into a user's fatigue, and can be implemented using libraries like Pynput. Furthermore, eye-tracking technologies, leveraging computer vision models, can be integrated to detect signs of fatigue such as frequent blinking, slow gaze movements, or prolonged periods of eye closure.

Aside from fatigue detection, Breather can easily expand to include more wellness features. For example, a smart break timer could be implemented to periodically remind users to take breaks, encouraging users to rest both during long continuous computer usage sessions and when physiological indicators of stress or fatigue are detected.

Built With

• python
• qt