TouchGrass

Inspiration

A few semesters ago, one of our team members had taken on too many projects and responsibilities and wasn’t keeping tabs on how much he was stretching himself thin. As a result, he burnt out, and he had to take some time off to recover.

This got us thinking: what if we could proactively detect signs of burnout early and prevent it before it’s too late? That’s why we built TouchGrass, a tool that empowers coders to monitor their productivity and gain better self-awareness.

What it does

Our web app monitors your webcam to estimate your mood and how much time you're spending at your desk each day. It also monitors your Linear account to see how quickly you're crushing story points. It uses all of this data to predict when you might be prone to burnout and proactively remind you to take breaks if your risk reaches a threshold.

How we built it

We used Convex – a platform-as-a-service that gives real-time reactivity out of the box and type safety across the entire stack – to power our database, blob storage, and server actions. We paired Convex with a React frontend built with Vite, styled with Tailwind, and deployed with Vercel. Users are authenticated with Auth0.

Our app integrates with three services to gather data about the user. To estimate the user’s mood at each moment, we upload 10-second chunks of video from their webcam to Convex as blobs then send them to the TwelveLabs Pegasus API, the state-of-the-art video-to-text understanding model, for analysis. We also collect statistics on how much time the user spends coding using Wakatime, an IDE time tracking extension, and coding productivity stats from GitHub and Linear (a project management tool).

Challenges we ran into

Roman hadn’t seriously used Convex before, so there was a bit of a learning curve to understand the best way to structure the TwelveLabs video processing flow. We

Accomplishments that we're proud of

• Successfully integrated a variety of external APIs
• Auth0 implementation
• Talked to many people for feedback

What we learned

• How to use Convex for backend logic
• What the TwelveLabs API is capable of and how to produce structured responses

What's next for TouchGrass

If TouchGrass was scaled up and gained many users, we could validate our burnout risk prediction against real user data and train an ML model to make more accurate predictions. We could also add a host of other integrations to gain as much context about the user as possible: in particular, we originally wanted to collect data on the user’s sleep quality using a Google Health integration.

Finally, burnout is a sensitive topic, and usage stats and webcam video are invasive forms of data collection if used irresponsibly. We’d like to implement end-to-end encryption and comply with data privacy regulations to keep users safe and make them feel comfortable using our software, while still using techniques like federated learning to use the data to improve our burnout prediction model for everyone.

Built With

Convex, React, Vite, Tailwind, TwelveLabs, Auth0, Wakatime, GitHub, Linear

Built With

• convex
• linear
• react
• tailwind
• twelvelabs
• vite