The Posture and Focus Study Buddy

Posture apps and screen-time monitors just don't work. They send a polite push notification that people immediately get rid of, and usually these apps only track one metric at a time; it could be a posture app or a focusing app. There are no real consequences for ignoring them.

LumbarJack is a fully local computer vision monitor that actively enforces focus and good posture. It monitors your posture and your visual attention. If you slouch, lean close to the screen, or look down at your phone, it starts the protocol to get you focused and sitting healthy:

1. Pauses your media automatically via keyboard hooks so that you don't miss your lecture or informational video.
2. Gives you time to get back to being productive with first a simple desktop notification popup.
3. After a few seconds a text message is sent to your phone number telling you to get back to work!
4. Finally, Mr. Hyde himself comes and tells you to "LOCK BACK IN!"

This project was built around the dual hackathon theme.

• Dr. Jekyll: A silent, background health helper. It tracks your posture and spine health, making sure that you are working without hurting your body. Like a guardian of sorts.
• Mr. Hyde: A troll that punishes if you don't focus. The second you lose focus or compromise your posture, it takes control of your machine by stopping media, sends an SMS message to your phone, and publicly calls you out over your speakers!

The entire stack is local, except for the optional SMS messaging service (which is paid but free trial is used here).

OpenCV and Google's MediaPipe Holistic model (v0.10.14) was used to generate 100+ facial and body landmarks locally. No video data is sent to the cloud so that the user can rest easy knowing privacy is not an issue.

Vector math is being used here in all aspects of calculating where you are looking and what your posture is looking like. For example, the Eye Aspect Ratio (EAR) is calculated to catch phone-scrolling or looking down, absolute-value Yaw/Pitch values are used for head turns, and shoulder to facial distance comparison to detect slouching"

Plyer is used to give the user desktop notifications. Twilio, which is a cloud-based communication service, is used to give mobile text notifications. Pygame is used to play audio of Mr. Hyde calling the user out. While the media interrupts are done with PyAutoGui.

You can run the core vision server locally with just a standard webcam, for the demo, DroidCam was used.

Note on Twilio: The SMS feature is just optional. If you do not provide Twilio API keys, the application will just skip the text messaging step and proceed directly to the Mr. Hyde audio. It is a paid service.

Install the required Python packages:

pip install opencv-python mediapipe==0.10.14 plyer pyautogui pygame twilio python-dotenv

If you want to enable the SMS warnings, create a .env file in the root directory:

TWILIO_ACCOUNT_SID=your_sid_here
TWILIO_AUTH_TOKEN=your_token_here
TWILIO_PHONE_NUMBER=+1234567890
YOUR_REAL_PHONE_NUMBER=+1987654321

If you are skipping the SMS feature, simply ensure ENABLE_SMS = False is set in your main.py file.

Make sure that there is an audio file named lock_in.mp3 in the root directory.

Sit in a natural and healthy posture so the application can calibrate your baseline, then execute the script:

python main.py