Movement Coach

Movement Coach is a fitness assistant that tracks your body movements in real time and provides live coaching feedback, providing a hands-free workout experience powered through your webcam.

Comment

🧠 Inspiration

Many people struggle to maintain proper form while exercising, especially when working out alone. Poor technique can reduce effectiveness and even lead to injury. We wanted to create an accessible, real-time solution that provides instant feedback—like having a personal trainer anywhere, anytime.

⚙️ What it does

Movement Coach is a browser-based fitness assistant that tracks your body movements in real time and provides live coaching feedback.

It can:

  • Detect exercises such as squats, push-ups, lunges, jumping jacks, and planks
  • Count repetitions locally for instant responsiveness
  • Analyze posture and movement patterns
  • Deliver short coaching cues using AI
  • Speak feedback aloud while the user is exercising

The result is an interactive, hands-free workout experience powered entirely through your webcam.

🛠️ How we built it

We combined computer vision and AI to create a responsive coaching system:

  • Pose Tracking: Implemented using MediaPipe Pose Landmarker for real-time body landmark detection in the browser
  • AI Coaching Layer: Integrated Google Gemini via a secure local proxy to interpret movements and generate coaching feedback
  • Frontend: Built with JavaScript and browser APIs for real-time rendering and interaction
  • Backend Proxy: Node.js server to securely handle API requests and protect API keys
  • Voice Feedback: Text-to-speech system for real-time audio coaching

To ensure low latency, repetition counting and pose tracking are processed locally, while AI is used for higher-level insights and feedback.

⚔️ Challenges we ran into

One major challenge was balancing real-time performance with AI integration. Continuous API calls would introduce lag, so we separated responsibilities:

  • Local processing for fast repetition counting
  • AI used only for high-level coaching cues

Another challenge was accurate exercise detection across different users and environments. Variations in camera angles, lighting, and body types required careful tuning of thresholds and logic.

We also had to ensure secure API usage, keeping sensitive keys server-side while maintaining a smooth frontend experience.

🏆 Accomplishments that we’re proud of

  • Built a fully browser-based real-time pose tracking system
  • Achieved low-latency rep counting without relying on the cloud
  • Successfully integrated AI coaching without compromising responsiveness
  • Created a hands-free, voice-guided fitness experience
  • Designed a scalable architecture separating tracking, logic, and AI layers

đź“š What we learned

  • How to use real-time computer vision in the browser
  • Trade-offs between local computation and cloud AI
  • Designing systems for low latency and responsiveness
  • Handling noisy real-world data (lighting, angles, movement variability)
  • Structuring full-stack apps with secure API handling

🚀 What’s next for Movement Coach

We plan to expand the platform with:

  • Personalized workout plans
  • Form correction with visual overlays
  • Progress tracking and analytics dashboard
  • Mobile optimization
  • Support for more exercises and advanced movements

We also see potential in accessibility, helping users who may not have access to professional training.

Built With

Share this project:

Updates