Form Wise

Inspiration

One of our engineers recently watched a demo of AI interpreting sign language through a camera, and it sparked the thought: what if similar technology could be used to improve wellness and fitness? Exercise injuries often come from poor form, and most people don’t have access to a personal trainer every time they work out. We believed AI could act as a real-time fitness coach, giving immediate, personalized feedback that helps people train safely, stay consistent, and prevent harm. That idea grew into Form Wise—a lightweight, accessible way to bring real-time coaching into any workout.

What it does

Form Wise transforms any camera into a real-time exercise coach. Using MediaPipe’s pose detection, TensorFlow.js, and a React front end, it tracks a user’s body as they perform exercises like squats and planks. The system calculates joint angles, detects improper form, and provides instant auditory and visual feedback. Users hear cues such as “Straighten your body” or “Rep 5,” while seeing live visual feedback on their screen. The result is a simple, accessible tool that makes at-home exercise safer, more effective, and more engaging—without expensive equipment or trainers.

What Makes Form Wise Special?

Real-time movement feedback: Users get immediate voice cues and on-screen corrections as they exercise.

Form analysis engine: Uses angles from multiple body landmarks to assess squat depth, plank alignment, and more.

Lightweight & accessible: Runs entirely in the browser with no special hardware—just a webcam or phone camera.

Rep tracking & motivation: Counts reps, recognizes states (up/down), and provides encouraging prompts to keep users engaged.

Scalable foundation: Built to expand into more exercises and potentially support group sessions or live trainer integration.

How we built it

Our system combines a modern front-end with efficient AI libraries to deliver a seamless experience:

Frontend UI (React + Tailwind CSS)

Clean, responsive design for exercise mode switching, rep counts, and feedback panels.

Tailwind and custom CSS give a polished “fitness app” look with minimal overhead.

Pose Detection (MediaPipe + TensorFlow.js)

MediaPipe Pose Landmarker provides landmark data for shoulders, hips, knees, and ankles.

TensorFlow.js handles backend processing with GPU acceleration in-browser.

Custom angle calculations, power rep detection, and form analysis logic.

Feedback & Voice Coaching

State machines for squats and planks determine transitions (“down → up”).

Speech synthesis API delivers short, calm coaching cues in real time.

The feedback box shows textual guidance for clarity.

DevOps & Environment

Built with Vite for fast bundling and hot reload.

Fully client-side: no external servers needed.

Deployable as a web app, designed for future mobile packaging.

Challenges we ran into

We ran into several challenges throughout development:

API experimentation: We initially tried integrating Google’s Gemini API for analysis, but it didn’t align well with real-time pose detection. Pivoting to MediaPipe and TensorFlow solved the issue.

Mobile limitations: Running high-frequency pose estimation on mobile devices proved resource-intensive, making optimization a key hurdle.

Calibration: Getting thresholds for squat depth and plank straightness accurate enough to feel natural took repeated testing and tuning.

Accomplishments that we're proud of

We’re proud that Form Wise works live and delivers real-time coaching cues. In under 48 hours, we created a functioning fitness coach that detects movements, counts reps, and provides actionable guidance—all within a polished UI. Our biggest accomplishment is achieving live feedback that feels responsive and natural, showing the potential of AI as a fitness companion.

What we learned

We learned how to integrate MediaPipe and TensorFlow.js for real-time pose detection and how to combine state machines with angle calculations to track exercise progress. We also gained experience in optimizing real-time video in React, balancing visual rendering and AI inference. Most importantly, we learned how to design user-friendly feedback loops—keeping cues short, clear, and motivational makes the tool feel like a real coach.

What's next for Form Wise

Form Wise already supports multiple exercises beyond squats and planks, and our next steps focus on scaling its capabilities. We plan to optimize the system for mobile devices, support multi-user sessions for group workouts and trainer-led coaching, and enhance progress tracking with personalized feedback and performance history. Long-term, we want to gamify the experience with challenges and achievements, making fitness both engaging and accessible anywhere.