One Perfect Form

Hero — One Perfect Form

Inspiration

Our application Hero was inspired by the popular Japanese anime One Punch Man, where an average salesman named Saitama becomes an unstoppable superhero capable of defeating any enemy with just one punch. We took that same idea—ordinary consistency turning into extraordinary results—and applied it to fitness. Hero uses Saitama’s training theme as motivation, but the real superpower we focus on is something more practical: perfect form.

What it does

Starting a training journey is hard—especially when you don’t know if you’re doing the movement correctly. Hero helps people build confidence by analyzing core exercises with rep-level feedback.

Upload a workout video (currently supports push-ups, squats, and deadlifts).
Hero detects reps and returns a 1–10 form rating plus mistakes and corrections for each rep.
The results page visualizes performance per rep and can render a skeleton overlay (pose landmarks) to make feedback concrete.
For quick coaching, Hero also generates an audio overview of the top corrections so users can listen and immediately retry.

How we built it

Frontend: React + TypeScript (Vite), styled with Tailwind + shadcn/ui for a fast, clean web experience; the UI drives upload, polling, rep-by-rep visualization, skeleton overlay, and audio playback.

Backend: FastAPI handles video uploads and exposes result/health endpoints; the frontend is deployed on Vercel and routes API calls to the Vultr-hosted backend.

Movement: OpenCV decodes and samples frames, and MediaPipe Pose extracts landmark tracks; we derive compact biomechanical signals (e.g., joint/segment angles) that make the analysis robust and explainable.

Worker: Celery runs the long analysis jobs asynchronously with Redis as the broker + result backend; the worker calls the Gemini API (Google GenAI SDK) to return strict JSON coaching feedback, and buffers landmark data for the overlay.

Challenges we ran into

Our number one challenge was feature-engineering the prompt + schema for the Gemini API so the model reliably produced correct ratings and actionable feedback for an exercise (and did so consistently across reps). We also had to build a pipeline that’s robust to real-world video issues—different angles, lighting, partial reps, and noisy pose estimates—and still produce usable coaching.
Another challenge was keeping the experience responsive: video analysis is heavy, so we designed the system around asynchronous processing (Celery/Redis) and polling-based results. Finally, we had to make the feedback feel real: tying mistakes to rep windows and visual landmarks took careful iteration.

Accomplishments that we’re proud of

Shipped an end-to-end experience: upload → async processing → rep-by-rep results → visual overlay → audio coaching.
Deployed the backend on Vultr with a repeatable Docker workflow (and kept the frontend deploy simple on Vercel).
Built an analysis pipeline that blends computer vision signals (MediaPipe) with LLM reasoning (Gemini) instead of relying on either alone.
Built for scale from day one using Celery workers, so multiple users can submit videos without the API stalling.

What we learned

We learned how to use Celery workers to support multiple users at once, how to deploy a production backend on Vultr, and how to engineer an AI system where the model output is constrained, parseable, and user-facing. We also learned the hard truth of hackathons: the “last 10%” (reliability, UX clarity, and deployment) takes as much energy as the first 90%. And yes... we learned how to stay up all night.

What’s next for Hero

Upgrade to a more rigorously tested movement model (and run real evaluation on real athlete footage).
Expand exercise support and improve rep segmentation and fault localization.
Ship a mobile app (Android/iOS) so users can record and get feedback in one flow.
Add persistence and personalization (e.g., user history, progress tracking).