VerticAI

Jump aspect highlight and AI summary
Jump analysis after a user uploads a video
Annotated jump footage using pose detection model
VerticAI Home Page

Inspiration

If you ever played basketball or volleyball, then you know how cool it is to have a better vertical jump than everybody else. So what if you could quantify your jump just by recording you jumping on your phone? And then you could get feedback on your technique through computer vision?

That’s why we created VerticAI. We think that out there, there are tons of casual sport players who want to improve on their skills, including their vertical jump. We were inspired by computer vision-based tools like SwingVision in tennis that give acessible personalized coaching.

What it does

You record on your phone a video of you jumping, and when you upload it to the website, it gives you:

Your estimated jump height
Your jump form
An annotated video of the different angles your joints make as you jump, with segmentation of the different jump phases
A composite score of your overall jump

How we built it

We used pre-trained models like Ultralytics and MediaPipe Pose Landmarker to mark the joints of your body in each frame. We used this information to calculate the angle between important joints in the jump (e.g the angle between your shoulder, hip, and knee), and compared these angles to literature "ideal values". For calculating the estimated jump height, we estimated the time of flight by detecting the takeoff and landing frames.

Challenges we ran into

Ensuring that the jump height is accurate was the hardest challenge that we ran into. We tested out different methods, and in the end we restricted the type of footage you can upload to strictly footage that are taken from a side angle to minimize the effects depth has. Also, as a team of beginner hackers, we had alot of issues integrating everything together due to planning. We also wanted to use Modal's GPUs to speed up our analysis algorithm process, but we ultimately ran out of time and focused on our core idea more. In the future, we will get to finishing this up.

Accomplishments that we're proud of

We built an end-to-end pipeline that takes a raw phone video and turns it into biomechanical feedback in minutes. We implemented automatic jump phase segmentation (approach, loading, takeoff, flight, landing) instead of just spitting out a single number and pretending that’s coaching. We calculated joint angles frame-by-frame and visualized them directly on the annotated video, so users can actually see where their form breaks down.

What we learned

Computer vision works great until you have to take into account depth. Full-stack integration is harder than building the model itself. Coordinating frontend uploads, backend processing, and annotated video output took more effort than expected. Small implementation details matter. A few frames off in takeoff detection can significantly skew jump height.

What's next for VerticAI

Next, we want to measure the height of the vertical jump by letting the user calibrate an object of known height with the app. This way, we can measure the height of the jump by measuring the pixels instead of using time of flight. Modal will be utilized in a large way to scale our project when we have users from all across the world using our website to further their vertical jump journey.