AI Physiotherapy Coach
Progress page
Interactive body map for pain tracking
MediaPipe Joint Keypoints
Freesio Therapist Dashboard
Exercise Calendar

About Freesio Therapist

Inspiration

One of our friends' struggles with home physiotherapy recovery from scoliosis sparked the idea. Despite having detailed therapist instructions, they constantly questioned their exercise form, leading to inconsistent practice and slower recovery. Also, giving up on the process entirely due to the complications. We talked to around 8 other people who either said they skipped their home physiotherapy because they didn't know how to do it or were demotivated because they are already fatigued. We realized that this is a universal problem, and millions of patients fail to maintain proper form at home, resulting in poor adherence and suboptimal outcomes. We envisioned bringing professional physiotherapy guidance into every patient's living room through AI-powered real-time coaching.

What it does

Freesio Therapist is an AI-powered home physiotherapy platform that provides personalized, real-time exercise guidance. It features interactive body mapping for pain assessment, AI-generated exercise prescriptions based on individual conditions, real-time pose estimation using MediaPipe for form validation, comprehensive progress tracking with motivational systems, and dynamic exercise calendars that adapt to patient sessions. The platform bridges the gap between clinic visits and home practice, ensuring patients receive professional-grade guidance anytime.

How we used Gemini

The way it works is there's no hardcoded exercises (other than fallback ones), instead the user's demographics such as weight, and height and the location of the pain (using the body-map shown in the picture) are passed to Gemini API to generate the exercise relevant to the pain the user has, the joint indices correlating with MediaPipe (check pictures) to be used in calculating the angle of their movement, and the peak angle, which, when the user reaches with their movement, a repetition is counted.

How we built it

We built Freesio Therapist using Next.js 14 with TypeScript and Tailwind CSS for the frontend, and*Supabase* for backend database management and authentication. The core AI features include MediaPipe Pose for real-time 33-point body tracking, Google Gemini AI for intelligent exercise generation, and custom angle calculation algorithms for precise movement analysis. The platform integrates comprehensive user profiling, medical condition tracking, session management, and real-time form validation with instant feedback systems.

Challenges we ran into

Technical challenges included optimizing real-time video processing for smooth pose estimation across different devices, ensuring cross-platform compatibility with varying browser implementations, and creating accurate angle calculations for different body types and movement patterns. UX challenges involved designing intuitive body mapping interfaces, balancing gamification with healthcare professionalism, and ensuring accessibility for users with different technical comfort levels. Healthcare integration challenges included implementing robust data privacy measures, ensuring medical accuracy in exercise recommendations, and creating truly personalized programs that adapt to individual progress.

Accomplishments that we're proud of

We successfully created real-time pose estimation with sub-degree accuracy, AI-powered exercise generation that adapts to individual conditions, and a comprehensive data architecture that seamlessly integrates patient information. Our interactive body mapping transforms subjective pain into quantifiable data, while our real-time coaching provides immediate, actionable feedback. We've built a scalable solution that bridges the clinic-home gap with professional-grade guidance available 24/7, maintaining the quality of care while improving accessibility for millions of patients worldwide.

What we learned

We discovered that real-time video processing requires careful optimization and fallback strategies, pose estimation accuracy depends heavily on camera positioning and lighting, and cross-browser compatibility needs extensive testing. In healthcare applications, motivation is deeply tied to visible progress and immediate feedback, while personalization significantly improves user engagement and treatment adherence. We learned that interdisciplinary collaboration between technical and healthcare expertise is essential, and user feedback is invaluable for healthcare applications requiring continuous iteration and real-world validation.

What's next for Freesio Therapist

Immediate goals include expanding our exercise library, enhancing AI algorithms for more precise form validation, developing mobile apps, and integrating wearable devices. Medium-term plans involve building a therapist dashboard for remote monitoring, adding video consultation features, developing advanced analytics with machine learning insights, and forming partnerships with healthcare providers for clinical validation. Long-term vision includes multi-language support for global accessibility, EHR system integration, advanced AI features with predictive analytics, and expansion to other rehabilitation domains beyond physiotherapy.