AccuBoxer AI

Inspiration

In boxing and martial arts, we’ve all seen how subjective judging can lead to unfair outcomes. Unfair decisions often overshadow the fighters' hard work - we knew there had to be a better way. AccuBoxer AI was born out of our desire to bring consistency and fairness to the sport we love.

We’ve all experienced the frustration of watching a fight and feeling like the wrong fighter got the win. It’s not only about differences in opinion—sometimes, it's the inherent flaws in human judgment. We needed a solution that could objectively assess what happens in the ring, free from the influence of bias or error. The challenge of ensuring every punch is counted fairly, especially in the heat of the moment, we aimed to create a tool that tracks and analyzes every movement with precision, giving fighters and fans alike the confidence that every decision is made with accuracy.

Our backgrounds in Boxing, BJJ, Muay Thai, and MMA gave us a deep appreciation for the intricacies of combat sports. We didn’t just want to improve judging—we wanted to revolutionize it. AccuBoxer AI is our commitment to the challenges faced by athletes, coaches, and spectators who demand fairness in every match.

What it does

Introducing AccuBoxer AI, your best friend in eliminating human error and bias in martial arts judging. AccuBoxer leverages its uniquely trained AI detection model, breaking boxing videos down frame by frame and analysing the punches thrown and clean hits to the head of boxers, displaying the final decision of the AI in an easy-to-use interface along with the AccuBoxer's analysis of the video.

We differentiate ourselves from human judges with a more precise analysis which determines punches based on each frame by analysing 60 frames in each second of footage. As the human eye can only perceive visual cues in 30-40 frames a second, this gives AccuBoxer a superior perception of the fight.

On top of that, AccuBoxer solves the lack of transparency around decisions with its recording of timestamps for punches landed. This tells the user exactly when AccuBoxer recognises a hit, offering a detailed breakdown of the fight. Unlike traditional boxing scorecards, which only provide a final score without any explanation of when or why points were awarded. This level of detail eliminates the mystery and subjectivity often associated with traditional scoring, giving everyone a clear and objective view of the fight's critical moments.

The functionailty of AccuBoxer includes:

MP4 Uploading Frontend which intakes any boxing video
Frame by frame analysis of video with replay and playback speed
Statistics of each boxers performance based on headshots
Statistics of each boxer based on punches thrown
Timestamps of recognised punches for justification of AI's decision

How we built it

Frontend

ReactJS + JavaScript
HTML/CSS

Backend Python with:

FlaskAPI for web requests and APIs
OpenCV for frame analysis
Moviepy for frame concatenation
Inference for AI Model Pipeline
Roboflow for AI Detection Training

How did you approach the development process for your project, and can you discuss the technical implementation, including the stream-specific technologies you used?

Our journey to revolutionize boxing judging started with a bold vision: to build an AI that could accurately count punches and tackle the frustration of biased scoring. We wanted to restore fairness to the sport and ensure every punch was counted with precision.

We began by exploring Mediapipe, a powerful tool for tracking human poses. Our initial idea was to use Mediapipe to overlay hand and head areas to detect punches. To complement this, we trained a YOLOv7 model with about 400 images to dynamically crop and analyze frames. However, the fast-paced nature of boxing led to blurred poses and unreliable detection, while processing frames in parallel proved too resource-intensive.

Undeterred, we shifted to MoveNet Lightning, which offered multi-pose detection capabilities. This seemed promising given the complexity of boxing matches. Yet, MoveNet often detected irrelevant figures such as referees and spectators, making it difficult to focus solely on the boxers. The rapid movement and frequent disappearance of hands further complicated accurate tracking. Our prototype, available as "Movenet Prototype" on GitHub, highlights these challenges and our iterative improvements. To refine our approach, we adjusted MoveNet’s detection limits defined by our previous YOLOv7 model and focused on key figures: the boxers and the referee. This strategy aimed to reduce noise from extraneous detections, however was still not successful.

Realizing the limitations of these methods, we pivoted to retraining another YOLOv7 model with a dataset of 600 hand-annotated images. Despite the small dataset size, this approach allowed us to accurately distinguish between boxers, gloves, and heads. By processing frames and applying circular overlays, we effectively detected and counted punches. Our custom algorithm filtered boolean outputs to ensure precise counting across frames.

Our YOLOv7 model, trained with fewer images than typically required, showcased the potential of our approach. With tens of thousands of images, we could achieve even greater accuracy. This iterative process and deep dive into various technologies confirmed that our solution not only met but exceeded our initial goals, providing a reliable, unbiased system for boxing judging.

Tell us about why your project is innovative / creative

Our project redefines boxing judging with a groundbreaking approach. Traditional scoring systems, like the 10-9 system, are riddled with subjectivity, leading to inconsistent and often controversial decisions. By contrast, AccuBoxer AI delivers a game-changing solution with its objective, frame-by-frame analysis.

We tackle the inherent flaws in human judgment by scrutinizing every punch with pinpoint accuracy. Our technology goes beyond mere observation—imagine scaling this to multiple camera angles and a larger dataset. The result? A revolutionary system that captures every aspect of the fight, enhancing fairness and providing deeper insights into boxing performance.

AccuBoxer AI isn't just an improvement; it's a leap into the future of boxing, where every match is judged with unrivaled precision and transparency.

Accomplishments that we're proud of

Publishing a Functional Prototype: We successfully built a working prototype of AccuBoxer AI within the tight timeframe of the hackathon. This prototype not only demonstrated the core functionality of our idea but also laid the groundwork for future development.

Ambitious Innovation: We took a complex, ambitious idea and turned it into a reality. By applying cutting-edge AI technology to the world of sports, we created something that has the potential to make a real impact on how boxing and other martial arts are judged. Our collective level of experience with programming paled in comparison to the complexity of our idea, but thanks to our drive to succeed and our genuine passion for our product, AccuBoxer AI successfully came into fruition!

What we learned

Functional Requirement Priority: As this was our first hackathon, each of our team members started with grand plans and high expectations for the functionality of AccuBoxer AI. However, as the reality of the time limit became more apparent, we quickly realised that we had to adjust our goals and prioritise the most essential features.

Recovery and Adaptation: At many points in the project's timeline, AccuBoxer faced performance issues due to the constraints of machine learning libraries as well as our lack of expertise as amateur software engineers. These challenges were significant and sometimes frustrating, but they also became valuable learning experiences as we learnt to transform our frustration and failures into determination. When our initial models struggled with accuracy or speed, we had to pivot quickly, experimenting with different libraries, frameworks, optimisation techniques or thinking of new solutions entirely.

What's next for AccuBoxer AI

Given limited time, AccuBoxer AI was initially trained on boxing to demonstrate its capabilities and impact. However, the technology and methodologies we've developed are highly adaptable and can easily be extended to other martial arts disciplines. This means that the same commitment to fairness, precision, and transparency that revolutionised boxing can be brought to a wide array of martial arts.

AccuBoxer's AI detection model was trained on 600 images of data which took roughly 10 hours of training. In the future this could easily be scaled to 1000, 2000 or even 10000 images making its potential for consistency limitless.

With a reliable model, we see this being implemented in real global boxing matches to be used as a second opinion to the judges. But why stop there? We can run the program again for a third opinion, or again for a fourth opinion! We could even have 100 AccuBoxer's running on one boxing match and take an average of the results! Whatever its future usage may be, we firmly believe that AccuBoxer AI will strongly benefit the martial arts world.

Team & Roles

Jakie focused on video processing and backend development with Python, while also handling AI training. Nathan shared similar responsibilities, working on video and backend aspects with Python, and also contributed to AI training. Finian brought his expertise in video processing, full-stack development, and AI training, using JavaScript and React for the frontend. This combination of skills and technologies enabled us to create a robust and well-rounded system.

What key aspects of your project would you like to highlight that might be overlooked during judging?

For our boxing project, AccuBoxer AI, one key aspect that may be overlooked during judging is the innovation and effort we put into optimizing the project despite the challenges posed by limited resources.

Manual Annotation and Small Dataset Training: We manually annotated all the images for our YOLO models, which is a time-intensive process. However, it allowed us to ensure high accuracy and relevance in our object detection tasks. Additionally, we focused on training with a smaller dataset to demonstrate that our approach is not only effective but also scalable. This showcases the potential of our solution, especially when considering larger datasets in the future.

Customization and Precision in Detection: We faced challenges with fast hand movements during punch detection, which can be especially difficult to track accurately. To address this, we developed custom algorithms designed to handle these complexities. We could also integrate multiple camera angles, with each punch detection becoming more accurate, as the system can cross-reference movements from different perspectives. This multi-angle approach significantly enhances the precision and reliability of our AI, ensuring it consistently delivers accurate punch counts even under the most dynamic conditions.

Scalability and Future Potential: While we've focused on boxing, the framework we've developed can be adapted for other sports and motion analysis tasks. This scalability potential makes our project not just a solution for boxing analysis but a versatile tool for broader applications.

These elements highlight the depth of our project and the careful consideration we've put into overcoming technical challenges to deliver a highly functional and innovative AI solution.

Demo Video of the videoplayer functionality came out laggy due to Zoom Screen Recording. Actual program is much smoother <3