Edge Royale Analytics (ERA)

Inspiration

Our friend group's trophy diversity is extremely high. This unfortunately let to many fights among the group because 1-2 guys would constantly destroy the other guys. We wanted to change this and give even the most beginner players an edge. With a love and curiosity for machine learning this project was an idea we have been wanting to implement and bring to life

What it does

When you're playing Clash Royale on your computer (via an emulator or screen mirror), this program watches your screen and automatically identifies which cards your opponent is using. It tracks their full deck of 8 cards, shows which 4 cards are currently in their hand, and estimates how much elixir (the resource used to play cards) they have available. This gives you a strategic advantage because you can see what cards they might play next and plan your moves accordingly. The program displays all this information in a clean overlay on your screen while you play, using AI to detect the cards and smart algorithms to predict what your opponent might do next.

How we built it

This project was built using Python with OpenCV for real-time screen capture and video processing, and ONNX Runtime for fast AI inference. The core detection model is a YOLOv8 object detection neural network that was custom trained locally on a dataset of Clash Royale gameplay screenshots (Just us playing clash). The training process involved collecting and manually labeling hundreds of images of each card type (Baby Dragon, Knight, Giant, etc.) using annotation tools (this step took forever), then training the model on a local GPU using PyTorch. Once trained, the model was exported to ONNX format for optimized inference speed. The application uses the MSS library for efficient screen capture, a custom GameState class to track opponent card cycles and elixir count over time. The entire overlay system runs at 8+ FPS using numpy for fast array operations and strategic frame-skipping techniques to balance accuracy with performance.

Challenges we ran into

We ran into a few core hurdles: making the overlay resizable and reliably always-on-top; getting a locally trained model to a solid mAP with limited compute (we trained only 8 cards to ship); stabilizing detection boxes (NMS/threshold tuning, ROI/alignment issues); and low FPS from the ONNX inference path, which made the UI feel laggy. Other hurdles were capture-model mismatches (color space, resize, aspect ratio), timing/sync between frames, detections, and GameState (e.g., elixir ticking even when no detections fire), screen-capture latency and variable FPS, dataset quality (label noise, class imbalance, weak augmentation), keeping train/infer preprocessing identical, provider/driver quirks (CUDA/DirectML, VRAM), small memory leaks, config/reproducibility, and the pain of packaging a transparent overlay consistently across OSes.

Accomplishments that we're proud of

In under 24 hours, we trained a model and wired it end-to-end: screen capture feeds the ONNX engine, detections update GameState, and a clean overlay renders real-time cues. The GameState logic—elixir, current hand, and cycle—works reliably, and we shipped a playable MVP even with only eight cards trained. It’s demo-ready, easy to iterate on, and gives us a solid base to expand the card set and tune FPS.

What we learned

Throughout this project, we gained valuable hands-on experience in several key areas of machine learning. We learned the fundamentals of training models, from data preprocessing to model evaluation, and developed a deeper understanding of the complete machine learning workflow. Additionally, we honed our collaborative skills by working effectively as a team, learning how to delegate tasks appropriately, coordinate our efforts, and leverage each team member's strengths to successfully complete a complex project together.