CourtVision

User Info Page
Hoop Detection Page
Shot Statistics Page
Data Augmentation with Stable Diffusion

Dark Blue with Space Photos Movie Ticket-4

Inspiration

The fastest way to improve at anything is to keep track of every single metric you possibly can. In basketball, professional players have multiple coaches who track how a player performs in order to find ways they can improve. How can we democratize this? This is where CourtVision comes into the picture.

What it does

CourtVision keeps track of multiple shots’ speed, release angle, arc, and score using two locally stored computer vision models: Shot Classifier and Hoop Detector. The user can either use real time feed or upload a preexisting video. After the user has completed a set number of shots, the app will output a summary of all the shot arcs, average speed, average release angle, and total shots that went in.

How we built it

UI/UX Design

We used swift to get all the different view controllers and many of the packages to create our more advanced features. One helpful package was the game state manager. This allowed us to switch between inactive and different tracking stages.

Backend Development

Screen Shot 2023-04-16 at 10 37 28 AM

Data Pre-Processing

We collected data ourselves, online datasets, and also stable diffusion. While collecting data, we found a cool use case for Stable diffusion. Generate data variants that you aren’t able to find otherwise. In our case, this was generating pictures of different types of hoops from multiple angles. For the hoop detection model, we used Roboflow to annotate the hoops by drawing bounding boxes in our dataset before feeding it into our model. For the shot classifier, we had to collect the data ourselves since there was no dataset online nor usable videos. We went to the Activity Center and recorded hundreds of shots with different people. Then using veed.io we split up the videos into shots and not shots to pass into the model.

Data Visualization

gif text

Challenges we ran into

Parameters

Training the models was tedious. The parameters had to be constantly fine tuned in order to achieve the final result. We eventually set the number of iterations to 300 and split the training/validation set into a 7:3 ratio. We were able to evaluate the dataset using a testing dataset which was 1% of the training dataset.

Integrating models with Swift

Once we had trained the model, we had trouble getting data from the model in a format that the app could read and use. Eventually, we figured it out by using some of the different packages that swift has. We also wanted to use Amazon AWS to store data and login information on the cloud to create different users. However, importing the package into Swift messed up the XML files in our Info.plist, which corrupted our project. We were urged to restart our entire project as a result at 12am in the morning.

What we learned

We learned how to use the packages of Create ML to get the most data out of our model. We also learned how to use these packages to integrate the models into the app. We are not really familiar with swift, and so doing this project gave us a hands on experience with Swift as well as learning how to incorporate machine learning models into the project.

What's next for CourtVision

We want to build for an individual that wants to improve their game and doesn’t have a coach to help with it. Next steps would be offering drills such as instructing a player to shoot from different locations and also output ways for them to improve. Later, we want to be able to track multiple players in-game. Then, we would create a leaderboard for players to compete against each other based on multiple metrics which we’ll use to compute the players score. The best way to test a player’s strengths and weaknesses is in a full court 5v5 game. We believe that will take our product to the next level.