Magic Mirror

Mirror mirror on the wall, is my outfit the best of them all?

Comment

Inspiration

Our inspiration stems from our own struggles with choosing clothes that we feel good in for a day of classes or productive work. As students who can sometimes procrastinate in the morning to get that extra bit of sleep, we value being able to choose our outfits faster and better, maybe even helping us get to class on time. Our biggest integration of the OpenCV video streaming was inspired by our struggles with online shopping and general decision making with clothing (i.e. not being able to visualize clothes on you or paired with a top/bottom, seeing what colors look good on OUR skin or features).

What it does

Our application allows users to create their wardrobe online by submitting images of their clothing. After they've submitted their wardrobe, the data is saved and can be used later on the "Try it on" page, where you will be able to shuffle clothing pairings. These pairings are then evaluated by our AI model, which will also give you a quick message about the outfits' overall rating. And most importantly, you can see the shuffled clothing on you using your camera!

How we built it

We built our project using Flask to connect our Python code, which uses OpenCV and SQL, to our HTML and CSS code on the front end. We used Tensorflow to train the AI model, specifically using keras to create a Convolutional Neural Network (CNN) for image recognition and prediction. We also used datagen to process the images and numpy as well as Adam to optimize the performance of the AI model.

Challenges we ran into

Clothing Overlay: It was challenging to get the virtual clothing to align accurately with the user standing in front of the camera. We had to constantly refine how the shirts and pants were positioned on the person to prevent glitches or misalignment. Another major hurdle was ensuring the camera was tracking the user without quitting or freezing mid-use. We explored multiple options and implemented a reliable method to reload the video each time.

Training an Accurate AI Model: Another major challenge was ensuring our AI model correctly classified the various clothing from the user’s personal wardrobe as “good” or “bad”. Gathering enough diverse data was difficult, and we had to carefully curate and label images for accuracy. Merging our trained model with our system also posed challenges, and required us to fine-tune it constantly to work. Limited Experience with OpenCV: Since our group was relatively new to using OpenCV, we faced a few difficulties initially to understand how to use it effectively for image processing and pose tracking. We had to spend a little extra time researching and resolving technical issues, such as inaccuracies in detecting body positions and integrating our AI model with OpenCV’s tracking functions.

Accomplishments that we're proud of

We are most proud of learning how to create and fix the overlaying of the clothes on the live video feed. Additionally, we are proud we were able to shuffle clothing and have it updated as the pairs change. Additionally, this was our first time training an AI model! We learned a lot about neural networks and optimization. We are also proud of overcoming all of our challenges!

What we learned

We came into this project with very limited knowledge of OpenCV and AI model training, allowing us to learn how to use OpenCV and AI model training to help us create our project. Additionally, this project helped most of us learn Flask and TensorFlow! Our biggest takeaway is being able to practice using GitHib and Git, understanding how branches work and committing to main. We reinforced the important of fetching from main making sure our branches are organized so that auto-merge can be done easily. A good amount of our issues came from making small mistake using Git. This project reinforced our skills and helped us learn from our mistakes!

What's next for Magic Mirror

We have so many cool ideas for the development of Magic Mirror! Here are some ideas we had in mind:

  1. Changes in AI - Rather than our AI rating your random outfit, ideally, we want the AI itself to recommend only good pairs of clothing.
  2. Weather - We would like to see the weather of the user's location be integrated into the AI's choices by suggesting extra layers of clothing to wear for colder weather or more appropriate summer clothing for warmer weather.
  3. Removing Background - Right now, the program works for images that have transparent backgrounds, which we were able to do easily using the iPhone's sticker feature. In the future we would like to integrate a process where we will be able to automatically remove the background from a photo of a piece of clothing.
  4. Additional Features - Users should be allowed to mark down what they've worn for a day so that according to their preferences, clothing pieces are not suggested again. For example, an option for "No Repeats" would be 4 days where if clothing was worn, it will not be suggested for the next 4 days.
  5. Scaling - Currently, we are using a limited dataset to train. We would like to use a larger dataset with different "good" and "bad" shirt/pant options. Also, currently the only pieces of clothing that can be used are shirts and pants. So we would like to expand on the clothing items to add things like jackets, shorts, etc.
Share this project:

Updates