MagicusHattus

With a sparkle of tech and computer vision tricks at hand, become the master magician you've always wanted to be!

Comment

🧙‍♂️ Inspiration 🧙‍♂️

We were inspired by Snapchat filters in which users' face are overlaid with images. To keep with the magical spirit, we decided to create a filter that puts a magical hat on the user. In addition, we also tried to read the user's mood by detecting their emotion and adding special effects.

🎩 What it does 🎩

We created an application that places a magic hat on a person's head. The person's emotion is also read, and a magical superpower is assigned to them.

How we built it

We utilized MATLAB and the KLT face detection algorithm to detect a person's face. Once a person's face is detected, an overlay of a random hat or headwear is placed on top of the person's head. Our application was deployed using the Gradio library in Python. Through Gradio, the user can take a live picture or video using their webcam or upload a picture or video clip to be processed. Once processed, the image or clip is sent to our MATLAB script. The MATLAB program performs face detection and emotion classification to assign the person a hat and a superpower. Basic understanding of the KLT algorithm for feature extraction was helpful towards building our project.

Challenges we ran into

For MATLAB, KLT and computer vision algorithms were used to develop our project. None of us had experience using the Computer Vision Toolbox in MATLAB, so we spent some time learning about its many features.

When working with the images of hats, we had issues with making the images transparent. The hat images would appear with a black background, which we solved by specifying a transparency variable when reading the image, as well as using some boolean indexing.

For half our team, it was their first time using Gradio. Furthermore, Gradio wasn't working locally on their computers, which we got around by using repl.it. We spent a lot of time looking for a platform that ran the Gradio module. Since it was the first time that some of us used Gradio, some time was spent looking up documentation and testing what the functions display. We ran into a few bugs here and there with getting the webcam up and running, and displaying the output image. We also had trouble trying to find a way to convert the image taken in from Gradio (a nparray type in Python) into a MATLAB readable image input and vice versa, but it was quickly solved thanks to forum posts from the MATLAB community.

🎉 Accomplishments that we're proud of 🎉

We successfully created a filter that puts a random magic hat on the user. In addition, we were impressed with how our emotion classification model performed on our data. The model scored well on our testing set, with an overall classification accuracy of 85%.

📚 What we learned 📚

  • How to create a web interface with Gradio
  • MATLAB's Computer Vision Toolbox
  • Theoretical aspects of how the KLT face detection works
  • Convolutional Neural Networks

What's next for MagicusHattus

  • Gesture recognition: If the user waves their hand, a dove may appear to fly out. Or if the user salutes the camera, a rabbit can jump from the corner of the screen.
  • Manual selection of hats and superpowers: A user can choose which hat and superpower they want to wear instead of complying with our program's random assignment of hats and emotion-driven superpowers.
  • Spirit Detection: It would be quite cool to improve the algorithm to detect spirits and their emotions too. Just a maybe though.

Prize Categories We're Aiming for

  • Best Use of MATLAB: Half of our team members were using MATLAB and computer vision for the first time. We used the KLT face detection model from the Computer Vision Toolbox to find where to place the hats, and trained a convolutional neural network (CNN) in Deep Network Designer to classify facial emotions. To connect the MATLAB programs our Python frontend, we used MATLAB engine. Mixing these ingredients together, we were able to overlay a magic hat and special superpowers onto an image. Poof, just like magic!
  • Master Magician Hack: We can make everyone feel like a magician at the heat of the moment. Just open the app and you'll see your inner magician staring back at you!
  • Best Domain Name from Domain.com: As we have 4 members on our team, we will have 4 domain names. Our submissions are: magicushattus.tech,magicify.tech, magicalme.tech, and bibbidi-bobbidi.tech
  • Dream Big and Create More Cheers with AB InBev: We had a lot of fun creating our hack and we truly hope that users of our hack will have fun too. Our hack empowers people of all ages to have fun. Everyone can be a magician with their own unique superpowers!

Built With

Share this project:

Updates