Fairest Match

Inspiration

Personality quizzes

What it does

Our work searches for the most similar image using a Neural Network and a cosine similarity score to match images.

How we built it

For the image processing, we used a prebuilt model of ResNet50 in a convolutional Neural Network in Pytorch as a feature extractor in the cosine similarity function. The image is strung to a 244x244 image and passed through the model to find the features and outputs a vector of features. The features are then compared using a cosine similarity function, and the result is used to see which images are closest.

Challenges we ran into

We originally ran into problems with getting the TensorFlow to work since we didn’t have C++ build tools on our laptops. Thus, we pivoted to the PyTorch library for our primary Neural network needs. We then resulted in using ResNet50 and a CNN to score the similarity between our images, but it would have been better if we could have used a facial recognition module like DeepFace to better our models. We also scored the images on the fly rather than building a model, as it worked best with our limited time capacity and fast development. In future iterations, we would most likely look towards build a full model that is preprocessed and a more expansive dataset to train off of.

We were looking to use a fine-tuned LLM trained on detecting character personalities, but all the existing models on LLMs had issues. Some were trained to detect with longer blocks of text (like full novels), which made it difficult with the typical short dialogue format of Disney princess lines.