-
A rough example of how the design might look, with pictures from Trondheim, Norway.
Inspiration
A walk across campus looking at all the historical buildings and statues of the University of Pennsylvania.
What it does
The project will be in the form of a web/mobile app. One or several locations are provided to the user. Along with each location an image is provided. The user aims to find the actual location and take a photo that best possibly replicates the image provided. The user has now "unlocked" said location. The images that are intended to be used are historical images of landmarks, which are displayed to the user with a relatively low opacity and some interesting facts about the landmark. Making it both a thrilling geocaching-like and an educational experience (re-live history!).
How we built it
Computer vision, API-handling, and web development will be critical elements of this project. The biggest part of the application was to find a way to accurately decide if two pictures contain the same landmark, thus we spent a fairly big chunk of our development period just trying to make a model that worked as intended. We built the web app by making a Flask backend, where we provided the utility to upload a picture to compare with, as well as implemented our Siamese Neural Network.
Challenges we ran into
As the hackathon format doesn't allow for thoughtful and methodical development of AI, we decided to use a pre-existing Computer Vision model to be able to recognize a place, instead of having to train a model for ourselves. A problem we ran into was that most Open Source Computer Vision models are intended for object detection and classification, rather than comparison between two pictures of the same object. Unable to find a suitable model, we were forced to train our own, settling on a Siamese neural network.
Accomplishments that we're proud of
We're especially proud of providing a proof of concept, with a self-trained Siamese neural network that works as intended. (with the pictures about the same angle)
What we learned
We learned that getting everything to run smoothly and deploying a functioning model is more of a challenge than expected. Our plane also arrived very late, so we definitely learned that it's beneficial to get stated as soon as possible.
What's next for Photo cache
A natural next move would be to make the concept into a functioning app for IOS/Android, and fully implement the more educational part of the application.
A website link should be provided in the GitHub repo.
Log in or sign up for Devpost to join the conversation.