Inspiration
Our inspiration comes from the Black Mirror winter special. People gain the ability to block other people in real life, which results in the targets being removed from their vision, replaced by a grey silhouette of a person.
What it does
The program takes a live video stream, tries to recognise people, and greys out people who have been identified from the blocked list.
How we built it
We built it using Python, using a library called ultralytics (with the YOLO model) for finding the outlines of people and a library called face_recognition for recognising faces based on sample images.
Challenges we ran into
Certain models we tried originally were too slow for real time processing. After trying out some other models, we were able to find a replacement for body segmentation that run much faster.
However, our face recognition model was still causing a low frame-rate. We tried to work around the cost of the model by using multiple cores, however, due to Python's Global Interpreter Lock, it was not possible to do this without causing further issues. We had to instead sacrifice the accuracy of the model to give a more acceptable framerate.
Accomplishments that we're proud of
We managed to get the visual effect to look pretty close to the show, notice the black outline and pixelated inside on a blocked out person.
What we learned
What's next for White Christmas
If we were to improve the project, we could rewrite it in C++ to avoid some of the python specific performance issues. Alternatively, we could wait for libraries to support Python 3.13 so that we can try to run Python without the GIL and potentially have much better performance.
With more time, we could also try out even more models to see if we can make the face recognition more accurate. Running the program on a VR headset would also make it more immersive and closer to the show.
However, we hope that this won't find a use outside of a sci-fi hackathon!
Log in or sign up for Devpost to join the conversation.