Bose Frames Preview

GIF
Program attaching the Bose Frames on the website

Inspiration

With the closing of 119 brick and mortar stores, there are less chances for users to get an interactive experience with the Bose Frames before buying them. This project aims to help people make informed decisions before purchasing based on how the Frames look on them, directly on the website.

What it does

Our program uses the webcam to provide a virtual experience of the Bose Frames.

How I built it

This project was built in python with openCV, dlib, and flask. The live video feed comes from the webcam and each frame is processed with openCV and then dlib in order to detect faces and identify the 68 landmark points on each face. Then the locations of specific points corresponding to the eyes and nose are used to overlay the png image of the Bose Frames onto the face and this new frame is sent to the website that is being hosted with flask.

Challenges I ran into

Originally, the plan was to make an AR experience so the user could have a 3D preview of the frames, but unfortunately this did not work because the 3D model of the Bose Frames seemed to be incompatible with the DeepAR SDK we were using. Therefore, we decided to use openCV and dlib instead.

Accomplishments that I'm proud of

We were able to accomplish a decently working program in the end that does follow the face and accurately place the frames over its eyes.

What I learned

In this project, we learned a lot about AR and why it is difficult to import your own models since they require specific file formats and types. Additionally, we learned about openCV and dlib which are the heart of the face landmark detection. Therefore we had to use image processing and learned about the matrix arithmetic necessary to apply masks and more. We also learned about flask and how to host a basic test website on the local machine which made it much easier to use the written python script and incorporate it into the website.