PostAR

Posters! Everyone loves them. They express who you are and what you like. But, instead of tons of physical posters, you can immersive your whole room in a new world of digital content with AR.

Comment

Inspiration

One of my friends is a very artistic type of person, and because of this, he has a ton of posters in his room. From his favorite bands to the most obsecure art pieces, they really bring his room alive and give it a ton of personality. On the other hand, my room is unforunately as devoid of life as Dune's Arrakis. To fix this, one would typically buy physical posters and simply hang them up, but this would result in a ton of posters eventually finding themselves to the nearest trashcan. Instead, what if we had a way of having interchangable, dynamic digital content in our own rooms? With the explosion of Augmented Reality (AR) and Virutal Reality (VR) consumer experiences, this idea was something I could possibly achieve within the hackathon's time-window.

What it does

During this hackathon, I created a pipeline for taking in data from a cheap camera that is mounted on a cheap Google Cardboard headset, projecting digital content on to the RGB scene, and relaying it to phone for display in the VR headset.

Diagram of Pipeline: Pipeline of the entire project

How we built it

I utilized various image processign techniques to each a reliable projection method. The first component would be the HSV-based segmentation method, which is a fairly simple method of segmentation. The next components are simply to reduce the complexity of the features from the segmentation masks and a method to find a rectangle from the reduced mask. From then, it is just a simple distortion and split screen filter to create a VR-like view. Since all of this is happening on the laptop, we would want to stream it to the display (a phone) via MJPEG streaming, which just uses a HTML webpage.

Challenges we ran into

The biggest challenge had to be finding a way of generating a quadaliteral, which could have given me more information on the prespective of the projection screen. With this, we could have orientated the image in more realistic method as well as orienting 3D objects.

Accomplishments that we're proud of

I'm very proud of the entire pipeline presented here. It was a difficult feat to pull off within a day or two. I am especially proud of the image processing portions, which I had a lot of fun tinkering around with.

What we learned

I learned a lot about image processing techniques, especially HSV Segmentation. I also learned a ton about video streaming methods such as RTSP (an unused method) and MJPEG streaming.

What's next for PostAR

I would like to move the processing that is happening from the laptop to the embedded chip within the OAK-D. Additionally, a simple quadliteral regression neural network could generate fairly accurate quadilateral outputs.

Built With

Share this project:

Updates