Did You Forget?

Flow chart for program organization

Inspiration

The inspiration is of a personal origin. Often times we go out to the store to purchase some items, but during the shopping experience we forget the one item we left the house for. If there was a way to be notified that we forgot an item before we leave the store, it would save us from this situation.

What it does

The program is composed of two parts: a loop that monitors customers as they enter and a loop monitoring customers as they check out. As customers walk into the store they are identified. A profile displaying their commonly bought and/or forgotten items is shown. If there's not a facial profile create an empty one and display a default profile compiled from all customers.

The second part monitors customers as they check out. When a unique face is identified at the register the profile associated with it is loaded. The service then checks on a regular basis (<1 sec). When a purchase is made at the register it is associated with the face at the register during the time period of the transaction. If the profile is empty all of the purchase history of the customer is requested from the NCR API and a profile is built. Collaborative filtering is used to analyze the most recent transaction and compare it to the customer's history. If there is a high correlation between items recently purchased and an item that wasn't purchased, it is recommended that the user buy this item.

How we built it

We used Postman to test the NCR API calls, and Flask to implement them in python. We used the webcam on a laptop and OpenCV library to implement customer identification. Collaborative filtering was accomplished with the lightfm.

Challenges we ran into

The first challenge the team encountered was understanding the scope of the problem at hand. Once the team finally had settled on an idea and began investigating it initially appeared feasible to implement a small scale version of our idea. After digging in we ran into authentication problems with NCR's web request. The longer the hack went on the more problems we realized we were having with implementing pretty much every feature. Detecting that a face is on camera with OpenCV is trivial, uniquely identifying one is not. To be able to do so requires a large database of known faces.

Accomplishments that we're proud of

We're proud that we were able to come up with an interesting idea. We feel it's something that could possibly help customers and businesses. We're proud to have met each other and learned more about working as a group.

What we learned

During the hack we learned the basics of Postman and how to use Flask to implement routes once you understand the syntax of an API. We learned a little more about using github and pushing changes.

What's next

While we didn't fully implement our idea as laid out above, along the way we side stepped features that could certainly add functionality or provide better matches by correlating with other data. For example we could measure the time a person spend in the store by using the facial id to calculate the time from when they entered to when they checked out. Another thing we considered was adding a frequency of purchase measurement. That is if more than the average time between buying an item has passed increase the match rating of that item. (For example if you buy toilet paper every three weeks, and it has been four weeks since your last purchase. If you check out without toilet paper an alert would be given.