Inspiration
Do you remember what’s in your fridge and why you bought it?
Fridges have remained as a “white box” for centuries, and with IoT and image classification technologies, it has still remained the same despite some unsuccessful attempts like the expensive Samsung Family Hub.
We want to create a cheap IoT camera that’s attached to your ordinary scanner. By recognizing what’s in your fridge, we can remind you about the expiration date, offer recipe advice, and most importantly, advise your daily nutrition.
How we built it/What it does
We created a Node.js server that receives images from a laptop webcam. It then spawns a python process that queries the Google Vision API to determine the type of food that the user wants to put in the fridge. We then used socket.io to push the label to our React web app, where we display the current contents of the fridge and expiry dates to the user. We also have a sqlite database of nutritional information that we could display.
Challenges we ran into
Our project idea was slightly derailed when we couldn't acquire a Raspberry Pi camera at the hackathon (there was a lottery and the organizers ran out) and the Raspberry Pi kit we received was non-functional. We pivoted by deciding to use the laptop webcam instead to create a proof of concept.
We originally planned to use PaddlePaddle and VGG-16 for classification of food. However, PaddlePaddle is not well documented and there's no image classification model that we can use. After switching to Tensorflow, we found VGG-16's architecture is not suitable for identifying the distinctions between our categories.
Accomplishments that we're proud of
Team cooperation. We all got along well together. :)
What we learned
- Bring your own hardware
- Bring your own wifi
- Bring an umbrella
What's next for Smartest FridgeCam
The main thing is to redo this project using Raspberry Pi and the Pi camera instead of the laptop webcam. We plan on developing this project on the side beyond this hackathon because we believe this idea is both feasible to implement and useful for users.
Log in or sign up for Devpost to join the conversation.