myPantry

• 

  Login Screen

• 

  Join a House

• 

  Pantry

Inspiration

According to the EPA, over 30% of the food produced in the US is not eaten. This results in many detrimental environmental impacts, including greenhouse gas emissions, excessive fertilizer usage to grow food, and a large percentage of this food waste accumulating in landfills (nearly 20%). We asked ourselves, "What can we do to stop ourselves and the people in our community from contributing to this problem ?".

One of the biggest things we noticed in our day-to-day lives, both in college and back home, was that many of the groceries bought by roommates and family members were going unused before expiring. Our app, myPantry, aims to solve this problem by allowing people living together to easily see the foods still available in their houses so they can make an effort to use them before they go to waste.

What it does

myPantry allows users to join “homes,” or groups of people, whether they are family members or roommates living in the same home. It then allows them to upload their grocery receipts, which the app then parses through to extract ingredients using Apple's Vision computer vision framework. It also estimates their expiration dates using Google Gemini and then stores this info in a shared database accessible to everyone in the house. In addition, myPantry also displays recipes for dishes using ingredients set to expire soon to prevent food from being wasted.

How we built it

We used Swift and SwiftUI to develop the iOS mobile application and Firebase for user authentication and to store" house" and "grocery data. To read grocery receipts, we used the Apple Vision computer vision framework. Finally, we used Google Gemini's API to estimate expiration dates and recipes that can be made from the items set to expire soon based on the ingredients extracted from the grocery receipt.

Challenges we ran into

One of the big challenges we ran into was implementing a functioning camera with a shutter button that could capture, save, and read images of grocery receipts. In addition, we had some challenges using the Apple Vision computer Framework to read grocery receipts and extrapolate the right information from them.

Accomplishments that we're proud of

We’re really proud that we were able to build a functioning app in the allotted timeframe. Before the hackathon, we were a little worried because neither of us had much Computer Vision experience, nor did I (Abhinav) have too much experience with Swift.

What we learned

We learned how to use git to work on a project together while making progress, use SwiftUI to create a functioning iOS app, store data in a database effectively, do prompt engineering, and navigate the Apple Vision computer vision framework.

What's next for myPantry

In the future, we plan to add more functionality to the home page showing the number of days till the expiration date and an easier way to remove ingredients from the database once someone in the house has used them.

Built With

• avphotocapture
• firebase-auth
• firestore
• google-generative-ai
• swift
• swift-computer-vision