Foodify

Inspiration

After noticing the abundance of unhealthy foods presented to us in daily life, we had the desire to create a product to make it easier for people to track their nutrition and make healthier choices about what they eat.

What it does

Using an app that runs on their phone, the user can take a picture of the food they eat and upload it. One model identifies the primary food present in the image and offers some healthier alternatives to it. Another model classifies the three main categories of food present in the image and uses them to determine whether the meal is balanced.

How we built it

The backend was built using python and flask. The artificial intelligence models are based on the Inception V3 architecture as a starting point. After extensive research, the food11 dataset of 16,643 images was chosen for the task of determining the categories of food as it contained 11 different classes that best represented whether a meal was balanced. The food101 dataset was chosen for the task of identifying the specific food which contained 101 classes with 100 images each for a total of 101,000 images. The two models were then trained on the food11 and food101 datasets, respectively, to be able to perform their desired tasks. Train generators had to be used due to the large size of the datasets. The food identification model ran for 50 epochs and the food categories model ran for 15 epochs. The frontend was built using react native to run as an app on mobile devices so users can use it on the go.

Challenges we ran into

By far, the largest difficulty was getting the frontend to communicate with the backend. We were able to send simple messages but when we tried to send the image to the server, it didn't work. We spent a whole day debugging and fixing the issue.

Accomplishments that we're proud of

We are proud that we were able to successfully create the product that we envisioned that could perform all of its intended functionality.

What we learned

We learned a lot about how artificial intelligence functions and how to combine it with a front end to be used seamlessly.

What's next for Foodify

The performance of the two AI models could be improved so that the predictions for the food and food categories are more accurate. Additionally, we'd like to add more functionality to the app such as a history and more food categorization features.