ScanBites

Food Inventory
Scan Ingredients/Nutrition Label
Nutrition Details
Health Results
Health Rating
Chatbot
More Health Results

Inspiration

As college students, we often struggle with balancing a busy schedule, coursework, and maintaining a healthy lifestyle. Cooking meals from fresh produce isn’t always an option due to time constraints, and constantly eating out can be too expensive. So, we end up turning to pre-packaged or processed foods. However, it's difficult to tell which processed foods are safe and healthy. With nutrition goals in mind, we wanted to create a solution that would help students make informed food choices effortlessly while saving time and reaching their dietary goals.

What it does

ScanBites is a mobile app that makes it easy to analyze the nutritional quality of prepackaged food. Users simply take a photo of the nutrition label, ingredients list, or the food itself, and ScanBites processes the data, identifying potentially harmful ingredients and assessing the overall healthiness of the product. The app uses AI and image recognition to break down calories, proteins, sugars, fats, and other nutrients, offering a health score based on user goals and dietary restrictions.

How we built it

Utilized Flutter Framework for the app development. Next.js for backend. The flow is as follows:

The user uploads some pictures of a food package.
We use Google ML Kit to perform OCR text extraction on the images.
From Flutter, we make API requests to our backend, build on Next.js API routes.
In the backend, we feed the payload into our carefully engineered prompts for Cohere’s new command-r-plus model to fill in the required information in JSON format.
The returned JSON response is parsed and displayed in a fun, gamified interface.

Challenges we ran into

Flutter was a new piece of technology for some of our team members, so the learning curve posed some unique challenges for us. Leveraging the navigation and component system turned out to be more challenging than we had anticipated.
More commonly expected with LLMs, the prompts and responses we worked with tended to take upwards of 10 seconds to generate, which exceeded Vercel’s free limits. We found clever ways to reduce the generation time by polishing our prompts and input, as well as a workaround to host the API locally through a LAN network for the weekend.
We encountered many conflicting opinions when it came to the UI/UX design of the app. We wanted to keep the UI design relatively consistent throughout the app, but found that this heavily clashed with the user experience and new user-friendliness of the application, and thus we had to find a way to strike a balance between the two.

Accomplishments that we're proud of

Designed all the graphics and ui from pixel art
WE FINISHED EVERYTHING WE SET OUT TO DO
Worked at a comfortable and productive pace, which gave us more leeway during the life-threatening merge conflicts we encountered (and resolved!)