ScanCare

Inspiration

ScanCare is a skincare web app that allows users to upload pictures of ingredient lists from their skincare products or cosmetic products and our app gives personalized recommendations by analyzing the ingredient list. My inspiration for this project was my family's history of eczema and how that condition has made finding products tha work for my skin type difficult. Along with this, I have noticed that ingredient lists are often confusing, overwhelming, and difficult to understand.

What it does

ScanCare allows users to take a short quiz about their skin. Then it gives them the option to ypload a photo of a products ingredient list from skincare or makeup. Then the app will use AI powered recommendations to give a personalized breakdown of the ingredient list based on your skin profile.

How we built it

The frontend utilizes React, TypeScript, and Tailwind CSS. The backend uses Node.js and Express server to handle AI inference requests. I used Tesseract.js for the ingredient extraction from images and an AI Model from Hugging Face to provide recommendations for the user. Along with this, I used Hugging Face Inference API.

Challenges we ran into

A challenge I ran into is that initially, when uploading images the extraction was difficult as it was extracting text that was not part of the ingredient list. I was able to fix this by consulting the mentors at the mentor table to clean my extracted data prior to displaying it to the user. Another issue was finding an AI API model that could provide me with a proper analysis as I had to purchase credits to utilize many platforms.

Accomplishments that we're proud of

Implementing the scanner portion of the project that extracts text from images uploading successfully.

What we learned

I learned how to implement an API into my code using vision models.

What's next for ScanCare

Some other features I plan to implement is a feature that lets the user track what products caused a reaction, flareup, or breakout and the AI recommends them to avoid the product in the future. Also I want to add a feature that gives a full breakdown of the ingredient list.

Built With

• css
• html
• node.js
• note.js
• react
• tesseract.js
• typescript