Your GF's Lipsticks

Online Lipstick Preview Website. Pick a holiday present for your GF!

Comment

Vertical

Consumer

Inspiration

Having a girlfriend/boyfriend and trying to maintain that relationship isn't easy at all. You run into numerous different difficulties. One of which deals with a particular merchandise – lipsticks. There are a wide variety of lipstick colors to choose from, and naturally, you would want to try them on... However, due to the current pandemic situation, it is getting really hard to actually try on lipsticks at your local stores. So, we made this platform for those who have trouble buying gifts for gfs and girls who don't want to go to local stores to shop! This project not only better connects couples and helps them cope with difficult situations, it helps sellers sell more because customers are more likely to buy a lipstick after trying it.

What it does

  • A robust database of different lipstick colors to choose from (We have all the lipsticks you want). We also included price tags if you are looking for a purchase.
  • Digitally apply the lipsticks you chose to your digital face (your photo)
  • An online platform for you to register and apply digital makeup anywhere you want!

How we built it

We used Python, PHP, HTML/CSS with MySQL to build this online platform. MySQL takes care of the user accounts and their personal details for the website. It stores the essential data that we need to make the website fully functional. PHP is utilized as a bridge between the frontend, the database, and the AI backend. We used Python to build the AI digital makeup backend that is the core of this project. A Pre-trained Deep-Learning model is used to recognize the edges of the lips in submitted photos, and a custom filter we made ourselves was applied to the makeup, and we used Flask framework to communicate with the PHP backend(we choose to use PHP as a bridge because PHP has written code to fetch user photo and validate signin cookies). The custom filter was designed carefully to reflect the true ambient lightings of the input image by using HSV transformations while not making the after-image to "watery" - in the final product, the lipstick color looks just like it was an actual lipstick applied on the person's lip!

# Find facial features using face_rec library
faceImageObj = Image.fromarray(faceImage)
faceImageObj = faceImageObj.convert('RGBA')
faceImage4L = np.array(faceImageObj)

face_features = face_recognition.face_landmarks(faceImage)

# "Drawing" the makeup to the facial features
widthToDraw = 2
totalColorExisted = 0
for i in range(1,widthToDraw+1):
    j = widthToDraw - i + 1
    lineAlpha = round(a * (j / widthToDraw))
    realAlpha = lineAlpha #lineAlpha - totalColorExisted
    d.line(ff["top_lip"], fill=(r, g, b, realAlpha), width=j)
    d.line(ff["bottom_lip"], fill=(r, g, b, realAlpha), width=j)
    totalColorExisted += realAlpha

d.polygon(ff["top_lip"], fill=(r, g, b, a))
d.polygon(ff["bottom_lip"], fill=(r, g, b, a))

# In the def makeup_burn function (which is the filter function), we called the colorsys library to use the rgb to hsv function
hsvMask = colorsys.rgb_to_hsv(maskRGB[0],maskRGB[1],maskRGB[2])
hsvImage = colorsys.rgb_to_hsv(imageRGB[0],imageRGB[1],imageRGB[2])
newPixelHSV = (hsvMask[0],hsvMask[1],(hsvImage[2] * 2.0 + hsvMask[2]) / 3.0)
newPixel = colorsys.hsv_to_rgb(newPixelHSV[0],newPixelHSV[1],newPixelHSV[2])
# we used this to apply our own filter

Repo for the Python Program

Repo for the Web

Challenges we ran into

In this project, we faced problems finding databases with various brands and sets. As different brands have their own descriptions and names, we need to unify their colors to RGB numbers. Besides that, the filter was not easy to code. We tried using a lot of the different libraries like blend_modes to make the lipstick applied to look more realistic and natural. However, most of the premade filters produce undesirable results. So, we decided to make one ourselves that converts RGB to HSV. HSV can keep the authenticity of the lipstick color, while still keeping the brightness details of the orginal image. We designed a special weight pair for original image's brightness and mask's brightness to achieve the most realistic effect.

Accomplishments that we're proud of

Designing a comprehensive website to directly solve the needy problem. Applying accurate face recognition with colors on lips, just like what people do in the real life. Involving databases with huge samples. Ranging from products with serval dollars to luxuries to meet different needs. Having the potential to involve machine learning, artificial intelligence, and other up-to-date technology.

What we learned

Built the website with a clean and neat appearance. Merged databases with the website to interact. Brainstormed ideas. Learned programming, co-working, and presentation skills.

What's next for Your GF's Lipsticks

  1. a shopping list feature. (Actually, we already started this, but there isn't enough time for us to finish this feature) In this way, users can buy lipsticks using our website with ease.
  2. we could implement some form of data collection technique using automated machine learning to include EVERY SINGLE lipstick color that exists online, and link that color to its distributor. We can also use artificial intelligence to training our website to recommend suitable products base on your interests.
  3. Change Backend to Restful API and detach frontend from backend. (This was not achieved because of time limit)

Note

Due to Heroku Bandwidth Limit and Deployment Settings, Color Changing Might result in NETWORK_ERROR. You can setup a local instance by visiting our github, and then each "trial" of lipstick should only take 3-10 seconds to load, depending on your CPU.

Built With

Share this project:

Updates