Inspiration
As a university student, it isn't easy to find furniture that matches the aesthetic of my room without having to travel to multiple locations and spend large amounts of time, such as IKEA. To conserve cost and time, we have decided to build a customizable interior designer, Decora.
What it does
Using Decora, we have built an application whereby simply inputting an image of your current room, you would get several pieces of furniture that match the colour palette, aesthetic and type you like. In addition, the name, price, and description are included with a direct link to the IKEA app for you to place your order.
How we built it
The website has been built using HTML and CSS for the front end with the support of Bootstrap, and flask (Python) for the back end. The data has been provided by an IkeaAPI that returns furniture (name, colour, price, type) and a transformer-based image processing model called 'VILT' fine-tuned on the 'VQA' dataset that can annotate and serve as a query answerer about the image.
The user inputs the URL of an image that depicts their desired aesthetic preferences and the article of furniture they seek, this image is processed by the VILT model provided by huggingface and annotates and attaches the image with adjectives relating to 3 things ( color, vibe and style of the furniture) based on these 3 criteria we then ask the ikea api to return all catalogue items of that article of furniture and compare the similarity of the image based on the criteria. selecting candidate items that match the general vibe and aesthetic.
Challenges we ran into
Our main challenge was trying to find a successful deep learning model that could annotate the inserted image and could tag it with adjectives for ambience matching and ensure that the model size and prediction time is minimized. However, after understanding the model, we faed an issue with parsing the ikea-api json dump and thus used pydantic models for validation of the necessary data,
Accomplishments that we're proud of
We can use models that can sequentially annotate and compare the ambience, color and the furniture aesthetic of the input (inspirational image) and the items that are available on the catalogue. We were able to synergize full stack web development and complex deep learning architechture.
What we learned
We learnt the main idea of Deep Learning, which managed to not only create metadata but handled the large amounts of metadata present within an API using AI models, alongside the ability to enforce our knowledge of full-stack development.
What's next for Decora
Next up, we will be trying to make use of augmented reality or VR models to project the furniture using ChromeOS which allows you to make use of extensions that help project the selected furniture in your room, as well as keep track of real-time price changes and stock changes by web scrapping other furniture sources.
Log in or sign up for Devpost to join the conversation.