Chameleon

Generated #1
Original #1
Generated #2
Original #2

Inspiration

At first, we focused on the e-commerce industry. There are many online vendors who could benefit from an aesthetic and themed website that matches their products and brands. We thought that we could build a tool for Shopify that would allow their customers to input some images of their products and have AI-generated frontend code that creates a pretty online shop for them that presents their products in the best way. We then realized that this could be made more customizable for users if we allowed inputting images of design ideas rather than products and generating a frontend page that matches the user’s design aesthetics.

Our final goal: Make designing aesthetic websites/online shops more accessible to everyone and not only those who possess technical skills.

What it does

Chameleon simplifies the process of designing aesthetic websites by eliminating the need for technical skills. Users can submit images of their design ideas, and Chameleon generates beautiful, customized frontend pages that match their design aesthetics.

How we built it

The application has four high-level steps: image upload, JSON generation, JSON to React component translation, and customization. In the image upload step, the user provides a sketch or screenshot of the design that they desire. During JSON generation, the image is translated to a JSON structure with a well-defined schema that describes the layout of the view. The backend returns this JSON to the frontend, where the JSON is converted into a tree of React components and rendered for the user. Finally, in the customization step, the user can iteratively adjust the design using natural language.

FastAPI was used for the backend, with the OpenAI SDK and GPT 4o used for image and text ingestion, as well as Structured Outputs. React was used for the UI.

Challenges we ran into

ChatGPT struggled to interpret the true size and format of images in design as well as reading in whitespace which impacted its ability to render visual elements accurately. Additionally, ChatGPT faced difficulties in generating JSON that could be effectively converted into React components even though it has no problem generating React code…

Accomplishments that we're proud of

We are proud of ourselves for developing a fully integrated product that brings our MVP features—and more—to life:

Users can upload any image and simply click submit.
We designed a specific JSON template for ChatGPT to follow.
We created a recursive TypeScript script that processes ChatGPT’s JSON output into frontend code.
We successfully rendered frontend UI for basic React components.

What we learned

We learned the importance of task delegation. Dividing tasks among team members allowed us to tackle different aspects of the project more efficiently and leverage each team member's strengths. Additionally, we now know that integrating components early in the development process is extremely recommended. We discovered that unforeseen issues often arise, and resolving these can be more time-consuming than anticipated.

What's next for Chameleon

Adding support for exporting designs to Shopify so that users can further continue customizing their designs on Shopify.
Supporting all of the HTML tags with more complex JSON structures.
Enabling the creation of multiple pages of the website rather than one.
Enhancing functionality with interactive elements such as forms.