DecoAR

Homescreen
Scanning Screen
Scanning
Screen when placing object
Placing, Rotating and Moving Object

Inspiration

As college freshmen, while moving into our dorm rooms, we faced a common moving struggle: finding furniture to fit our ideal room aesthetic while balancing practical factors like cost and size. To solve this issue, we decided to create deco.ar, an iOS app that searches stores based on your interests, provides cost analysis to find the best deals, and uses AR technology to display the furniture inside your room.

What it does

Our application uses AR technology to decorate rooms, finding furniture online using user guidance and giving an accurate representation of how it could look in a room. It saves all the furniture a user looks at, allowing them to go back and purchase anything they may find appealing.

How we built it

We built deco.ar by coding a Swift frontend in XCode and a Python backend with a flask server using Google Cloud. Our two most utilized dependencies were ARKit and RealityKit. ARKit allowed us to make a 3D USDZ file of whatever surface was being scanned (in our trials, we scanned in dorm rooms, lobbies, and study rooms). This file was sent to the server (along with user-provided furniture suggestions that were analyzed using Gemini), where the python web scraper finds furniture from various retailers. We then created 3D models from the 2D furniture images and dimension information by using Trellis by Microsoft. The USDZ file of the specific furniture item is returned to the Swift frontend and is displayed through Apple's RealityKit inside the user's room.

Challenges we ran into

We experimented heavily with new software to grow during this hackathon, but it also pushed us tremendously. None of us had ever used ARKit or RealityKit in a full project; however, after brainstorming the idea of demo.ar, we decided to integrate them into the project. This took hours of learning and debugging; specifically, displaying USDZ files of the individual furniture items required us to request upgrades for the server, reset the server multiple time, and also adapting both the current frontend and backend code to adhere to the new software.

Accomplishments that we're proud of

We are especially proud of successfully integrating augmented reality into our application to solve a real-world problem many consumers go through on the daily. Current furniture visualization does not account for all products and with AR as an emerging technology with the potential to fundamentally transform how users interact with products we believe we can truly transform the furniture market. This creates a more immersive and informed decision-making experience, reducing uncertainty when purchasing furniture. Beyond the technical implementation, we are proud that our solution demonstrates how AR can bridge the gap between digital browsing and real-world interaction, providing meaningful value to consumers and highlighting the future potential of this application.

What we learned

During the hackathon, we gained hands-on experience across multiple new languages, technologies and development workflows. We learned how to develop native iOS applications using Swift, including managing UI, state, and device capabilities. We also learned how to integrate augmented reality features using Apple’s AR frameworks, which required understanding spatial positioning, object placement, and real-time rendering. Additionally, we learned how to design and connect a backend system using Flask, and how to establish communication between the backend and the Swift frontend through APIs. This taught us important concepts related to full-stack architecture, including data flow, serialization, and asynchronous communication. Overall, the hackathon strengthened our ability to quickly learn new technologies, work across the full technology stack, and build a complete, functional product under time constraints.

What's next for deco.ar

Our next steps are to deploy deco.ar onto the iOS App Store and to provide Android integration. After that, our goal is to expand into Virtual Reality, using the same technology in the current version of deco.ar to make a more immersive version using VR. By achieving these milestones, are goal is to transform the current market of furniture shopping by introducing modern tech (AR, VR, etc.) in a way that makes the experience more fun and more efficient for consumers.