🧬 Inspiration
In modern web design, analysts are often stuck in a slow loop of manual A/B testing to find the optimal layout. We realized that by combining LLM-driven agency with predictive modeling, we could collapse this process. We envisioned a world where AI agents don't just suggest designs, but actually "live" in them as synthetic users, allowing designers to simulate months of traffic in seconds to predict performance before a single line of production code is changed.
🚀 What it does
Darwin is an AI-powered design ecosystem that transforms flat web layouts into dynamic, 3D data visualizations.
Agentic Design: Uses Gemini agents to act as "Designers" that generate diverse A/B test variations and "Users" that simulate intent-driven interactions (following F-patterns).
Predictive 3D Heatmaps: A custom PyTorch neural network analyzes these synthetic interactions to predict click-through rates (hits) for any element based on its CSS attributes and coordinates.
Immersive Visualization: Designers can step into a 3D Three.js environment where "Swarm" particles represent users. These users are physically drawn toward elements with higher predicted engagement, turning abstract analytics into a living, breathing crowd simulation.
Full-Stack Integration: Users can edit code, analyze real-time data, and push updates directly to GitHub and Firebase without leaving the platform.
🛠️ How we built it
Frontend: Built with React and Tailwind CSS. The 3D engine uses Three.js and Post-processing (Bloom/Vignette) to create a high-fidelity data scene.
Backend & AI:
- Gemini API: Powers a multi-agent workflow which creates JSX variations and simulates user behavior.
- Neural Network: A PyTorch regression model trained on vectorized CSS data (colors, positioning, measurements) to forecast user "hits".
- FastAPI: Serves as the bridge, allowing the 3D frontend to query the model for real-time hit predictions.
🧠 Challenges we ran into
Agent Reliability: We found that LLMs can sometimes skip complex CSS constraints and agent instructions. We solved this by breaking the workflow into smaller, discrete phases (Layout vs. Design) and implementing a strict vectorize_css parser to validate agent output. We also supplemented the agent's data by performing additional cleaning and sampling on it.
System Synergy: Syncing a 3D physics-based swarm with a Python-based machine learning model required careful state management to ensure that as you drag an element in 3D, the "users" react instantly to the new predicted data.
🏆 Accomplishments that we're proud of
Synthetic Data Pipeline: Successfully training a functional neural network entirely on data generated by Gemini agents.
Innovative UX: Moving beyond boring 2D charts to a 3D spatial UI where data density is represented by "gravity" and crowd movement.
Deep Integration: Seamlessly connecting the platform to external dev tools like GitHub and Firebase.
💡 What we learned
Prompt Engineering for Data: We learned how to use classes in Gemini to ensure AI agents produce structured, machine-readable code that can be fed directly into a ML training pipeline.
3D Optimization: Mastering instancedMesh in Three.js allowed us to render thousands of individual "user" particles simultaneously without lagging the browser.
🔮 What's next for Darwin
Multi-Modal Vision: Implementing Gemini Vision to allow the agents to "see" the visual balance of the page rather than just reading the CSS code. This would allow our agents to be more robust and sample from the page more effectively.
Log in or sign up for Devpost to join the conversation.