Yume - Step into your dream

Inspiration

Children naturally believe that imagination can change reality. A simple drawing of a castle or a monster can represent an entire world in their minds.

We were inspired by the idea of “神笔马良” (The Magic Brush) — a legend where anything you draw becomes real.

At the same time, recent advances in AI world models and spatial generation allow images to be transformed into explorable 3D environments.

We asked a simple question:

What if a child’s drawing could become a world you could actually walk through?

Yume explores this idea by turning imagination into a navigable space, letting users capture memories from the dream they created.

What it does

Yume is an imagination-to-world engine.

A user uploads a drawing — anything from a child’s crayon sketch to a concept illustration. Yume interprets the drawing and generates a fully explorable 3D world based on it.

Inside the world, the user can freely look around while the camera slowly drifts in a gentle dream mode.

During exploration, the user carries a Polaroid camera that can capture up to six moments from the dream world.

After the sixth photo is taken, the experience concludes and Yume generates a Polaroid strip containing:

the user’s original drawing
six captured moments from the generated world

The result is a small keepsake — a memory of the dream that started from a simple drawing.

How we built it

Yume is built as a lightweight pipeline that transforms drawings → scenes → explorable worlds.

Frontend

A web interface built with React where users upload drawings and explore the generated world.

The interface is designed to feel like opening a sketchbook, with soft transitions and a tactile Polaroid camera UI.

AI Interpretation Layer

The uploaded drawing is processed by a multimodal AI model that interprets the scene and extracts semantic descriptions such as:

objects
lighting
mood
spatial layout

World Generation

The structured description is sent to the World Labs Marble API, which generates a navigable 3D environment.

Marble returns assets including Gaussian splats and scene meshes that can be rendered in the browser.

World Viewer

The generated world is rendered using WebXR+Spark.js in the browser.

A custom dream camera system slowly drifts through the environment to enhance the feeling of being inside a dream.

Polaroid Capture

A virtual Polaroid camera captures screenshots from the viewer.

After six captures, a photo strip collage is generated combining the images with the original drawing.

Challenges we ran into

One challenge was translating simple drawings into coherent 3D environments.

Children’s drawings are often abstract, so the AI interpretation layer needed to infer spatial structure and atmosphere from minimal visual information.

Another challenge was handling the asynchronous generation pipeline. World generation can take tens of seconds, so we had to design loading states and transitions that maintained the dream-like experience rather than feeling like a traditional loading screen.

Rendering large Gaussian splat scenes in the browser also required careful performance tuning to ensure the experience remained smooth on typical hardware.

Accomplishments that we're proud of

One thing we’re particularly proud of is how intuitive the interaction feels.

A single drawing becomes an entire experience with a clear beginning, middle, and end.

We also love the Polaroid mechanic. Instead of simply exploring a generated world, users actively choose meaningful moments to capture, turning the experience into a small narrative.

Most importantly, we created a system where imagination becomes spatial.

A drawing is no longer just an image — it becomes a place.

What we learned

Through building Yume, we learned how powerful world generation models can be when combined with simple creative inputs.

We also learned that in immersive experiences, structure matters. By limiting the user to six photos, we created a natural arc:

exploration → discovery → reflection

Finally, we learned that AI tools are most magical when they amplify creativity, rather than replacing it. The drawing is still the heart of the experience — the AI simply brings it to life.