Neko Dreams

• 

  App Logo

Inspiration

The game is inspired by Maddy Van's daily struggle to feed her cat, Estrella, as well as the innovation possible with on-demand world model generation.

What it does

Neko Dreams is a WebXR (VR) game where you try to make a picky cat happy. Neko has a secret idea of the “perfect” world and only gives you short, cryptic hints. You satisfy Neko by generating 3D worlds and seeing if they match.

You pick up to 10 items (e.g. couch, cat tree, fish tank) from a list; Google Gemini turns that list into a cozy room concept and a text prompt. That prompt is sent to WorldLabs Marble, which generates a 3D scene as a Gaussian splat that loads and plays in VR. Neko (powered by Anthropic Claude) secretly chooses what they want and gives hints; when you generate or load a world, Claude judges whether it matches. If it does, Neko is happy and you get a little celebration. You can also use “Mock Generate World” to try random descriptions without a full generation, and a saved worlds gallery lets you jump back into previous rooms. The whole flow runs in the headset with an in-world UI panel.

How we built it

We used the Immersive Web SDK (IWSDK) for WebXR, Three.js (super-three) for rendering, and Vite + TypeScript for the app. The 3D cat (Kat) is a GLB model with a speech-bubble system and simple movement. World generation goes through WorldLabs Marble (Gaussian splats); we poll the API, resolve the splat URL, then load the splat in the scene. Neko's personality, hints, and “does this world match?” decisions are handled by Anthropic Claude (messages API) with a fixed system prompt and structured output. The “pick items → room concept” step uses Google Gemini (via a small backend proxy) to turn the item list into a title, prompt, and theme. We cache generated worlds (e.g. in worldCache) and surface them in a saved-worlds system so users can reload past rooms from the panel.

Challenges we ran into

Initially we wanted to do the same project using the Unity kit. However, the generated Marble files, even at their lowest size, were running so poorly on the Meta Quest headset that we were targeting that the user was unable to properly look around and interact with the UI. So, we had to pivot to a WebXR app at 7pm :,) Additionally, we had trouble loading the generated Gaussian Splat from Marble into the right position and orientation in the application.

Accomplishments that we're proud of

A full VR loop: pick items → AI room concept → real 3D world generation → AI cat that judges and reacts. Neko feeling like a character: secret desire, cryptic hints, annoyed and happy phrases, and confetti when the player finally gets it right. They're fun to interact with. A practical “Mock Generate World” path so we can test Neko's logic and UX without burning WorldLabs credits every time. Saved worlds so players can revisit favorite rooms and we can demo multiple scenes without regenerating. A single in-headset UI for items, generation, progress, rating, and saved worlds, all in spatial UI.

What we learned

Performance and platform choice: For our Marble + Gaussian splat workflow on Quest, WebXR gave us playable frame rates and usable UI where our Unity build did not, even at the lowest splat size—and sometimes the “simpler” stack is the right one under constraints. Pivoting late: We learned to switch tech (Unity → WebXR) late in the day and still ship: scoping to what the new stack could do well and reusing our existing APIs (Claude, Gemini, WorldLabs) made the pivot feasible. How to load, display, and animate Gaussian splats in a WebXR app and how to hook that into an async generation API. How to combine multiple AI APIs (Claude for character and judging, Gemini for creative prompts) in one user flow.

What's next for Neko Dreams

More 3D models: Richer environments and maybe interactive objects or props that Neko can react to. Text-to-speech: Give Neko a voice so hints and reactions are spoken in addition to (or instead of) text bubbles. Voice input: Let players describe or refine the world by speaking (e.g. “make it more cozy” or “add a window”) and feed that into the prompt or concept step. UI improvements: Clearer hierarchy and feedback in the panel, better progress and error messages, and refinements to layout and readability in VR.

Built With

• claude
• cursor
• gemeni
• marble
• three.js
• typescript
• webxr
• worldlabs