CineBlock

Turn location photos into explorable 3D worlds and block your scenes with virtual mannequins — a pre-vis studio powered by World Labs Marble.

Comment

## Inspiration

Filmmakers spend hours scouting locations, sketching storyboards, and trying to communicate camera framing to their
crew — often with nothing more than overhead diagrams and verbal descriptions. We asked: what if you could turn a few
phone photos of a location into a full 3D world, then walk through it with a virtual camera and block your scene with
character stand-ins?

CineBlock was born from the intersection of generative 3D world models and practical filmmaking pre-visualization.

## What it does

CineBlock is a web-based virtual production studio that lets filmmakers:

  1. Upload 2–4 location photos (mapped to compass directions) and generate an explorable 3D Gaussian splat world via the World Labs Marble API
  2. Define scene structure — characters, props, shots with camera types (Wide, Close-Up, OTS, POV, etc.)
  3. Enter a virtual studio — orbit freely through the generated world, place color-coded character capsules and prop boxes via raycast click-to-place, and frame shots through a cinematic viewfinder overlay (16:9, 2.39:1, etc.)
  4. Capture reference frames — screenshot the viewfinder crop for Start and End frames of each shot, mark hero
    frames with a star toggle
  5. Export everything — download a structured ZIP of all captures plus dual-format JSON (human-readable CineBlock
    format and Aiuteur-compatible integration format)

## How we built it

  • Frontend: React 19 + TypeScript + Vite 6 with Tailwind CSS 4 for a dark, cinematic UI
  • 3D Rendering: Three.js (r181) via React Three Fiber, with SparkJS for Gaussian splat rendering. We built a custom overlay rendering pipeline that clears only the depth buffer between passes so mannequins always render on top of
    splats without z-fighting.
  • World Generation: World Labs Marble API — images are uploaded with azimuth angles, a world is generated, and we
    load the resulting SPZ (splat) file plus a GLB collider mesh for raycast-based interaction.
  • State Management: A single useReducer + Context pattern with ~20 action types covering the full lifecycle from setup through export.
  • Capture Pipeline: canvas.toDataURL() → calculate viewfinder pixel bounds → crop via offscreen canvas → store as data URL with hero metadata.
  • Testing: 86 Vitest tests covering the reducer across all phases — pure state-mutation tests with no DOM overhead.

## Challenges we faced

  • SparkJS + OrbitControls conflict: SparkJS's Level-of-Detail system deep-clones the camera every frame, but
    OrbitControls attaches non-clonable DOM references that crash the clone. We solved it with a camera clone interceptor
    that skips problematic properties.
  • World orientation: Getting the generated world to render right-side-up required careful azimuth mapping and
    debugging the Marble API's coordinate system.
  • Mannequin visibility over splats: Gaussian splats don't participate in the normal depth buffer, so mannequins
    would either render behind everything or in front of everything. Our two-pass overlay render (clear depth, keep color) solved this cleanly.
  • Viewfinder-constrained capture: The viewfinder is a CSS overlay, not a WebGL element, so we had to map its
    screen-space bounds back to canvas pixel coordinates for precise cropping.

## What we learned

  • Gaussian splat worlds are surprisingly compelling for filmmaking workflows — the organic, photographic quality makes them feel like real locations.
  • SparkJS is powerful but requires careful integration with existing Three.js patterns.
  • A reducer-based architecture scales well for complex multi-phase applications — every state transition is testable
    and predictable.

## What's next

CineBlock is designed to integrate into Aiuteur, a larger AI-powered filmmaking assistant. The Aiuteur-compatible
JSON export maps directly to Aiuteur's data model, where hero frames will feed into Stage 10 (Frame Generation) for
AI-assisted shot rendering. We also plan to add animation timelines, lighting controls, and higher-fidelity mannequin
models.

XX

Built With

Share this project:

Updates