Lumina

"Life moves pretty fast. If you don't stop and look around once in a while, you could miss it."– Ferris Bueller

Comment 4

Inspiration

In the face of the ever-accelerating pace of the world, we wanted to create a casual gaming experience that showcases the benefits of slowing down.

Heavily inspired by Wrapped – a beautifully animated short film from 2014 – we set out to explore themes of presence, nature, and the passage of time while creating a visually stunning immersive environment.

Other inspirations include Wisp's visual language development, NPC interactions, and plants’ growth cycle; ABZÛ's and Ark: Aberration's art and environment development; and Pillows's relaxing ambiance and use of the ceiling to create otherworldly environments.

Our project also heavily influenced by meditative, atmospheric games like Journey and Flower.

Main Concept

Through Lumina MR, we set out to create an antidote to the busy world we live in while showcasing the unique potential of Mixed Reality.

Leaning on the concept of Skychology, our game targets the ceiling as a blank canvas for an imaginative and magical garden.

However, the player does not have control over how the garden will take shape. The kinds of plants and where they will grow are a mystery, causing surprise and delight for players. We decided this is the optimal approach to motivate players to plant more seeds and be immersed in the action.

The game encourages playful actions through intuitive interactions with seeds & plants and also rewards inaction through stillness and observation.

What it does

In Lumina MR, your presence fosters an immersive virtual garden on your ceiling, composed of 3 main elements: Seeds, Islands, and Plants.

Seeds are spawned randomly on the walls of your room every time you enter the garden. You can use pinch gestures (up close or from a distance) to grab them. Joining two seeds together causes them to merge and change properties. Throwing/flinging the seeds toward the ceiling causes them to pop and give origin to islands and plants.

Islands are the base for plants to grow on.

Plants are affected by the passing of time. They grow naturally in your presence, but faster once you stop to admire them; they will also wither away in your absence. A fully grown plant can be harvested, giving way to new seeds. Turn both your palms towards the fully grown plant to give origin to a new set of seeds.

The game has two distinct modes: Building and Gazing. These modes affect your room's atmosphere, color, sound, and growth rate of plants. Gazing mode is enabled when the player pauses to observe the garden. Plants grow faster in this mode. The main soundtrack also changes to enhance the player’s viewing experience.

Lastly, instead of having a typical menu. Our interactive NPC plays the dual role of a companion and the main menu.

How we built it

Lumina MR is built with Unity (version 2022.3.22f1), leveraging some of Meta Presence Platform’s Mixed Reality capabilities:

  • Passthrough
    Besides using passthrough to allow the player to see their world, we're using Color Look-Up Tables (LUTs) to stylize passthrough to communicate our two game states: Building and Gazing.
  • Spatial Anchors
    We're using (locally stored) Spatial Anchors to maintain Islands and Plants positions, and ensure the garden's consistency between sessions.
  • Scene
    We're using Scene understanding capabilities alongside the Mixed Reality Utility Kit toolset to identify valid spawn points for the Plants in the player's room ceiling.
  • Depth
    We're using depth occlusion for an increased immersion effect.

The interactions throughout the experience rely on Hand Tracking and Interaction SDK:

  • Seeds are set up as HandGrabInteractable and DistanceHandGrabInteractable components and, as so, respond to HandGrab, HandDistanceGrabInteractors, and DistanceHandGrabInteractors.
  • PokeInteractibles allow the player to select the game options provided by the NPC.
  • We used a combination of ShapeRecognizer and TransformRecognizer components to make the most of Hand Pose Detection capabilities. This allows for the following gestures:
    • Single-Handed Palm Up: This gesture summons our cute NPC character.
    • Dual-Handed Palm Away: This gesture, when a user's palms are facing a fully-grown plant, works to harvest seeds from these plants.

Lastly, we've used the Audio SDK for spatial sound effects integration. We've tweaked reverb and gain settings on each component to give it a unique effect that fits well with the atmosphere we are trying to create.

Challenges we ran into

The biggest challenge throughout the development of Lumina MR was undeniably adjusting the scope of the concept to match the team's capabilities and availability.

To achieve our concept we strived to design simple yet meaningful hand interactions, which proved a difficult task when it came to pose identification and ensuring multiple pose interactions didn't overlap or conflict (e.g. grabbing objects with your hand, depending on the rotation of your wrist, can sometimes be mistaken for the palm up gesture used to summon our NPC character). In the final result, we tailored our interactions to mitigate this issue as much as possible, although in some rare cases interaction conflicts are a known issue.

Another challenge during this process was creating a satisfying balance between giving the player a sense of control in planting the seeds without compromising immersion and the garden's visual balance by making sure plants were spawned sparsely, at sensible locations, without colliding with the room's furniture and boundaries. Our solution consists of a blend of a modified version of the Meta Quest Presence Platform's Anchor Prefab Spawner Building Block and physics collision detection between seeds and the ceiling and islands. The Anchor Prefab Spawner Building Block allows us to identify valid spawn positions within the room which we use for targeting the seed's fling/throw trajectory, and then spawn an island (if there is none yet) or a plant on the collision point of the seed with the ceiling (or an existing island).

Accomplishments that we're proud of

We're proud of the visual interactive language we built to create an immersive sense of relaxation and wonder. Overall, we are proud of the team's agility in pivoting and prioritizing the aspects that would make for a satisfying and cohesive experience that conveys our concept's metaphor.

What we learned

We learned a lot about the Meta Quest Presence Platform. Namely, Passthrough stylizing effects and transitions, Scene understanding and object placement, depth occlusion, hand interactions, and pose detection.

What's next for Lumina MR

  • Improved game flow:
    • Better room setup handling and communicating of the game's Scene understanding requirements
    • Provide a fallback in case the player doesn't grant Spatial Data permission
    • Seamsless title screen and onboarding integration (e.g. Cubism)
  • NPC development:
    • Enjoyable autonomous behaviors
    • Meaningful collaboration with the player (collecting, combining and planting seeds, harvesting), explore the integration of voice interactions
  • Improved object placement:
    • Ensuring fully grown plants don't collide with furniture in the room
    • Ensuring plants' boundaries don't intercept with the room mesh or other plants
  • Gameplay development:
    • Improved planting experience (fine-tune throwing seeds and seeds' trajectory toward valid plant spawn points)
    • Improved time passage conveyance (e.g. using moving lights during Gazing mode to create a timelapse effect, night and day lighting, and passthrough effects)
  • Narrative development:
    • Communicate a sense of gameplay progression through the development of the bond between the player and the NPC
    • Add narrative elements that further illustrate the importance of slowing down and being present
  • Improved environment and game element interactions:
    • More satisfying plant and seed interactions
    • More meaningful correlation between seed combinations and plant outcomes
    • Increased contrast between Building and Gazing modes through added visual elements that strongly convey the idea of the garden thriving when the player devotes their time and attention to it
  • Improved sound design
  • Improved development support:
    • Ability to fully run and test the entire experience (and select parts) using Meta XR Simulator
  • Exploring multiplayer capabilities (co-located and remote)

Built With

+ 13 more
Share this project:

Updates