Getting started with Manipulation Component
A simple but powerful component to interact with entities in RealityKit.
Overview
This new component in visionOS 26 provides a set of combined gestures for moving, rotating, and scaling entities. It has reasonable defaults with a few things we can change.
We can add this just like any component.
let subject = Entity()
let mc = ManipulationComponent()
subject.components.set(mc)
content.add(subject)Just like with system gestures, the entity needs to have an input target component and a collision. ManipulationComponent has a convenient utility for adding these. We don’t need to manually create and add the component when using this helper.
let subject = Entity()
ManipulationComponent.configureEntity(subject, collisionShapes: [.generateBox(width: 0.25, height: 0.25, depth: 0.25)])
content.add(subject)With just a few lines of code we can
- Pinch and drag with one hand
- Translate (move) the entity based on our hand input
- Rotate the entity based on the orientation of our
- Hand the entity off from one hand to the other
- Pinch and drag with two hands to
- Scale the entity based on the distance between or hands
- Rotate the entity based on the orientation of our hand relative to each other
- Release the entity and it will return to the original transport from before the gestures began
All of this was possible in visionOS 1 and 2 using System Gestures. But ManipulationComponent makes these interactions very easy to use.
We also have a handful (haha) of behaviors we can customize.
releaseBehavior: reset or stay at the new transformdynamics.translationBehaviorlets us specify if the gesture should move the entitydynamics.primaryRotationBehaviorlets us specify if the entity rotate with the dragging handdynamics.secondaryRotationBehaviorlets us specify if the entity should rotate with two handsdynamics.scalingBehaviorlets us specify if the gesture should scale the entitydynamics.inertialets us customize the amount of inertia if the entity has physics mass.
There is a lot more to learn about this new component. There are events, system audio settings, and event he ability to delegate one entity to manipulate another.
Video Demo
Example Code
struct Example087: View {
@State private var releaseBehavior: ManipulationComponent.ReleaseBehavior = .reset
@State private var translationBehavior: ManipulationComponent.Dynamics.TranslationBehavior = .unconstrained
@State private var rotationBehaviorPrimary: ManipulationComponent.Dynamics.RotationBehavior = .unconstrained
@State private var rotationBehaviorSecondary: ManipulationComponent.Dynamics.RotationBehavior = .unconstrained
@State private var scalingBehavior: ManipulationComponent.Dynamics.ScalingBehavior = .unconstrained
@State private var inertia: ManipulationComponent.Dynamics.Inertia = .zero
// An entity we can manipulate
@State private var subject = createStepDemoBox()
var body: some View {
RealityView { content in
subject.position.y = -0.2
// We'll use configureEntity to set up input and collision on the subject
ManipulationComponent.configureEntity(subject, collisionShapes: [.generateBox(width: 0.25, height: 0.25, depth: 0.25)])
// OR Create the component and add it to the entity
// let mc = ManipulationComponent()
// Add the component and
// subject.components.set(mc)
content.add(subject)
}
.debugBorder3D(.white)
.ornament(attachmentAnchor: .scene(.topBack), contentAlignment: .top, ornament: {
// Just a whole bunch of buttons to change the values
VStack(alignment: .leading) {
Button(action: {
releaseBehavior = releaseBehavior == .reset ? .stay : .reset
if var mc = subject.components[ManipulationComponent.self] {
mc.releaseBehavior = releaseBehavior
subject.components.set(mc)
}
}, label: {
Text("Release Behavior:")
Spacer()
Text("\(releaseBehavior == .reset ? "Reset" : "Stay")")
})
Button(action: {
translationBehavior = translationBehavior == .unconstrained ? .none : .unconstrained
if var mc = subject.components[ManipulationComponent.self] {
mc.dynamics.translationBehavior = translationBehavior
subject.components.set(mc)
}
}, label: {
Text("Translation Behavior:")
Spacer()
Text("\(translationBehavior == .unconstrained ? "Unconstrained" : "None")")
})
Button(action: {
rotationBehaviorPrimary = rotationBehaviorPrimary == .unconstrained ? .none : .unconstrained
if var mc = subject.components[ManipulationComponent.self] {
mc.dynamics.primaryRotationBehavior = rotationBehaviorPrimary
subject.components.set(mc)
}
}, label: {
Text("Primary Rotation:")
Spacer()
Text("\(rotationBehaviorPrimary == .unconstrained ? "Unconstrained" : "None")")
})
Button(action: {
rotationBehaviorSecondary = rotationBehaviorSecondary == .unconstrained ? .none : .unconstrained
if var mc = subject.components[ManipulationComponent.self] {
mc.dynamics.secondaryRotationBehavior = rotationBehaviorSecondary
subject.components.set(mc)
}
}, label: {
Text("Secondary Rotation:")
Spacer()
Text("\(rotationBehaviorSecondary == .unconstrained ? "Unconstrained" : "None")")
})
Button(action: {
scalingBehavior = scalingBehavior == .unconstrained ? .none : .unconstrained
if var mc = subject.components[ManipulationComponent.self] {
mc.dynamics.scalingBehavior = scalingBehavior
subject.components.set(mc)
}
}, label: {
Text("Scaling Behavior:")
Spacer()
Text("\(scalingBehavior == .unconstrained ? "Unconstrained" : "None")")
})
// If we're using physics...
Text("Inertia:")
HStack {
Button(action: {
inertia = .zero
if var mc = subject.components[ManipulationComponent.self] {
mc.dynamics.inertia = inertia
subject.components.set(mc)
}
}, label: {
Label("Zero", systemImage: inertia == .zero ? "checkmark.circle" : "circle")
})
Button(action: {
inertia = .low
if var mc = subject.components[ManipulationComponent.self] {
mc.dynamics.inertia = inertia
subject.components.set(mc)
}
}, label: {
Label("Low", systemImage: inertia == .low ? "checkmark.circle" : "circle")
})
Button(action: {
inertia = .medium
if var mc = subject.components[ManipulationComponent.self] {
mc.dynamics.inertia = inertia
subject.components.set(mc)
}
}, label: {
Label("Medium", systemImage: inertia == .medium ? "checkmark.circle" : "circle")
})
Button(action: {
inertia = .high
if var mc = subject.components[ManipulationComponent.self] {
mc.dynamics.inertia = inertia
subject.components.set(mc)
}
}, label: {
Label("High", systemImage: inertia == .high ? "checkmark.circle" : "circle")
})
}
.controlSize(.small)
}
.frame(width: 400)
.padding()
.glassBackgroundEffect()
})
}
}Support our work so we can continue to bring you new examples and articles.
Download the Xcode project with this and many more examples from Step Into Vision.
Some examples are provided as standalone Xcode projects. You can find those here.
Follow Step Into Vision