Spatial SwiftUI: Manipulation modifiers

Joseph Simpson

We can use the SwiftUI equivalent of Manipulation Component as a modifier.

Overview

We’ve explored the Manipulation Component in RealityKit in detail in previous examples.

See also: Learn visionOS: System Gestures

Let’s take a look at the SwiftUI version of manipulation. We can add manipulable() to a Model3D view.

Model3D(named: "Earth")
  .manipulable()

We can now translate, rotate, and scale the model. When we release it will snap back to its place in the layout.

Note: we can apply this modifier to any view, but it seems to be intended for Model3D view. All other views are clipped by window/volume bounds.

If we want to place input on one view, but move another view, we can redirect manipulation. We add .manipulationGesture() to the Earth model and pass it some state to update. Then we use manipulable(using:) to update the Moon model when the state changes.

@State private var manipulationState = Manipulable.GestureState()
...

ModelViewSimple(name: "Earth", bundle: realityKitContentBundle)
    .manipulationGesture(updating: $manipulationState)

ModelViewSimple(name: "Moon", bundle: realityKitContentBundle)
  .manipulable(using: manipulationState)

We can customize the behavior of manipulable. For example, to remove rotation but allow translation and scaling.

ModelViewSimple(name: "ToyRocket", bundle: realityKitContentBundle)
    .manipulable(operations: [.translate, .scale])

The onChange event can be useful if we need to constrain movement or if we need to capture manipulation data.

ModelViewSimple(name: "ToyRocket", bundle: realityKitContentBundle)
    .frame(width: frameSize, height: frameSize)
    .manipulable(operations: [.translate, .scale], onChanged: { event in
        let translation = event.value?.transform?.translation ?? .zero
        onChangeVectorExample = SIMD3<Float>(Float(translation.x), Float(translation.y), Float(translation.z))
    })

Video Demo

Example Code

struct Example105: View {

    @State private var manipulationState = Manipulable.GestureState()
    @State private var onChangeVectorExample: SIMD3<Float> = .zero
    @State private var showDebugLines = false
    @State private var manipulationEnabled = true

    let frameSize: CGFloat = 100

    var body: some View {
        VStackLayout().depthAlignment(.center) {

            /// Demo 01: Default implementation of manipulable()
            HStackLayout().depthAlignment(.center) {

                VStack {
                    Text("manipulable()")
                        .font(.title)
                    Text("Default implementation")
                        .font(.caption)
                }
                .padding()
                .background(.black)
                .cornerRadius(24)
                .shadow(radius: 20)
                .frame(width: 220, height: 160)
                .manipulable()
                .hoverEffect()

                ModelViewSimple(name: "Earth", bundle: realityKitContentBundle)
                    .frame(width: frameSize, height: frameSize)
                    .manipulable()
                    .hoverEffect()

                ModelViewSimple(name: "ToyRocket", bundle: realityKitContentBundle)
                    .frame(width: frameSize, height: frameSize)
                    .manipulable()
                    .hoverEffect()

            }
            .debugBorder3D(showDebugLines ? .white : .clear)

            /// Demo 02: Redirecting input using manipulable and manipulationGesture
            HStackLayout().depthAlignment(.center) {

                HStackLayout().depthAlignment(.center) {

                    VStack {
                        Text("Redirected")
                            .font(.headline)
                        Text("Input is redirected from Earth to the Moon")
                            .font(.caption)
                    }
                    .padding()
                    .background(.black)
                    .cornerRadius(24)
                    .shadow(radius: 20)
                    .frame(width: 180, height: 120)

                    SpatialContainer(alignment: .topLeadingFront) {
                        ModelViewSimple(name: "Earth", bundle: realityKitContentBundle)
                            .frame(width: frameSize, height: frameSize)
                            .padding(24)
                            .manipulationGesture(updating: $manipulationState)
                            .hoverEffect()

                        ModelViewSimple(name: "Moon", bundle: realityKitContentBundle)
                            .frame(width: frameSize * 0.3, height: frameSize * 0.3)
                            .manipulable(using: manipulationState)
                    }

                }
                .debugBorder3D(showDebugLines ? .white : .clear)


                /// Demo 03: Customizing manipulable() using operations and onChanged event
                HStackLayout().depthAlignment(.center) {

                    VStack {
                        VectorDisplay(title: "Position", vector: onChangeVectorExample)
                    }
                    .padding()
                    .background(.black)
                    .cornerRadius(24)
                    .shadow(radius: 20)
                    .frame(width: 180, height: 120)

                    ModelViewSimple(name: "ToyRocket", bundle: realityKitContentBundle)
                        .frame(width: frameSize, height: frameSize)
                        .manipulable(operations: [.translate, .scale], onChanged: { event in
                            let translation = event.value?.transform?.translation ?? .zero
                            onChangeVectorExample = SIMD3<Float>(Float(translation.x), Float(translation.y), Float(translation.z))
                        })

                }
                .debugBorder3D(showDebugLines ? .white : .clear)

            }

        }
        // Controls to modify the example
        .ornament(attachmentAnchor: .scene(.trailing), contentAlignment: .leading, ornament: {
            VStack(alignment: .center, spacing: 8) {
                Button(action: {
                    withAnimation {
                        manipulationEnabled.toggle()
                    }
                }, label: {
                    Text("Enabled")
                })
                Button(action: {
                    showDebugLines.toggle()
                }, label: {
                    Text("Debug")
                })
            }
            .padding()
            .controlSize(.small)
            .glassBackgroundEffect()

        })
    }
}

fileprivate struct VectorDisplay: View {
    let title: String
    let vector: SIMD3<Float>

    var body: some View {
        VStack(alignment: .leading) {
            Text(title)
                .fontWeight(.bold)
            VStack {
                ForEach(["X", "Y", "Z"], id: \.self) { axis in
                    Text("\(axis): \(String(format: "%8.3f", axis == "X" ? vector.x : axis == "Y" ? vector.y : vector.z))")
                        .frame(width: 120, alignment: .leading)
                }
            }
        }
    }
}

Support our work so we can continue to bring you new examples and articles.

Support our Work

Download the Xcode project with this and many more examples from Step Into Vision.
Some examples are provided as standalone Xcode projects. You can find those here.

Follow Step Into Vision

Questions or feedback?