RealityKit Basics: Entity Observation

Joseph Simpson

Starting in visionOS 26, we can observe changes to component data on entities.

Overview

Since visionOS 1, we’ve been able to update RealityKit entities based on changes to data in SwiftUI. We commonly do this with the update closure on RealityView. Starting with visionOS 26 it is possible for SwiftUI to observe changes to RealityKit entities. We now have two-way communication between these frameworks.

To explore this, we’ll create a scene that orbits a subject around a central point. There are four way points we can collide with. We’ll animate the subject with a timeline that runs continuously.

The demo scene in Reality Composer Pro

When we reach one, we’ll capture the name of the way point and store it in a custom component.

fileprivate struct WaypointTracker: Component, Codable {
    public var waypoint: String = "not set"
    public init() {}
}
_ = content.subscribe(to: CollisionEvents.Began.self, on: subject)  { collisionEvent in
        subject.components[WaypointTracker.self]?.waypoint = collisionEvent.entityB.name
        print("Waypoint Collision: \(subject.components[WaypointTracker.self]!.waypoint)")
    }

We can observe changes by using a new observable interface on Entity. We can access common values like position, transform, or children.

entity.observable.position
entity.observable.transform
entity.observable.children

We can also observe chances in components, including custom components.

entity.observable.components[WaypointTracker.self]?.waypoint

We can observe this component directly in a SwiftUI view.

WaypointMap(waypoint: .constant(subject.observable.components[WaypointTracker.self]?.waypoint ?? "Not Found"))

Or we can listen for changes to it. This would give us a chance to apply side effects such as saving changes in a data store.

// A place to capture the data
@State var observedWaypoint: String = "Not Set"

// Listen for changes
    .onChange(of: subject.observable.components[WaypointTracker.self]?.waypoint) { _, newValue in
        if let newValue = newValue {
            // Apply any side effects here
            print("Observed Waypoint: \(newValue), applying side effects...")
            self.observedWaypoint = newValue
        }
    }

// Later
WaypointMap(waypoint: $observedWaypoint)

See also

Video Demo

Example Code

struct Example138: View {

    @State var collisionSubjectBegan: EventSubscription?
    @State var subject = Entity()
    @State var observedPosition: SIMD3<Float> = .zero
    @State var observedWaypoint: String = "Not Set"

    init() {
        WaypointTracker.registerComponent()
    }

    var body: some View {
        RealityView { content in

            guard let scene = try? await Entity(named: "EntityObservationDemo", in: realityKitContentBundle) else { return }
            content.add(scene)
            guard let subject = scene.findEntity(named: "Subject") else { return }
            subject.components.set(WaypointTracker())
            self.subject = subject

            // This scene uses a timeline to animate the subject around the scene.
            // We'll use this collision to update the custom component so we can observe changes from it
            collisionSubjectBegan = content
                .subscribe(to: CollisionEvents.Began.self, on: subject)  { collisionEvent in
                    subject.components[WaypointTracker.self]?.waypoint = collisionEvent.entityB.name
                    print("Waypoint Collision: \(subject.components[WaypointTracker.self]!.waypoint)")
                }
        }
        // Listen for changes, peform side effects, and apply the data to state
        .onChange(of: subject.observable.position) { _, newValue in
            // Apply any side effects here
            self.observedPosition = newValue
        }
        .onChange(of: subject.observable.components[WaypointTracker.self]?.waypoint) { _, newValue in
            if let newValue = newValue {
                // Apply any side effects here
                print("Observed Waypoint: \(newValue), applying side effects...")
                self.observedWaypoint = newValue
            }
        }
        .ornament(attachmentAnchor: .scene(.bottomFront), ornament: {
            VStack(alignment: .leading, spacing: 10) {
                // Example 1: Direct usage of the observed data in SwiftUI views
                Vector3Display(title: "Direct Usage", vector: subject.observable.position)

                HStack(spacing: 10) {
                    WaypointMap(waypoint: .constant(subject.observable.components[WaypointTracker.self]?.waypoint ?? "Not Found"))
                    Spacer(minLength: 0)
                }

                Divider()
                    .padding(.vertical, 2)

                // Example 2: Observing state that is updated using onChange
                Vector3Display(title: "onChange Examples", vector: observedPosition)

                HStack(spacing: 10) {
                    WaypointMap(waypoint: $observedWaypoint)
                    Spacer(minLength: 0)
                }
            }
            .padding()
            .background(.black)
            .clipShape(.rect(cornerRadius: 12))

        })
        .onDisappear {
            collisionSubjectBegan?.cancel()
            collisionSubjectBegan = nil
        }
    }
}

fileprivate struct WaypointMap: View {

    @Binding var waypoint: String

    private var symbolName: String {
        switch waypoint.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() {
        case "left":
            return "circle.grid.cross.left.filled"
        case "right":
            return "circle.grid.cross.right.filled"
        case "front":
            return "circle.grid.cross.down.filled"
        case "back":
            return "circle.grid.cross.up.filled"
        default:
            return "questionmark.circle"
        }
    }

    var body: some View {
        HStack(spacing: 8) {
            Image(systemName: symbolName)
                .font(.system(size: 22, weight: .semibold))
                .contentTransition(.symbolEffect(.replace))
                .animation(.easeInOut(duration: 0.25), value: symbolName)

            Text(waypoint)
                .font(.system(size: 14, weight: .medium, design: .rounded))
        }
    }
}

fileprivate struct WaypointTracker: Component, Codable {
    public var waypoint: String = "not set"
    public init() {}
}

Support our work so we can continue to bring you new examples and articles.

Support our Work

Download the Xcode project with this and many more examples from Step Into Vision.
Some examples are provided as standalone Xcode projects. You can find those here.

Follow Step Into Vision

Questions or feedback?