Using custom sounds with Manipulation Component

Joseph Simpson

We can silence the provided system sounds and play our own using Manipulation Events.

Overview

When we use Manipulation Component on an entity there are a few sounds that visionOS plays be default. Generally, I like these sounds. They fit in with the rest of visionOS. But what if these sounds don’t fit with the theme of an app? Maybe we want something more playful, or something that blends in with other audio. Or maybe we don’t want to play a sound at all.

We can opt out of the default sounds by setting audioConfiguration to .none.

var mc = ManipulationComponent()
mc.audioConfiguration = .none // turn off the default sounds

Apple didn’t provide a way to provide alternative sounds directly to the component. Instead, the docs suggest we should use Manipulation Events to play sounds.

To disable system audio or use custom sounds, set audioConfiguration to .none, and use component events to trigger your own audio responses.

Here is an example of playing a sound on begin and on release.

// Play a sound when we start the gesture
_ = content.subscribe(to: ManipulationEvents.WillBegin.self) { event in
    event.entity.playAudio(Example089.drop001Audio)
}

// Play a sound when we release the gesture
_ = content.subscribe(to: ManipulationEvents.WillRelease.self) { event in
    event.entity.playAudio(Example089.drop002Audio)
}

Note that we’re using WillRelease here instead of WillEnd. WillRelease fires as soon as the user stops the gesture, where as WillEnd doesn’t fire until after any return animations finish.

Example Code

struct Example089: View {

    // An entity we can manipulate
    @State private var subject = createStepDemoBox()
    
    // Audio resources
    static let drop001Audio = try! AudioFileResource.load(named: "drop_001")
    static let drop002Audio = try! AudioFileResource.load(named: "drop_002")

    var body: some View {
        RealityView { content in

            let audioComponent = AmbientAudioComponent()
            subject.components.set(audioComponent)

            // We'll use configureEntity to set up input and collision on the subject
            ManipulationComponent
                .configureEntity(
                    subject,
                    hoverEffect: .spotlight(.default),
                    collisionShapes: [.generateBox(width: 0.25, height: 0.25, depth: 0.25)]
                )

            // Create the component and add it to the entity
            var mc = ManipulationComponent()
            mc.audioConfiguration = .none // turn off the default sounds

            // Play a sound when we start the gesture
            _ = content.subscribe(to: ManipulationEvents.WillBegin.self) { event in
                event.entity.playAudio(Example089.drop001Audio)
            }

            // Play a sound when we release the gesture
            _ = content.subscribe(to: ManipulationEvents.WillRelease.self) { event in
                event.entity.playAudio(Example089.drop002Audio)
            }

            // Add the component and
            subject.components.set(mc)
            content.add(subject)

        }
    }
}

Credits

The audio files used in this post were generously provided to the commons by Kenney Assets.

Support our work so we can continue to bring you new examples and articles.

Support our Work

Download the Xcode project with this and many more examples from Step Into Vision.
Some examples are provided as standalone Xcode projects. You can find those here.

Follow Step Into Vision

Questions or feedback?