Skip to content

MVP for Audio Playback (converted to project) #5181

@jleibs

Description

@jleibs

This issue has been converted to a project MVP for Audio Playback


Context

There's a much more detailed request for a broader set of audio features here: #2852

This issue is about a more minimal feature to just get audio playback working in the most minimal capacity.

Proposal

Make it possible to log audio assets as a single (or small number of) blobs and then have these play back only during "play" mode in the viewer. Avoid the hard questions around hings like audio-packetization, time-spans, and cross-timeline synchronization.

Simplifying assumptions:

  • The timestamp of the audio asset is the time at which audio should start playing on the given timeline.
  • The audio file should only be logged to timelines where the time scale matches the native audio time scale of the file.
  • Only support playing back one audio stream at a time.
  • Only play back audio when specifically in "play" mode with a 1.0 time-scale.

Suggested implementation

  • Add the ability to log a binary blob containing audio data in a small number of common formats using a similar approach to Asset3D.
  • Create an Audio "space view" which only supports a single audio asset. The view probably just shows something like an audio icon and maybe a volume / mute control.
  • When scrubbing time, the view will load the approach "LatestAt" audio payload given the current time selection, and then offset the stream to the correct relative timepoint.
  • When time is "playing" the view will playback the audio file using the native clock and track to make sure the audio clock and the native clock continue to be aligned.

Crates

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestuser-requestThis is a pressing issue for one of our users📺 re_vieweraffects re_viewer itself

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions