Skip to content

Add the MediaTrack struct and the associated PlayerEvent::MediaTracksChanged event #473

@tharkum

Description

@tharkum

Currently the gstreamer-rs backend has common media information change callback for any pipeline property change (including the stream collection change)
https://docs.rs/gstreamer-play/latest/src/gstreamer_play/auto/play_signal_adapter.rs.html#144
https://docs.rs/gstreamer-play/latest/src/gstreamer_play/auto/play_media_info.rs.html

And there are conversion from PlayMediaInfo to our Metadata (here) and if there are any change in compare with the latest metadata the media backend sends PlayerEvent::MetadataUpdated event (here) and a client must parsing this message and identify what was changed.

The proposal is to add new PlayerEvent::MediaTracksChanged event which will allow to inform a client if stream collection was changed and need to adjust list of the media tracks on its side.

Add new MediaTrack struct something like instead of Vec<String>

struct MediaTrack {
   type: Audio/Video/...,
   kind: "main", "", ...
   id: "0", "1", ....
   label: ...
   language: ...
}

See https://html.spec.whatwg.org/multipage/media.html#dom-videotrack-kind (as example)

And DO NOT use codec as a possible field (see https://github.com/servo/media/blob/main/backends/gstreamer/player.rs#L62) because it could be changed dynamically multiple times due to internal issue(?) of the media engine (outdated gstreamer version 1.20-2?)

The following test
./mach test-wpt -r /html/semantics/embedded-content/the-video-element/video_size_preserved_after_ended.html on
the next media file media/test-1s.webm is constantly switching between On2 VP9 and VP9 video codecs during playback with 2-100+ media update info messages.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions