stream: add --stream-segmented-duration argument #5493

bastimeyer · 2023-08-09T23:54:49Z

This deprecates --hls-duration in favor of --stream-segmented-duration and adds support for stream durations to DASH streams.

Opening this as a draft for now since I want to have a look at --hls-start-offset first, as mentioned here:
#5450 (comment)

Also, as mentioned, the duration logic unfortunately can't be implemented in the SegmentedStreamWorker base class (for now), because HLSStreamWorker and DASHStreamWorker both yield different kinds of "segments". I'm putting this into quotes here, because the HLS implementation doesn't actually yield Segment objects, but Sequence objects, which are segment wrappers, which means it's incompatible and would require a deep refactoring first.

The related --hls-start-offset changes I've talked about in #5450 (once I find the time and motivation to work on it) will be done in a separate PR. But until I'm done with that or made a final decision that it's not worth changing it, or if it even can't be done, this PR will stay as a draft.

HLS

$ streamlink twitch.tv/shroud best --stream-segmented-duration 30
[cli][info] Found matching plugin twitch for URL twitch.tv/shroud
[cli][info] Available streams: audio_only, 160p (worst), 360p, 480p, 720p60, 1080p60 (best)
[cli][info] Opening stream: 1080p60 (hls)
[cli][info] Starting player: mpv
[plugins.twitch][info] Will skip ad segments
[plugins.twitch][info] Low latency streaming (HLS live edge: 2)
[plugins.twitch][info] Waiting for pre-roll ads to finish, be patient
[stream.hls][info] Filtering out segments and pausing stream output
[plugins.twitch][info] This is not a low latency stream
[stream.hls][warning] Encountered a stream discontinuity. This is unsupported and will result in incoherent output data.
[stream.hls][info] Resuming stream output
[stream.hls][info] Stopping stream early after 30.00 seconds
[cli][info] Stream ended
[cli][info] Closing currently open stream...

DASH

$ streamlink https://steamcommunity.com/broadcast/watch/76561199514732246 best --stream-segmented-duration 30
[cli][info] Found matching plugin steam for URL https://steamcommunity.com/broadcast/watch/76561199514732246
[cli][info] Available streams: 360p (worst), 480p, 720p, 1080p (best)
[cli][info] Opening stream: 1080p (dash)
[cli][info] Starting player: mpv
[stream.dash][info] Stopping stream early after 30.00 seconds
[stream.dash][info] Stopping stream early after 30.00 seconds
[utils.named_pipe][info] Creating pipe streamlinkpipe-38610-1-718
[utils.named_pipe][info] Creating pipe streamlinkpipe-38610-2-4207
[cli][info] Stream ended
[cli][info] Closing currently open stream...

DASH using dash-protocol plugin and the duration parameter

$ streamlink 'dash://https://qn-55555-fra01-1-02-1.steam-content-live-1.qwilted-cds.cqloud.com/cache3-ams1.steamcontent.com/broadcast/76561199514732246/8318933167585007892/manifest/0/qn-55555-fra01-1-02-1.steam-content-live-1.qwilted-cds.cqloud.com%2Fcache3-ams1.steamcontent.com/?broadcast_origin=ext1-tyo2.steamserver.net duration=30' best
[cli][info] Found matching plugin dash for URL dash://https://qn-55555-fra01-1-02-1.steam-content-live-1.qwilted-cds.cqloud.com/cache3-ams1.steamcontent.com/broadcast/76561199514732246/8318933167585007892/manifest/0/qn-55555-fra01-1-02-1.steam-content-live-1.qwilted-cds.cqloud.com%2Fcache3-ams1.steamcontent.com/?broadcast_origin=ext1-tyo2.steamserver.net duration=30
[cli][info] Available streams: 360p (worst), 480p, 720p, 1080p (best)
[cli][info] Opening stream: 1080p (dash)
[cli][info] Starting player: mpv
[stream.dash][info] Stopping stream early after 30.00 seconds
[stream.dash][info] Stopping stream early after 30.00 seconds
[utils.named_pipe][info] Creating pipe streamlinkpipe-39044-1-1610
[utils.named_pipe][info] Creating pipe streamlinkpipe-39044-2-5653
[cli][info] Stream ended
[cli][info] Closing currently open stream...

slhck · 2023-08-10T08:20:41Z

src/streamlink/session.py

+            * - stream-segmented-duration
+              - ``float | None``
+              - ``None``
+              - Limit the playback duration of segmented streams, rounded to the nearest segment


As an outsider, I still think this name is a bit confusing or hard to discover. Why are the other stream-segment options in present tense?

https://deploy-preview-5493--streamlink.netlify.app/cli#cmdoption-stream-segmented-duration

https://deploy-preview-5493--streamlink.netlify.app/api/session#streamlink.session.Streamlink.set_option

We can change the description's imperative, to make it more consistent with the other descriptions. As you can see in the diff, this was copied from the old option.

I have already explained the reasoning behind the option/CLI-argument naming. If you have a better name, then suggest one. It can't be stream-duration, because it's limited to segmented streams. The stream-segment-... options are meant for logic that affects single segments in segmented streams. The newly added stream-segmented-... options are for logic that affects segmented streams as a whole.

Okay, no worries, I have no better suggestion that wouldn't at the same time give the impression that it works for all streams.

slhck · 2023-08-10T08:23:36Z

src/streamlink/stream/dash.py

                    yield segment

+                    # init segments have duration set to None
+                    duration += segment.duration or 0.0


Is there a chance that other segments have no duration set (after the init segment?). If so there could be a warning about it, because the overall duration is wrong in that case.

The duration attribute is mandatory for segments in SegmentLists and SegmentTemplates, and so is the d attribute in SegmentTimelines.

See ISO/IEC 23009-1:2022 or the XML schema here:

https://github.com/Dash-Industry-Forum/MPEG-Conformance-and-reference-source/blob/c50c32e61b5c1d3401840872a45bc10f6ea2709a/conformance/MPDValidator/schemas/DASH-MPD.xsd#L311

https://github.com/Dash-Industry-Forum/MPEG-Conformance-and-reference-source/blob/c50c32e61b5c1d3401840872a45bc10f6ea2709a/conformance/MPDValidator/schemas/DASH-MPD.xsd#L371

Other kinds of DASH stream types are not supported by Streamlink.

We do however treat the duration attribute as optional:

https://github.com/streamlink/streamlink/blame/6.0.1/src/streamlink/stream/dash_manifest.py#L733

This was added in 577617a for some reason, so when I refactored the whole DASH code earlier this year, I kept it. While the specification needs to be strict, a DASH manifest parser shouldn't be, so even broken/invalid manifests can be played.

Each media Segment that gets yielded by the DASH manifest parser does have a duration set, and it can only be None if the duration attribute is missing, which is not Streamlink's fault in this case.

Apparently the duration can indeed be optional, because of manifests with only a single segment.

src/streamlink_cli/argparser.py

- Deprecate `--hls-duration` in favor of `--stream-segmented-duration`, as well as the respective session options - Update and rename `HLSStreamWorker.duration` attribute - Rename from `duration_limit` - Don't cast to `int` - Update log message when ending the stream early - Add tests

- Add `duration` parameter to `DASHStream`, similar to `HLSStream`, and add support for `--stream-segmented-duration` - Update, fix and add tests: Split up `worker` fixture of `TestDASHStreamWorker` into `stream`, `reader` and `worker`

bastimeyer · 2023-08-14T16:39:37Z

After having a look at the start-offset changes, I'm unsure if I want to continue this PR in the current state.

As said, I only want to merge the duration changes into master if the start-offset changes can be merged at the same time, as both are related.

I've already explained that the duration implementation should actually be moved to the SegmentedStreamWorker base class instead of the current solution. The reason for the current solution is that there is no Segment base class which the base worker class could expect and work with. This is due to historic reasons when the segmented streams were implemented during the Livestreamer project, with support for py2. The various segment implementations are all independent of each other because they subclass named tuples, and there's no further inheritance. There needs to be a base Segment dataclass and the various segmented stream implementations need to be updated accordingly with their own segments inheriting this base Segment class, all with proper typing information (generic classes and typevars).

This means that the HLS implementation needs to move away from the unnecessarily confusing Sequence wrapper. That also requires changes in the Twitch-specific HLS implementations, and probably others as well (haven't checked it yet). Other segmented stream implementations like UStreamTV are affected by this, too.

Otherwise, the HLS and DASH implementations require different logic for calculating the start-offset and durations, which is stupid. Refactoring this is very much beneficial, but it's also much more work.

bastimeyer · 2023-10-10T13:02:08Z

Closing in favor of #5601

bastimeyer added feature request stream: HLS stream: DASH CLI API: session labels Aug 9, 2023

bastimeyer mentioned this pull request Aug 9, 2023

Turn --hls-duration into a generic segmented-stream CLI arg and add support for DASH durations #5450

Closed

3 tasks

slhck reviewed Aug 10, 2023

View reviewed changes

bastimeyer mentioned this pull request Aug 11, 2023

utils.times: rewrite hours_minutes_seconds() #5495

Merged

bastimeyer added 2 commits August 12, 2023 17:21

stream.dash: add duration parameter

72ef570

- Add `duration` parameter to `DASHStream`, similar to `HLSStream`, and add support for `--stream-segmented-duration` - Update, fix and add tests: Split up `worker` fixture of `TestDASHStreamWorker` into `stream`, `reader` and `worker`

bastimeyer force-pushed the stream/segmented-duration branch from 3316ce8 to 72ef570 Compare August 12, 2023 15:22

bastimeyer mentioned this pull request Aug 15, 2023

Refactor segmented streams and use a common BaseSegment #5498

Closed

14 tasks

bastimeyer mentioned this pull request Oct 10, 2023

stream.segmented: add --stream-segmented-duration #5601

Closed

bastimeyer closed this Oct 10, 2023

bastimeyer deleted the stream/segmented-duration branch October 10, 2023 13:02

bastimeyer mentioned this pull request Oct 30, 2025

stream.segmented: add --stream-segmented-duration #6705

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

stream: add --stream-segmented-duration argument #5493

stream: add --stream-segmented-duration argument #5493

Uh oh!

bastimeyer commented Aug 9, 2023

Uh oh!

slhck Aug 10, 2023

Uh oh!

bastimeyer Aug 10, 2023

Uh oh!

slhck Aug 10, 2023

Uh oh!

slhck Aug 10, 2023

Uh oh!

bastimeyer Aug 10, 2023

Uh oh!

bastimeyer Aug 10, 2023

Uh oh!

Uh oh!

bastimeyer commented Aug 14, 2023

Uh oh!

bastimeyer commented Oct 10, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

stream: add --stream-segmented-duration argument #5493

stream: add --stream-segmented-duration argument #5493

Uh oh!

Conversation

bastimeyer commented Aug 9, 2023

HLS

DASH

DASH using dash-protocol plugin and the duration parameter

Uh oh!

slhck Aug 10, 2023

Choose a reason for hiding this comment

Uh oh!

bastimeyer Aug 10, 2023

Choose a reason for hiding this comment

Uh oh!

slhck Aug 10, 2023

Choose a reason for hiding this comment

Uh oh!

slhck Aug 10, 2023

Choose a reason for hiding this comment

Uh oh!

bastimeyer Aug 10, 2023

Choose a reason for hiding this comment

Uh oh!

bastimeyer Aug 10, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bastimeyer commented Aug 14, 2023

Uh oh!

bastimeyer commented Oct 10, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants