Skip to content

SegmentedStreamWriter.close() does not reliably finish before CLI exits (race condition) #4516

@yhhsin

Description

@yhhsin

Checklist

Streamlink version

Latest build from the master branch

Description

Background

I was doing some works on latest master branch, adding some feature on Segmented/HLS streams, and it requires some cleanup, so I added them in the end of SegmentedStreamWriter.close(), like

        self.closed = True
        self.reader.close()
        self.executor.shutdown(wait=True, cancel_futures=True)

        __my_extra_cleanup()

And I found my cleanup code does not always being executed, when HLS streams end normally.

Problem

When an HLS stream ends normally, the whole shutdown process is triggered by the last line of SegmentedStreamWriter.run(), self.close(). Then SegmentedStreamWriter.close() calls SegmentedStreamReader.close(), then the iteration loop of stream_cli.main:read_stream() will reach its end, then main() exits.

The problem is, SegmentedStreamWriter.run() -> SegmentedStreamWriter.close() was run in a separated thread, which means SegmentedStreamWriter.close() cannot reliably finish its work before main thread exits.

To reproduce

To reliably trigger the problem, adding a sleep to original SegmentedStreamWriter.close(), like

        self.closed = True
        self.reader.close()
        self.executor.shutdown(wait=True, cancel_futures=True)

        time.sleep(3)
        log.debug("SegmentedStreamWriter.close() ends")

Then run the CLI with a short HLS stream, the SegmentedStreamWriter.close() ends message never appears.

Debug log

......

[cli][info] Stream ended
[cli][info] Closing currently open stream...

Expected result

......

[cli][info] Stream ended
[cli][info] Closing currently open stream...
[stream.segmented][debug] SegmentedStreamWriter.close() ends

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions