This seems to have fixed it. I was having a lot of trouble reproducing this issue with Twitch streams, so if you'd like to test it yourself, I found it easier to generate a short HLS playlist using ffmpeg, create a copy with #EXT-X-ENDLIST removed, then play the playlist with streamlink and manually swap them during playback.

$ ffmpeg -i input.mp4 -c copy -hls_time 2 -hls_list_size 0 output/index_endlist.m3u8
$ sed 's/#EXT-X-ENDLIST$//' output/index_endlist.m3u8 > output/index.m3u8
$ streamlink -o /dev/null --hls-live-restart --hls-segment-queue-threshold=0.0 --hls-playlist-reload-time=segment --stream-timeout=240.0 --loglevel=all --logfile=log.txt "file://./output/index.m3u8" best &\
sleep 10 &&\
mv output/index_endlist.m3u8 output/index.m3u8 &&\
time wait

In the git master version, this will hang until stream-timeout is hit, with this commit it ends right away after the playlist swap.

I was having a lot of trouble reproducing this issue

That's what the added test is for. The added test alone fails without the changes in the HLSStreamWorker.

Thanks for validating on non-test data though. Going to merge new.