VideoCapture FFMpeg RTSP low fps fixes#22248
Conversation
…pture::open(). Reset interupt timer in grab if err = avformat_find_stream_info(ic, NULL); is interupted but open is successful.
|
@asmorkalov I should add that setting the number of threads is not just an RTSP issue/requirement. It seems like a feature that would be useful when streaming fron n camera's and/or when using raw reads ( For example on a mobile i7-12700H paried with an RTX 3070 Ti I can decode upwards of 60 1080p streams using |
alalek
left a comment
There was a problem hiding this comment.
BTW, FFmpeg emits warning about threads on i7-12700K (20 threads)
| //ic->start_time_realtime is in microseconds | ||
| return ((double)ic->start_time_realtime); | ||
| case CAP_PROP_N_THREADS: | ||
| return static_cast<double>(nThreads); |
There was a problem hiding this comment.
Could we retrive enc->thread_count value here?
There was a problem hiding this comment.
BTW, FFmpeg emits warning about threads on i7-12700K (20 threads)
I don't see a warning on a i7-12700H from cv::VideoCapture with FFMpeg backend do you mean the ffmpeg application?
Could we retrive
enc->thread_countvalue here?
Yes, that's a much better idea as the requested value can be overidden in calls to avcodec_open2.
There was a problem hiding this comment.
Try OPENCV_FFMPEG_DEBUG=1 env variable.
There was a problem hiding this comment.
Cool cheers,
[OPENCV:FFMPEG:24] Application has requested 20 threads. Using a thread count greater than 16 is not recommended.
The comment
https://github.com/FFmpeg/FFmpeg/blob/70f06dd63c9967881c399662646cc593cc5920fb/libavcodec/pthread_internal.h#L24
implies this is h264 only but I get the warning for all file types. Do you think slice threading is refering to time slice threading and performance or actual errors in the encode/decode?
Do you think this should be hardcoded (const int nCpus = min(16,get_number_of_cpus());), it may change and its possible that using more than the default number of threads may have a performance advantage at least for encoding.
https://trac.ffmpeg.org/ticket/8694
modules/videoio/test/test_ffmpeg.cpp
Outdated
| }; | ||
|
|
||
| INSTANTIATE_TEST_CASE_P(/**/, videoio_container, testing::ValuesIn(videoio_container_params)); | ||
| INSTANTIATE_TEST_CASE_P(/**/, videoio_container, testing::Combine(testing::ValuesIn(videoio_container_params), THREADS)); |
There was a problem hiding this comment.
- We prefer to keep already existed tests scenarios without changes.
- Added tests should be reduced (we don't want to increase the test time). Keep them disabled if they doesn't increase code coverage.
I believe it is enough to add one simple test for the new property.
…lude test combining thread change and raw read in the newly added videoio_read test case.
alalek
left a comment
There was a problem hiding this comment.
LGTM 👍 Thank you for the contribution!
* Allow the number of threads FFMpeg uses to be selected during VideoCapture::open(). Reset interupt timer in grab if err = avformat_find_stream_info(ic, NULL); is interupted but open is successful. * Correct the returned number of threads and amend test cases. * Update container test case. * Reverse changes added to existing videoio_container test case and include test combining thread change and raw read in the newly added videoio_read test case.
When using
VideoCapturewith the FFMpeg backend to stream from an RTSP source at a low frame rate on a machine with a large number of CPU cores,VideoCapture::read/grabcan fail due to the interrupt timer exceedingtimeout_after_ms. To try and fix this, this PR adds theCAP_PROP_N_THREADSwhich can be set on open to reduce the number of threads in this case. This is related to #20002 but I cannot tell if it fixes the issue as I am unable to re-create it on a machine with reading from a file with only 20 CPU cores.In addition to this
VideoCapture::opencan get stuck and therefore interupted on the call toerr = avformat_find_stream_info(ic, NULL);when streaming from an RTSP source at a low frame rate regardless of the number of CPU cores. The call toopenstill succeeds and the frame can be read but because the flaginterrupt_metadata.timeouthas been set inopenthe call tograb/readfails. To fix this theinterrupt_metadata.timeoutflag is reset on entry tograb/read.Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.