There is a memory leak in VideoCapture with FFmpeg backend when hardware acceleration is enabled. It's about 30-50MB per hour when processing a video stream with 25 fps.

After some testing we could eliminate the memory leak by replacing the av_frame_unref function with av_frame_free at cap_ffmpeg_impl.hpp#L1522.
We don't know why av_frame_unref was used in the first place, so we don't know if using av_frame_free has some unintended side effects.

System information (version)

• OpenCV => 4.5.4
• Operating System / Platform => Ubuntu 20.04
• Compiler => gcc