fix: preserve audio stream in process_video#2252
Merged
Borda merged 15 commits intoMay 19, 2026
Merged
Conversation
Add preserve_audio parameter to process_video. When True, copies the audio stream from source_path into target_path after frame processing using ffmpeg. Gracefully warns and skips if ffmpeg is not on PATH or returns a non-zero exit code, leaving a valid video-only output file. Closes roboflow#1923
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #2252 +/- ##
=======================================
Coverage 78% 78%
=======================================
Files 66 66
Lines 8347 8374 +27
=======================================
+ Hits 6475 6501 +26
- Misses 1872 1873 +1 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses #1923 by adding an opt-in preserve_audio parameter to process_video so that, after OpenCV frame processing (which drops audio), the original audio stream can be muxed back into the output using ffmpeg.
Changes:
- Added
_mux_audio()helper that usesffmpegto mux audio fromsource_pathinto the processedtarget_path. - Extended
process_video(..., preserve_audio: bool = False)to optionally invoke_mux_audioafter writing frames. - Added tests verifying the default behavior, the mux call path, and graceful handling when
ffmpegis missing or fails.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
src/supervision/utils/video.py |
Adds _mux_audio and wires preserve_audio into process_video. |
tests/utils/test_video.py |
Adds unit tests for the new audio-preservation behavior and failure modes. |
…ame-dir for atomic replace - Move `tempfile.mkstemp` + `os.close` inside `try` block so OSError (disk full, unwritable dir) is caught by the existing `except Exception` handler instead of propagating to the caller, preserving the warn-and-degrade contract - Pass `dir=os.path.dirname(os.path.abspath(video_path))` so the temp file is on the same filesystem as the output, restoring `os.rename` semantics in `shutil.move` - Initialise `tmp_path = None` before `try`; guard `finally` with `tmp_path is not None` to satisfy mypy and avoid referencing an unbound name --- Co-authored-by: Claude Code <noreply@anthropic.com>
…r (report: .reports/review/2026-05-19T10-05-17Z/review-report.md): add timeout=300 to subprocess.run Without a timeout, ffmpeg can hang indefinitely on malformed containers or stalled filesystems, silently freezing the caller's program. The existing `except Exception` block already catches `subprocess.TimeoutExpired`, so adding `timeout=300` is the minimal fix — it logs a warning and returns. 300 s is a conservative cap sufficient for typical video remux workloads. --- Co-authored-by: Claude Code <noreply@anthropic.com>
…ream mapping -map 1:a:0? With `-map 1:a:0` (no `?`), ffmpeg exits non-zero when source_path has no audio stream, triggering a spurious failure warning even though silence is a valid input. The `?` suffix makes the mapping optional: ffmpeg continues without audio instead of failing, so preserve_audio=True is a no-op on video-only sources rather than a false failure. --- Co-authored-by: Claude Code <noreply@anthropic.com>
…ess output and log stderr on failure - Add `-loglevel error -nostats` so ffmpeg only writes actual errors to stderr (eliminates progress/stats spam that would buffer in PIPE indefinitely) - Decode `result.stderr` and include it in the warning when ffmpeg exits non-zero, so failure messages surface diagnostically instead of being discarded --- Co-authored-by: Claude Code <noreply@anthropic.com>
…be (report: .reports/review/2026-05-19T10-05-17Z/review-report.md): fix docstring example in process_video - Change bare `process_video(...)` call to `sv.process_video(...)` so the example matches the public API pattern and does not raise NameError for users copying the snippet - Remove unused `import cv2` which was never referenced in the example body --- Co-authored-by: Claude Code <noreply@anthropic.com>
…be (report: .reports/review/2026-05-19T10-05-17Z/review-report.md): document silent-degrade contract for preserve_audio - Clarify that missing/failing ffmpeg warns and continues rather than raising - Add install hint for ffmpeg (apt/brew) - Note that audio is truncated to match the processed video duration (-shortest) --- Co-authored-by: Claude Code <noreply@anthropic.com>
…alist (report: .reports/review/2026-05-19T10-05-17Z/review-report.md): use module-qualified patch targets
Change `patch("shutil.which", ...)` → `patch("supervision.utils.video.shutil.which", ...)`
and `patch("subprocess.run", ...)` → `patch("supervision.utils.video.subprocess.run", ...)`
to follow project convention and be robust to future `from shutil import which` refactors.
---
Co-authored-by: Claude Code <noreply@anthropic.com>
…alist (report: .reports/review/2026-05-19T10-05-17Z/review-report.md): add _mux_audio happy-path and exception-branch tests - test_mux_audio_moves_file_on_success: mock subprocess.run returncode=0; assert shutil.move is called once with video_path as destination — catches any regression that drops the move call after a successful ffmpeg run - test_mux_audio_swallows_subprocess_exception: mock subprocess.run raising OSError; assert no exception escapes _mux_audio and original file is intact - Fix failed_result.stderr = b"" in test_mux_audio_warns_on_ffmpeg_failure to match the updated _mux_audio which now decodes result.stderr --- Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…ce test - Skip _mux_audio when writer_worker.is_alive() after join timeout to avoid muxing an incomplete output file - Fix test_mux_audio_moves_file_on_success: patch os.replace (not shutil.move) to match implementation changed in 2027938 --- Co-authored-by: Claude Code <noreply@anthropic.com>
- Move four test_mux_audio_* free functions into TestMuxAudio class - Strip mux_audio_ prefix from method names; class carries the unit - Condense multi-line docstrings to single-line per testing rules --- Co-authored-by: Claude Code <noreply@anthropic.com>
- Collapse test_warns_when_ffmpeg_missing, test_warns_on_ffmpeg_failure, test_swallows_subprocess_exception into one parametrized test_file_unchanged_on_failure[ffmpeg_missing|ffmpeg_fails|subprocess_raises] --- Co-authored-by: Claude Code <noreply@anthropic.com>
- Promote two class methods back to module-level functions - Collapse nested with-patch statements into single with a, b: form --- Co-authored-by: Claude Code <noreply@anthropic.com>
Borda
approved these changes
May 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1923.
process_videowrites frames via OpenCV which discards the audio track. This adds apreserve_audioparameter (defaultFalse) that copies the audio stream fromsource_pathintotarget_pathafter frame processing usingffmpeg.preserve_audio=False(default): existing behavior unchanged, no ffmpeg dependencypreserve_audio=True: callsffmpegto mux audio in-place; warns and skips gracefully if ffmpeg is absent or failsChanges
src/supervision/utils/video.py: add_mux_audiohelper andpreserve_audioparam toprocess_videotests/utils/test_video.py: 4 new tests covering the mux call, default behavior, missing ffmpeg, and ffmpeg failureTest plan
uv run pytest tests/utils/test_video.py -v- all 14 tests passuv run pre-commit run --files src/supervision/utils/video.py tests/utils/test_video.py- all hooks passprocess_video(..., preserve_audio=True)output retains audio track