Skip to content

fix: preserve audio stream in process_video#2252

Merged
Borda merged 15 commits into
roboflow:developfrom
satishkc7:fix/process-video-audio-stream
May 19, 2026
Merged

fix: preserve audio stream in process_video#2252
Borda merged 15 commits into
roboflow:developfrom
satishkc7:fix/process-video-audio-stream

Conversation

@satishkc7

Copy link
Copy Markdown
Contributor

Summary

Fixes #1923.

process_video writes frames via OpenCV which discards the audio track. This adds a preserve_audio parameter (default False) that copies the audio stream from source_path into target_path after frame processing using ffmpeg.

  • preserve_audio=False (default): existing behavior unchanged, no ffmpeg dependency
  • preserve_audio=True: calls ffmpeg to mux audio in-place; warns and skips gracefully if ffmpeg is absent or fails

Changes

  • src/supervision/utils/video.py: add _mux_audio helper and preserve_audio param to process_video
  • tests/utils/test_video.py: 4 new tests covering the mux call, default behavior, missing ffmpeg, and ffmpeg failure

Test plan

  • uv run pytest tests/utils/test_video.py -v - all 14 tests pass
  • uv run pre-commit run --files src/supervision/utils/video.py tests/utils/test_video.py - all hooks pass
  • Manual test with a video that has audio: process_video(..., preserve_audio=True) output retains audio track

Add preserve_audio parameter to process_video. When True, copies the
audio stream from source_path into target_path after frame processing
using ffmpeg. Gracefully warns and skips if ffmpeg is not on PATH or
returns a non-zero exit code, leaving a valid video-only output file.

Closes roboflow#1923
@codecov

codecov Bot commented May 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.29630% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 78%. Comparing base (b68b41a) to head (260013d).

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #2252   +/-   ##
=======================================
  Coverage       78%     78%           
=======================================
  Files           66      66           
  Lines         8347    8374   +27     
=======================================
+ Hits          6475    6501   +26     
- Misses        1872    1873    +1     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses #1923 by adding an opt-in preserve_audio parameter to process_video so that, after OpenCV frame processing (which drops audio), the original audio stream can be muxed back into the output using ffmpeg.

Changes:

  • Added _mux_audio() helper that uses ffmpeg to mux audio from source_path into the processed target_path.
  • Extended process_video(..., preserve_audio: bool = False) to optionally invoke _mux_audio after writing frames.
  • Added tests verifying the default behavior, the mux call path, and graceful handling when ffmpeg is missing or fails.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
src/supervision/utils/video.py Adds _mux_audio and wires preserve_audio into process_video.
tests/utils/test_video.py Adds unit tests for the new audio-preservation behavior and failure modes.

Comment thread src/supervision/utils/video.py Outdated
Comment thread src/supervision/utils/video.py Outdated
Comment thread src/supervision/utils/video.py
Borda and others added 9 commits May 19, 2026 12:44
…ame-dir for atomic replace

- Move `tempfile.mkstemp` + `os.close` inside `try` block so OSError (disk full,
  unwritable dir) is caught by the existing `except Exception` handler instead of
  propagating to the caller, preserving the warn-and-degrade contract
- Pass `dir=os.path.dirname(os.path.abspath(video_path))` so the temp file is on
  the same filesystem as the output, restoring `os.rename` semantics in `shutil.move`
- Initialise `tmp_path = None` before `try`; guard `finally` with
  `tmp_path is not None` to satisfy mypy and avoid referencing an unbound name

---
Co-authored-by: Claude Code <noreply@anthropic.com>
…r (report: .reports/review/2026-05-19T10-05-17Z/review-report.md): add timeout=300 to subprocess.run

Without a timeout, ffmpeg can hang indefinitely on malformed containers or
stalled filesystems, silently freezing the caller's program. The existing
`except Exception` block already catches `subprocess.TimeoutExpired`, so
adding `timeout=300` is the minimal fix — it logs a warning and returns.

300 s is a conservative cap sufficient for typical video remux workloads.

---
Co-authored-by: Claude Code <noreply@anthropic.com>
…ream mapping -map 1:a:0?

With `-map 1:a:0` (no `?`), ffmpeg exits non-zero when source_path has no
audio stream, triggering a spurious failure warning even though silence is a
valid input. The `?` suffix makes the mapping optional: ffmpeg continues
without audio instead of failing, so preserve_audio=True is a no-op on
video-only sources rather than a false failure.

---
Co-authored-by: Claude Code <noreply@anthropic.com>
…ess output and log stderr on failure

- Add `-loglevel error -nostats` so ffmpeg only writes actual errors to stderr
  (eliminates progress/stats spam that would buffer in PIPE indefinitely)
- Decode `result.stderr` and include it in the warning when ffmpeg exits
  non-zero, so failure messages surface diagnostically instead of being discarded

---
Co-authored-by: Claude Code <noreply@anthropic.com>
…be (report: .reports/review/2026-05-19T10-05-17Z/review-report.md): fix docstring example in process_video

- Change bare `process_video(...)` call to `sv.process_video(...)` so the
  example matches the public API pattern and does not raise NameError for users
  copying the snippet
- Remove unused `import cv2` which was never referenced in the example body

---
Co-authored-by: Claude Code <noreply@anthropic.com>
…be (report: .reports/review/2026-05-19T10-05-17Z/review-report.md): document silent-degrade contract for preserve_audio

- Clarify that missing/failing ffmpeg warns and continues rather than raising
- Add install hint for ffmpeg (apt/brew)
- Note that audio is truncated to match the processed video duration (-shortest)

---
Co-authored-by: Claude Code <noreply@anthropic.com>
…alist (report: .reports/review/2026-05-19T10-05-17Z/review-report.md): use module-qualified patch targets

Change `patch("shutil.which", ...)` → `patch("supervision.utils.video.shutil.which", ...)`
and `patch("subprocess.run", ...)` → `patch("supervision.utils.video.subprocess.run", ...)`
to follow project convention and be robust to future `from shutil import which` refactors.

---
Co-authored-by: Claude Code <noreply@anthropic.com>
…alist (report: .reports/review/2026-05-19T10-05-17Z/review-report.md): add _mux_audio happy-path and exception-branch tests

- test_mux_audio_moves_file_on_success: mock subprocess.run returncode=0;
  assert shutil.move is called once with video_path as destination — catches
  any regression that drops the move call after a successful ffmpeg run
- test_mux_audio_swallows_subprocess_exception: mock subprocess.run raising
  OSError; assert no exception escapes _mux_audio and original file is intact
- Fix failed_result.stderr = b"" in test_mux_audio_warns_on_ffmpeg_failure
  to match the updated _mux_audio which now decodes result.stderr

---
Co-authored-by: Claude Code <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment thread src/supervision/utils/video.py Outdated
Comment thread src/supervision/utils/video.py Outdated
Borda and others added 5 commits May 19, 2026 13:19
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…ce test

- Skip _mux_audio when writer_worker.is_alive() after join timeout to avoid
  muxing an incomplete output file
- Fix test_mux_audio_moves_file_on_success: patch os.replace (not shutil.move)
  to match implementation changed in 2027938

---
Co-authored-by: Claude Code <noreply@anthropic.com>
- Move four test_mux_audio_* free functions into TestMuxAudio class
- Strip mux_audio_ prefix from method names; class carries the unit
- Condense multi-line docstrings to single-line per testing rules

---
Co-authored-by: Claude Code <noreply@anthropic.com>
- Collapse test_warns_when_ffmpeg_missing, test_warns_on_ffmpeg_failure,
  test_swallows_subprocess_exception into one parametrized
  test_file_unchanged_on_failure[ffmpeg_missing|ffmpeg_fails|subprocess_raises]

---
Co-authored-by: Claude Code <noreply@anthropic.com>
- Promote two class methods back to module-level functions
- Collapse nested with-patch statements into single with a, b: form

---
Co-authored-by: Claude Code <noreply@anthropic.com>
@Borda Borda merged commit 7e28315 into roboflow:develop May 19, 2026
26 checks passed
@Borda Borda mentioned this pull request Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: Audio stream not captured in process_video

3 participants