MP4 DVR: Fix audio/video synchronization issues in WebRTC recordings. v6.0.172 v7.0.52#4230
Merged
winlinvip merged 5 commits intoossrs:developfrom Aug 12, 2025
Merged
Conversation
195435d to
97e2b64
Compare
2d8b405 to
45734e7
Compare
winlinvip
approved these changes
Aug 12, 2025
winlinvip
added a commit
that referenced
this pull request
Aug 12, 2025
… v6.0.172 v7.0.52 (#4230) Fixes #3993 - WebRTC streams recorded to MP4 via DVR exhibit audio/video synchronization issues, with audio typically ahead of video. **Note: This issue is specific to MP4 format; FLV recordings are not affected.** When WebRTC streams are converted to RTMP and then muxed to MP4, the audio and video tracks may start at different timestamps. The MP4 muxer was not accounting for this timing offset between the first audio and video samples in the STTS (Sample Time-to-Sample) table, causing the tracks to be misaligned in the final MP4 file. Introduces `SrsMp4DvrJitter` class specifically for MP4 audio/video synchronization: - **Timestamp Tracking**: Records the DTS of the first audio and video samples - **Offset Calculation**: Computes the timing difference between track start times - **MP4 STTS Correction**: Sets appropriate `sample_delta` values in the MP4 STTS table to maintain proper A/V sync - Added `SrsMp4DvrJitter` class in `srs_kernel_mp4.hpp/cpp` - Integrated jitter correction into `SrsMp4SampleManager::write_track()` for MP4 format only - Added comprehensive unit tests covering various timing scenarios - **Scope**: Changes are isolated to MP4 kernel code and do not affect FLV processing This fix ensures that MP4 DVR recordings from WebRTC streams maintain proper audio/video synchronization regardless of the relative timing of the first audio and video frames, while leaving FLV format processing unchanged. --------- Co-authored-by: Haibo Chen <495810242@qq.com> Co-authored-by: john <hondaxiao@tencent.com> Co-authored-by: winlin <winlinvip@gmail.com> Co-authored-by: OSSRS-AI <winlinam@gmail.com>
winlinvip
added a commit
that referenced
this pull request
Aug 12, 2025
… v6.0.172 (#4230) Fixes #3993 - WebRTC streams recorded to MP4 via DVR exhibit audio/video synchronization issues, with audio typically ahead of video. **Note: This issue is specific to MP4 format; FLV recordings are not affected.** When WebRTC streams are converted to RTMP and then muxed to MP4, the audio and video tracks may start at different timestamps. The MP4 muxer was not accounting for this timing offset between the first audio and video samples in the STTS (Sample Time-to-Sample) table, causing the tracks to be misaligned in the final MP4 file. Introduces `SrsMp4DvrJitter` class specifically for MP4 audio/video synchronization: - **Timestamp Tracking**: Records the DTS of the first audio and video samples - **Offset Calculation**: Computes the timing difference between track start times - **MP4 STTS Correction**: Sets appropriate `sample_delta` values in the MP4 STTS table to maintain proper A/V sync - Added `SrsMp4DvrJitter` class in `srs_kernel_mp4.hpp/cpp` - Integrated jitter correction into `SrsMp4SampleManager::write_track()` for MP4 format only - Added comprehensive unit tests covering various timing scenarios - **Scope**: Changes are isolated to MP4 kernel code and do not affect FLV processing This fix ensures that MP4 DVR recordings from WebRTC streams maintain proper audio/video synchronization regardless of the relative timing of the first audio and video frames, while leaving FLV format processing unchanged. --------- Co-authored-by: Haibo Chen <495810242@qq.com> Co-authored-by: john <hondaxiao@tencent.com> Co-authored-by: winlin <winlinvip@gmail.com> Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Fixes #3993 - WebRTC streams recorded to MP4 via DVR exhibit audio/video synchronization issues, with audio typically ahead of video. Note: This issue is specific to MP4 format; FLV recordings are not affected.
Root Cause
When WebRTC streams are converted to RTMP and then muxed to MP4, the audio and video tracks may start at different timestamps. The MP4 muxer was not accounting for this timing offset between the first audio and video samples in the STTS (Sample Time-to-Sample) table, causing the tracks to be misaligned in the final MP4 file.
Solution
Introduces
SrsMp4DvrJitterclass specifically for MP4 audio/video synchronization:sample_deltavalues in the MP4 STTS table to maintain proper A/V syncKey Changes
SrsMp4DvrJitterclass insrs_kernel_mp4.hpp/cppSrsMp4SampleManager::write_track()for MP4 format onlyThis fix ensures that MP4 DVR recordings from WebRTC streams maintain proper audio/video synchronization regardless of the relative timing of the first audio and video frames, while leaving FLV format processing unchanged.
Co-authored-by: Haibo Chen 495810242@qq.com
Co-authored-by: john hondaxiao@tencent.com