Reproduce steps:
Run SRS by rtc2rtmp.conf
Run OBS and publish WHIP to SRS.
Play HTTP-FLV, failed. http://localhost:8080/players/srs_player.html
No sequence header for RTMP or HTTP-FLV or HLS.
It appears that some issues need to be resolved in order to support the transmuxing of WebRTC to RTMP. The following is a pcap file captured by Wireshark, which includes ICE and RTP packets produced by OBS WHIP:
whip-ice-rtp.pcapng.zip
There is no RTCP Sender Report (SR) available to synchronize the audio and video timing. By default, SRS will drop the packet. For more information, refer to RTC: Fix rtc to rtmp sync timestamp using sender report. #2470
The SPS and PPS are sent as separate NALUs, with each NALU contained in an individual RTP packet. This causes a failure in constructing the sequence header from the RTP packet.
The IDR frame is divided into multiple slices, which must be combined to form a video frame for RTMP.
Let's examine an example to understand how Chrome sends RTCP SR and how SRS synchronizes the timestamp.
srs.log.zip
Refer to issue #2470 for using NTP in RTCP SR to synchronize the timestamp. At startup, the timestamp has not yet been synchronized:
[2023-09-05 18:07:07.050] Audio async rate=0, rtp=4098464126
[2023-09-05 18:07:07.061] Video async rate=0, rtp=2819409850
It takes approximately 5 seconds to receive the SR and synchronize the timestamp, which is:
[2023-09-05 18:07:12.592] Video async rate=90, rtp=2819907460, corrected=3065188167
[2023-09-05 18:07:12.609] Audio async rate=48, rtp=4098731006, corrected=3065188190
(2819907460-2819409850)/90. = 5529.0
(4098731006-4098464126)/48. = 5560.0
Note: We also use timebase for calculating timestamps, with 90k for video and 48k for audio.
About 30s later, the timestamp:
[2023-09-05 18:07:37.313] Video async rate=90, rtp=2822132440, corrected=3065212889
[2023-09-05 18:07:37.329] Audio async rate=48, rtp=4099917566, corrected=3065212908
(2822132440-2819409850)/90. = 30251.0
(4099917566-4098464126)/48. = 30280.0
About 1m, the timestamp:
[2023-09-05 18:08:07.201] Video async rate=90, rtp=2824822270, corrected=3065242775
[2023-09-05 18:08:07.210] Audio async rate=48, rtp=4101351806, corrected=3065242787
(2824822270-2819409850)/90. = 60138.0
(4101351806-4098464126)/48. = 60160.0
About 2m, the timestamp:
[2023-09-05 18:09:07.430] Audio async rate=48, rtp=4104242366, corrected=3065303005
[2023-09-05 18:09:07.444] Video async rate=90, rtp=2830243960, corrected=3065303016
(2830243960-2819409850)/90. = 120379.0
(4104242366-4098464126)/48. = 120380.0
About 12m, the timestamp:
[2023-09-05 18:19:07.531] Audio async rate=48, rtp=4133047166, corrected=3065903092
[2023-09-05 18:19:07.544] Video async rate=90, rtp=2884251520, corrected=3065903100
(4133047166-4098464126)/48. = 720480.0
(2884251520-2819409850)/90. = 720463.0
About 17m, the timestamp:
[2023-09-05 18:24:51.393] Video async rate=90, rtp=2915198560, corrected=3066246956
[2023-09-05 18:24:51.396] Audio async rate=48, rtp=4149553406, corrected=3066246963
(2915198560-2819409850)/90. = 1064319.0
(4149553406-4098464126)/48. = 1064360.0
Please see this post for detail.
Reproduce steps:
No sequence header for RTMP or HTTP-FLV or HLS.
Issues
It appears that some issues need to be resolved in order to support the transmuxing of WebRTC to RTMP. The following is a pcap file captured by Wireshark, which includes ICE and RTP packets produced by OBS WHIP:
whip-ice-rtp.pcapng.zip
SR NTP Sync
Let's examine an example to understand how Chrome sends RTCP SR and how SRS synchronizes the timestamp.
srs.log.zip
Refer to issue #2470 for using NTP in RTCP SR to synchronize the timestamp. At startup, the timestamp has not yet been synchronized:
It takes approximately 5 seconds to receive the SR and synchronize the timestamp, which is:
About 30s later, the timestamp:
About 1m, the timestamp:
About 2m, the timestamp:
About 12m, the timestamp:
About 17m, the timestamp:
Please see this post for detail.