Abstract	WebRTC RTX 笔记
Authors	Walter Fan
Category	learning note
Status	v1.0
Updated	2020-08-28
License	CC-BY-NC-ND 4.0

什么是 RTX

RTX 就是重传 Retransmission, 将丢失的包重新由发送方传给接收方。

Webrtc 默认开启 RTX (重传)，它一般采用不同的 SSRC 进行传输，即原始的 RTP 包和重传的 RTP 包的 SSRC 是不同的，这样不会干扰原始 RTP 包的度量。

RTX 包的 Payload 在 RFC4588 中有详细描述，一般 NACK 导致的重传包和 Bandwidth Probe 导致的探测包也可能走 RTX 通道。

为什么用 RTX

媒体流多使用 RTP 通过 UDP 进行传输，由于是不可靠传输，丢包是不可避免，也是可以容忍的，但是对于一些关键数据是不能丢失的，这时候就需要重传(RTX)。

在 WebRTC 中常用的 QoS 策略有

反馈：例如 PLI , NACK
冗余，例如 FEC, RTX
调整：例如码率，分辨率及帧率的调整
缓存: 例如 Receive Adaptive Jitter Buffer, Send Buffer

这些措施一般需要结合基于拥塞控制(congestion control) 及带宽估计(bandwidth estimation)技术, 不同的网络条件下需要采用不同的措施。

FEC 用作丢包恢复需要占用更多的带宽，即使 5% 的丢包需要几乎一倍的带宽，在带宽有限的情况下可能会使情况更糟。

RTX 不会占用太多的带宽，接收方发送 NACK 指明哪些包丢失了，发送方通过单独的 RTP 流(不同的 SSRC)来重传丢失的包，但是 RTX 至少需要一个 RTT 来修复丢失的包。

音频流对于延迟很敏感，而且占用带宽不多，所以用 FEC 更好。WebRTC 默认并不为 audio 开启 RTX
视频流对于延迟没那么敏感，而且占用带宽很多，所以用 RTX 更好。

RTX 相关的信令

RTX 的信令层主要是由发送方通过 SDP 告知接收方我支持 RTX 特性，并且约定原始包和重传包之间的关系由什么方式指定。

现在常用的方式有三种

APT - Associated Payload Type 关联荷载类型 - Chrome, Edge, Firefox, Safari 都支持
RID/RRID - RTP Stream Id 和 Repaired RTP Stream Id - - Chrome, Edge, Safari 支持, Firefox 不支持
SSRC Group - SSRC 分组 - Firefox 支持，其他三个现在优先用 rid/rrid

SDP Extensions

1) Associated Payload Type

在SDP 中可以指定 RTP 流所关联的 RTX 流的荷载类型 Associated Payload Type，参照 RFC 4588, 期望在 SDP 中有如下属性

a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96;rtx-time=3000
for examplev=0
o=mascha 2980675221 2980675778 IN IP4 host.example.net
c=IN IP4 192.0.2.0
m=video 49170 RTP/AVPF 96 97
a=rtpmap:96 MP4V-ES/90000
a=rtcp-fb:96 nack
a=fmtp:96 profile-level-id=8;config=01010000012000884006682C2090A21F
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96;rtx-time=3000

2) RID and RRID

As RFC 8853, 约定 RTP 包中增加 rid 和 rrid 的扩展头

a=extmap:2 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=simulcast...
a=rid:<rid-id> <direction> [pt=<fmt-list>;]<restriction>=<value>...

direction 可以是 send 或者 recv，pt 包含相关的 payload type, restriction 是指一些编码约束, 详情参见 RFC8851

3) SSRC-Group

还有一个方法就是 SSRC Group, 将相互之间有关联关系的媒体流的 SSRC 编配成一个个小组

1. FID SSRC-group for RTX

举例如下

a=ssrc:659652645 cname:Taj3/ieCnLbsUFoH
a=ssrc:659652645 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:659652645 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk
a=ssrc:659652645 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:98148385 cname:Taj3/ieCnLbsUFoH
a=ssrc:98148385 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:98148385 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk
a=ssrc:98148385 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc-group:FID 659652645 98148385

2. SIM SSRC-group for Simulcast

Simulcast 联播结合 RTX , 可做如下所示例中的分组

a=ssrc:659652645 cname:Taj3/ieCnLbsUFoH
a=ssrc:659652645 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:659652645 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk
a=ssrc:659652645 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:98148385 cname:Taj3/ieCnLbsUFoH
a=ssrc:98148385 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:98148385 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk
a=ssrc:98148385 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:1982135572 cname:Taj3/ieCnLbsUFoH
a=ssrc:1982135572 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:1982135572 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk
a=ssrc:1982135572 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:2523084908 cname:Taj3/ieCnLbsUFoH
a=ssrc:2523084908 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:2523084908 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk
a=ssrc:2523084908 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:3604909222 cname:Taj3/ieCnLbsUFoH
a=ssrc:3604909222 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:3604909222 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk
a=ssrc:3604909222 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:1893605472 cname:Taj3/ieCnLbsUFoH
a=ssrc:1893605472 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc:1893605472 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk
a=ssrc:1893605472 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb
a=ssrc-group:SIM 659652645 1982135572 3604909222
a=ssrc-group:FID 659652645 98148385
a=ssrc-group:FID 1982135572 2523084908
a=ssrc-group:FID 3604909222 1893605472

RTP 头扩展

根据 RFC8852: RTP Stream Identifier Source Description (SDES) 中的定义，RID 和 RRID 的扩展头格式如下

RtpStreamId 对每个 RTP stream 都是不同的(类似于 SSRC , 在RTP Session 中需要保持唯一性)

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|RtpStreamId=12 |     length    | RtpStreamId                 ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

RepairedRtpStreamId 只会出现在 Repair RTP Streams 中, 指明它所修复的 RTP 流的 rid

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Repaired...=13 |     length    | RepairRtpStreamId           ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

RTX 媒体包的格式

RFC4588 - "RTP Retransmission Payload Format" 中描述了 RTX RTP 包的格式。

RTP 头中会包含上面所述的 rrid
RTP 荷载中会有一个 OSN ，对应原始 RTP 包中的 sequence number

0                   1                   2                   30 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                         RTP Header                            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|            OSN                |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
|                  Original RTP Packet Payload                  |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

例如

SDP 中指定了 rid 的值和扩展头的标识

a=rid:1 send
a=rid:2 send
a=rid:3 send
a=simulcast:send 1;2;3a=extmap:8/sendrecv http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:4/sendrecv urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5/sendrecv urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:7/sendrecv urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id

原始的 RTP 包的格式如下

Real-Time Transport Protocol
[Stream setup by HEUR RTP (frame 62)]
10.. .... = Version: RFC 1889 Version (2)
..0. .... = Padding: False
...1 .... = Extension: True
.... 0000 = Contributing source identifiers count: 0
1... .... = Marker: True
Payload type: DynamicRTP-Type-97 (97)
Sequence number: 27303
[Extended sequence number: 92839]
Timestamp: 3417222624
Synchronization Source identifier: 0x9100cc9c (2432748700)
Defined by profile: Unknown (0xbede)
Extension length: 2
Header extensions
RFC 5285 Header Extension (One-Byte Header)
Identifier: 8
Length: 3
Extension Data: 6e8c4a
RFC 5285 Header Extension (One-Byte Header)
Identifier: 4
Length: 1
Extension Data: 30
RFC 5285 Header Extension (One-Byte Header)
Identifier: 5
Length: 1
Extension Data: 31
Payload: 9a2ba3655796f772c2c0159bd6570fb896b7f95142362c29381d926f75cf8c364f927912…

RTX RTP 包的格式如下

Real-Time Transport Protocol
[Stream setup by HEUR RTP (frame 62)]
10.. .... = Version: RFC 1889 Version (2)
..0. .... = Padding: False
...1 .... = Extension: True
.... 0000 = Contributing source identifiers count: 0
0... .... = Marker: False
Payload type: DynamicRTP-Type-124 (124)
Sequence number: 7863
[Extended sequence number: 73399]
Timestamp: 3417198504
Synchronization Source identifier: 0x58b41246 (1488196166)
Defined by profile: Unknown (0xbede)
Extension length: 2
Header extensions
RFC 5285 Header Extension (One-Byte Header)
Identifier: 8
Length: 3
Extension Data: 6e051f
RFC 5285 Header Extension (One-Byte Header)
Identifier: 4
Length: 1
Extension Data: 30
RFC 5285 Header Extension (One-Byte Header)
Identifier: 7
Length: 1
Extension Data: 31
Payload: 9d41d0efd4d67217f916c5854544005a847a64f0936f6620873be35ba26fb2ddfe465015…

WebRTC 是怎么实现 RTX 的

在 WebRTC 中，主要实现在两个方面

1)接收端生成 NACK：检查 Sequence Number , 如果发现有丢包，并且在合理范围之内，就会生成 NACK 包给发送方，要求重传。

NACK 包格式参见 RFC4585#page-34

0                   1                   2                   30 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|    1    |       205     |          length               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                  SSRC of packet sender                        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                  SSRC of media source                         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|            PID(SN)            |             BLP               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

BLP: 是指位掩码，bit 位为1 表示这个包丢失了
( bitmask of following lost packets 16bits, bit_i=1: lost )

在 SDP 中可以指定RTX 所支持的时长, 如果没有，那么 WebRTC 在发送端会维持一个所发送包的默认的长度，

a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96;rtx-time=3000

2) 发送端处理 NACK, 并发送 RTX 包

当收到 NACK 请求时

OnReceivedNack

void RTPSender::OnReceivedNack(const std::vector<uint16_t>& nack_sequence_numbers,int64_t avg_rtt) {packet_history_->SetRtt(TimeDelta::Millis(5 + avg_rtt));for (uint16_t seq_no : nack_sequence_numbers) {const int32_t bytes_sent = ReSendPacket(seq_no);if (bytes_sent < 0) {// Failed to send one Sequence number. Give up the rest in this nack.RTC_LOG(LS_WARNING) << "Failed resending RTP packet " << seq_no<< ", Discard rest of packets.";break;}}
}

于是，从发送历史中找到 NACK 中指明的包，构建 RTX 包以重传

nt32_t RTPSender::ReSendPacket(uint16_t packet_id) {int32_t packet_size = 0;const bool rtx = (RtxStatus() & kRtxRetransmitted) > 0;std::unique_ptr<RtpPacketToSend> packet =packet_history_->GetPacketAndMarkAsPending(packet_id, [&](const RtpPacketToSend& stored_packet) {// Check if we're overusing retransmission bitrate.// TODO(sprang): Add histograms for nack success or failure// reasons.packet_size = stored_packet.size();std::unique_ptr<RtpPacketToSend> retransmit_packet;if (retransmission_rate_limiter_ &&!retransmission_rate_limiter_->TryUseRate(packet_size)) {return retransmit_packet;}if (rtx) {retransmit_packet = BuildRtxPacket(stored_packet);} else {retransmit_packet =std::make_unique<RtpPacketToSend>(stored_packet);}if (retransmit_packet) {retransmit_packet->set_retransmitted_sequence_number(stored_packet.SequenceNumber());}return retransmit_packet;});if (packet_size == 0) {// Packet not found or already queued for retransmission, ignore.RTC_DCHECK(!packet);return 0;}if (!packet) {// Packet was found, but lambda helper above chose not to create// `retransmit_packet` out of it.return -1;}packet->set_packet_type(RtpPacketMediaType::kRetransmission);packet->set_fec_protect_packet(false);std::vector<std::unique_ptr<RtpPacketToSend>> packets;packets.emplace_back(std::move(packet));paced_sender_->EnqueuePackets(std::move(packets));return packet_size;
}

构建 RTX 包

std::unique_ptr<RtpPacketToSend> RTPSender::BuildRtxPacket(const RtpPacketToSend& packet) {std::unique_ptr<RtpPacketToSend> rtx_packet;// Add original RTP header.{MutexLock lock(&send_mutex_);if (!sending_media_)return nullptr;RTC_DCHECK(rtx_ssrc_);// Replace payload type.auto kv = rtx_payload_type_map_.find(packet.PayloadType());if (kv == rtx_payload_type_map_.end())return nullptr;rtx_packet = std::make_unique<RtpPacketToSend>(&rtp_header_extension_map_,max_packet_size_);rtx_packet->SetPayloadType(kv->second);// Replace SSRC.rtx_packet->SetSsrc(*rtx_ssrc_);CopyHeaderAndExtensionsToRtxPacket(packet, rtx_packet.get());// RTX packets are sent on an SSRC different from the main media, so the// decision to attach MID and/or RRID header extensions is completely// separate from that of the main media SSRC.//// Note that RTX packets must used the RepairedRtpStreamId (RRID) header// extension instead of the RtpStreamId (RID) header extension even though// the payload is identical.if (always_send_mid_and_rid_ || !rtx_ssrc_has_acked_) {// These are no-ops if the corresponding header extension is not// registered.if (!mid_.empty()) {rtx_packet->SetExtension<RtpMid>(mid_);}if (!rid_.empty()) {rtx_packet->SetExtension<RepairedRtpStreamId>(rid_);}}}RTC_DCHECK(rtx_packet);uint8_t* rtx_payload =rtx_packet->AllocatePayload(packet.payload_size() + kRtxHeaderSize);if (rtx_payload == nullptr)return nullptr;// Add OSN (original sequence number).ByteWriter<uint16_t>::WriteBigEndian(rtx_payload, packet.SequenceNumber());// Add original payload data.auto payload = packet.payload();memcpy(rtx_payload + kRtxHeaderSize, payload.data(), payload.size());// Add original additional data.rtx_packet->set_additional_data(packet.additional_data());// Copy capture time so e.g. TransmissionOffset is correctly set.rtx_packet->set_capture_time(packet.capture_time());return rtx_packet;
}static void CopyHeaderAndExtensionsToRtxPacket(const RtpPacketToSend& packet,RtpPacketToSend* rtx_packet) {// Set the relevant fixed packet headers. The following are not set:// * Payload type - it is replaced in rtx packets.// * Sequence number - RTX has a separate sequence numbering.// * SSRC - RTX stream has its own SSRC.rtx_packet->SetMarker(packet.Marker());rtx_packet->SetTimestamp(packet.Timestamp());// Set the variable fields in the packet header:// * CSRCs - must be set before header extensions.// * Header extensions - replace Rid header with RepairedRid header.const std::vector<uint32_t> csrcs = packet.Csrcs();rtx_packet->SetCsrcs(csrcs);for (int extension_num = kRtpExtensionNone + 1;extension_num < kRtpExtensionNumberOfExtensions; ++extension_num) {auto extension = static_cast<RTPExtensionType>(extension_num);// Stream ID header extensions (MID, RSID) are sent per-SSRC. Since RTX// operates on a different SSRC, the presence and values of these header// extensions should be determined separately and not blindly copied.if (extension == kRtpExtensionMid ||extension == kRtpExtensionRtpStreamId) {continue;}// Empty extensions should be supported, so not checking `source.empty()`.if (!packet.HasExtension(extension)) {continue;}rtc::ArrayView<const uint8_t> source = packet.FindExtension(extension);rtc::ArrayView<uint8_t> destination =rtx_packet->AllocateExtension(extension, source.size());// Could happen if any:// 1. Extension has 0 length.// 2. Extension is not registered in destination.// 3. Allocating extension in destination failed.if (destination.empty() || source.size() != destination.size()) {continue;}std::memcpy(destination.begin(), source.begin(), destination.size());}
}

3) 接收端收 RTX packet，重新构建 media packet , 找到对应的 media stream ，放入其接收缓冲

https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/call/rtx_receive_stream.cc;l=35

void RtxReceiveStream::OnRtpPacket(const RtpPacketReceived& rtx_packet) {RTC_DCHECK_RUN_ON(&packet_checker_);if (rtp_receive_statistics_) {rtp_receive_statistics_->OnRtpPacket(rtx_packet);}rtc::ArrayView<const uint8_t> payload = rtx_packet.payload();if (payload.size() < kRtxHeaderSize) {return;}auto it = associated_payload_types_.find(rtx_packet.PayloadType());if (it == associated_payload_types_.end()) {RTC_DLOG(LS_VERBOSE) << "Unknown payload type "<< static_cast<int>(rtx_packet.PayloadType())<< " on rtx ssrc " << rtx_packet.Ssrc();return;}RtpPacketReceived media_packet;media_packet.CopyHeaderFrom(rtx_packet);media_packet.SetSsrc(media_ssrc_);media_packet.SetSequenceNumber((payload[0] << 8) + payload[1]);media_packet.SetPayloadType(it->second);media_packet.set_recovered(true);media_packet.set_arrival_time(rtx_packet.arrival_time());// Skip the RTX header.rtc::ArrayView<const uint8_t> rtx_payload = payload.subview(kRtxHeaderSize);uint8_t* media_payload = media_packet.AllocatePayload(rtx_payload.size());RTC_DCHECK(media_payload != nullptr);memcpy(media_payload, rtx_payload.data(), rtx_payload.size());media_sink_->OnRtpPacket(media_packet);
}

存在问题

现在 WebRTC Library 对于 rrid(RepairedRtpStreamId) 的支持并不完善，仍然需要用 SSRC-Group 来指明 RTX stream 所使用的 SSRC , 然后才能进行丢包恢复，参见 Issue 10297: RTX does not work if SSRCs are not negotiated

参考资料

RFC4588: RTP Retransmission Payload Format
RFC4585: Extended RTP Profile for RTCP-Based Feedback
RFC8851: RTP Payload Format Restrictions
RFC8852: RTP Stream Identifier Source Description (SDES)
RFC8853: Using Simulcast in Session Description Protocol (SDP) and RTP Sessions
Implement RTX for WebRTC
https://w3c.github.io/webrtc-pc/#simulcast-functionality

本作品采用知识共享署名-非商业性使用-禁止演绎 4.0 国际许可协议进行许可

http://chatgpt.dhexx.cn/article/INWN4RYg.shtml

红与蓝：现代Webshell检测引擎免杀对抗与实践

上半年Webshell话题很火，业界举办了数场对抗挑战赛，也发布了多篇站在安全产品侧，着重查杀思路的精彩文章，但鲜有看到以蓝军视角为主的paper。作为多场挑战赛的参赛者及内部红蓝对抗的参与者，笔者试着站在蓝军角度，聊聊现代Webshell对抗的一些思路，也以PHP Webshell为例…

WebRTC 之 RTX