Created
May 27, 2026 02:54
-
-
Save jrmuizel/2c95d4770be3a97f0240a8cdf0c6038c to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Enabling transport-cc for audio codecs (the previous commit on this | |
| branch) added the transport-sequence-number RTP header extension and | |
| transport-cc RTCP feedback to audio. As a side effect, libwebrtc's | |
| bandwidth-estimation (BWE) and probing paths are now active for the | |
| audio send stream. This in turn exposes a long-standing accounting gap | |
| in Firefox's libwebrtc glue that causes BWE probing to spray | |
| padding-only RTP packets on the audio media SSRC. Those packets reach | |
| the encoded-receive frame transformer as audio frames with empty | |
| payloads and no audio-level extension, breaking | |
| webrtc-encoded-transform/tentative/RTCEncodedAudioFrame-audiolevel.html | |
| ("audioLevel present in audio receiver"), which reads ten consecutive | |
| frames and asserts that every one carries an audioLevel value. | |
| How the gap manifests | |
| ===================== | |
| In `webrtc::AudioSendStream::Start()` (third_party/libwebrtc, audio/ | |
| audio_send_stream.cc:366-376) the audio stream only opts into the | |
| "good" send path -- the one that hooks the stream into BWE and the | |
| paced sender's byte budget -- when all of: | |
| !config_.has_dscp | |
| config_.min_bitrate_bps != -1 | |
| config_.max_bitrate_bps != -1 | |
| (allocate_audio_without_feedback_ || | |
| config_.include_in_congestion_control_allocation) | |
| are satisfied. When they are, libwebrtc calls: | |
| rtp_transport_->AccountForAudioPacketsInPacedSender(true); | |
| rtp_transport_->IncludeOverheadInPacedSender(); | |
| rtp_rtcp_module_->SetAsPartOfAllocation(true); | |
| ConfigureBitrateObserver(); | |
| Firefox's `WebrtcAudioConduit` leaves all three relevant config fields | |
| at their defaults (min_bitrate_bps = -1, max_bitrate_bps = -1, | |
| include_in_congestion_control_allocation = false), so the conditional | |
| falls into the `else` branch and only `SetAsPartOfAllocation(false)` is | |
| called. The pacer therefore never gets `account_for_audio_ = true` and | |
| never gets `include_overhead_ = true`. Concretely: | |
| * `PacingController::OnPacketSent` (third_party/libwebrtc, modules/ | |
| pacing/pacing_controller.cc) does | |
| if ((!audio_packet || account_for_audio_) && ...) | |
| UpdateBudgetWithSentData(packet_size); | |
| so every audio packet Firefox sends is invisible to the pacer's | |
| media budget. The budget keeps growing as if no media were being | |
| sent at all. | |
| * Even if `account_for_audio_` were true, with `include_overhead_` | |
| false the pacer would only charge the payload+padding bytes, not | |
| the header. The new transport-sequence-number extension makes the | |
| audio header larger, so any future overhead accounting also needs | |
| `IncludeOverheadInPacedSender()` to be set. | |
| * Audio is never registered as a `BitrateAllocator` observer, so BWE | |
| has no audio stream to allocate bandwidth to. The audio encoder | |
| runs at its own Opus rate independent of the pacer. | |
| When BWE initial probing fires (driven by GoogCC creating probe | |
| clusters with a target rate measured in hundreds of kbps), the pacer | |
| asks `PacketRouter::GeneratePadding`. The packet router walks | |
| `send_modules_list_` looking for a module that returns true from | |
| `SupportsPadding()`. For an audio-only call, the only registered module | |
| is the audio one, and because transport-sequence-number is registered | |
| on it, `supports_bwe_extension_` is true and so `SupportsPadding()` | |
| returns true. `RTPSender::GeneratePadding` (modules/rtp_rtcp/source/ | |
| rtp_sender.cc:447-501) then enters the `rtx_ == kRtxOff` branch | |
| (audio has no RTX), assigns `padding_packet->SetSsrc(ssrc_)` -- the | |
| audio media SSRC -- reserves the BWE extensions, calls `SetPadding(N)` | |
| on it, and emits it. `RTPSenderAudio::SendAudio` is *not* on this code | |
| path, so the audio-level extension is never set on these padding | |
| packets, and the packet's payload is zero bytes. | |
| On the receive side these padding packets are delivered with the | |
| audio's SSRC and PT, so they pass `MediaPipelineFilter::Filter` and | |
| reach `webrtc::ChannelReceive::OnRtpPacket`. `ChannelReceive:: | |
| ReceivePacket` computes `payload_data_length = payload_length - | |
| header.paddingLength` (channel_receive.cc:684), which is 0 for a | |
| padding-only packet, and then unconditionally hands the result to | |
| `frame_transformer_delegate_->Transform(payload_data, header, ...)` | |
| (channel_receive.cc:728). The receive script transform sees a stream | |
| of "audio frames" with an empty payload and a header that has no | |
| audio-level extension. The first such frame fails the WPT's | |
| `"audioLevel" in metadata` check. | |
| Instrumenting `RtpSenderEgress::SendPacket` confirms the mechanism: | |
| each emitted `type=audio` packet has `has_audio_level_ext=yes`, while | |
| every interleaved `type=padding` packet on the same audio SSRC has | |
| `has_audio_level_ext=no`, sequence-numbered contiguously with the real | |
| audio. Once BWE settles (~50 ms after start-up) probing stops, the | |
| padding bursts stop, and every received packet from then on has the | |
| extension -- but by then the WPT has already read its first failing | |
| frame. | |
| Why Chrome does not hit this | |
| ============================ | |
| Chrome's `cricket::WebRtcVoiceMediaSendChannel::WebRtcAudioSendStream:: | |
| UpdateAllowedBitrateRange` (third_party/webrtc, media/engine/ | |
| webrtc_voice_engine.cc:1105-1126) sets a fixed audio bitrate range: | |
| const int kDefaultBitrateBps = 32000; | |
| config_.min_bitrate_bps = kDefaultBitrateBps; | |
| config_.max_bitrate_bps = kDefaultBitrateBps; | |
| if (config_.send_codec_spec && | |
| config_.send_codec_spec->target_bitrate_bps) { | |
| config_.min_bitrate_bps = *config_.send_codec_spec->target_bitrate_bps; | |
| config_.max_bitrate_bps = *config_.send_codec_spec->target_bitrate_bps; | |
| } | |
| i.e. min == max, defaulting to 32 kbps, or whatever the negotiated | |
| codec's `target_bitrate_bps` is. Combined with transport-sequence- | |
| number being registered on the audio send stream, this satisfies the | |
| `AudioSendStream::Start()` conditional in Chrome's libwebrtc copy | |
| (which uses `TransportSeqNumId(config_) != 0` in place of | |
| `include_in_congestion_control_allocation`, but is equivalent in | |
| effect). Chrome therefore takes the "good" branch: | |
| `AccountForAudioPacketsInPacedSender(true)` so audio packet sizes | |
| charge against the pacer budget, `IncludeOverheadInPacedSender()` so | |
| the RTP header (including the new transport-sequence-number bytes) is | |
| charged too, `SetAsPartOfAllocation(true)`, and | |
| `ConfigureBitrateObserver()` so audio is registered with the | |
| `BitrateAllocator` with `enforce_min_bitrate = true`. BWE allocates the | |
| fixed 32 kbps (or codec target) to audio, the pacer's budget reflects | |
| what audio actually sends, and probing has real media to fill its | |
| target with -- so padding-only packets on the audio SSRC are not | |
| spuriously generated. | |
| This matches Chrome's UpdateAllowedBitrateRange in webrtc_voice_engine.cc:1105–1126 (default 32 kbps, override with codec target if | |
| set; min == max for a fixed audio rate). The include_in_congestion_control_allocation flag is gated on mTransportCCFbSet so when | |
| transport-cc is not negotiated, behavior remains exactly as before (the conditional in AudioSendStream::Start() falls through to the | |
| else branch). | |
| With these three fields set + TWCC negotiated, AudioSendStream::Start() (audio_send_stream.cc:366-376) takes the path that calls | |
| AccountForAudioPacketsInPacedSender(true) + IncludeOverheadInPacedSender() + SetAsPartOfAllocation(true) + | |
| ConfigureBitrateObserver(). The pacer now sees audio bytes (and the new TWCC overhead), BWE allocates a real budget to audio, and | |
| probing has actual media to send instead of falling back to padding-only packets on the audio SSRC. | |
| All the debug logging from the prior turns has been reverted. Ready for you to build and rerun the test. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment