Skip to content

Instantly share code, notes, and snippets.

@jrmuizel
Created May 27, 2026 02:54
Show Gist options
  • Select an option

  • Save jrmuizel/2c95d4770be3a97f0240a8cdf0c6038c to your computer and use it in GitHub Desktop.

Select an option

Save jrmuizel/2c95d4770be3a97f0240a8cdf0c6038c to your computer and use it in GitHub Desktop.
Enabling transport-cc for audio codecs (the previous commit on this
branch) added the transport-sequence-number RTP header extension and
transport-cc RTCP feedback to audio. As a side effect, libwebrtc's
bandwidth-estimation (BWE) and probing paths are now active for the
audio send stream. This in turn exposes a long-standing accounting gap
in Firefox's libwebrtc glue that causes BWE probing to spray
padding-only RTP packets on the audio media SSRC. Those packets reach
the encoded-receive frame transformer as audio frames with empty
payloads and no audio-level extension, breaking
webrtc-encoded-transform/tentative/RTCEncodedAudioFrame-audiolevel.html
("audioLevel present in audio receiver"), which reads ten consecutive
frames and asserts that every one carries an audioLevel value.
How the gap manifests
=====================
In `webrtc::AudioSendStream::Start()` (third_party/libwebrtc, audio/
audio_send_stream.cc:366-376) the audio stream only opts into the
"good" send path -- the one that hooks the stream into BWE and the
paced sender's byte budget -- when all of:
!config_.has_dscp
config_.min_bitrate_bps != -1
config_.max_bitrate_bps != -1
(allocate_audio_without_feedback_ ||
config_.include_in_congestion_control_allocation)
are satisfied. When they are, libwebrtc calls:
rtp_transport_->AccountForAudioPacketsInPacedSender(true);
rtp_transport_->IncludeOverheadInPacedSender();
rtp_rtcp_module_->SetAsPartOfAllocation(true);
ConfigureBitrateObserver();
Firefox's `WebrtcAudioConduit` leaves all three relevant config fields
at their defaults (min_bitrate_bps = -1, max_bitrate_bps = -1,
include_in_congestion_control_allocation = false), so the conditional
falls into the `else` branch and only `SetAsPartOfAllocation(false)` is
called. The pacer therefore never gets `account_for_audio_ = true` and
never gets `include_overhead_ = true`. Concretely:
* `PacingController::OnPacketSent` (third_party/libwebrtc, modules/
pacing/pacing_controller.cc) does
if ((!audio_packet || account_for_audio_) && ...)
UpdateBudgetWithSentData(packet_size);
so every audio packet Firefox sends is invisible to the pacer's
media budget. The budget keeps growing as if no media were being
sent at all.
* Even if `account_for_audio_` were true, with `include_overhead_`
false the pacer would only charge the payload+padding bytes, not
the header. The new transport-sequence-number extension makes the
audio header larger, so any future overhead accounting also needs
`IncludeOverheadInPacedSender()` to be set.
* Audio is never registered as a `BitrateAllocator` observer, so BWE
has no audio stream to allocate bandwidth to. The audio encoder
runs at its own Opus rate independent of the pacer.
When BWE initial probing fires (driven by GoogCC creating probe
clusters with a target rate measured in hundreds of kbps), the pacer
asks `PacketRouter::GeneratePadding`. The packet router walks
`send_modules_list_` looking for a module that returns true from
`SupportsPadding()`. For an audio-only call, the only registered module
is the audio one, and because transport-sequence-number is registered
on it, `supports_bwe_extension_` is true and so `SupportsPadding()`
returns true. `RTPSender::GeneratePadding` (modules/rtp_rtcp/source/
rtp_sender.cc:447-501) then enters the `rtx_ == kRtxOff` branch
(audio has no RTX), assigns `padding_packet->SetSsrc(ssrc_)` -- the
audio media SSRC -- reserves the BWE extensions, calls `SetPadding(N)`
on it, and emits it. `RTPSenderAudio::SendAudio` is *not* on this code
path, so the audio-level extension is never set on these padding
packets, and the packet's payload is zero bytes.
On the receive side these padding packets are delivered with the
audio's SSRC and PT, so they pass `MediaPipelineFilter::Filter` and
reach `webrtc::ChannelReceive::OnRtpPacket`. `ChannelReceive::
ReceivePacket` computes `payload_data_length = payload_length -
header.paddingLength` (channel_receive.cc:684), which is 0 for a
padding-only packet, and then unconditionally hands the result to
`frame_transformer_delegate_->Transform(payload_data, header, ...)`
(channel_receive.cc:728). The receive script transform sees a stream
of "audio frames" with an empty payload and a header that has no
audio-level extension. The first such frame fails the WPT's
`"audioLevel" in metadata` check.
Instrumenting `RtpSenderEgress::SendPacket` confirms the mechanism:
each emitted `type=audio` packet has `has_audio_level_ext=yes`, while
every interleaved `type=padding` packet on the same audio SSRC has
`has_audio_level_ext=no`, sequence-numbered contiguously with the real
audio. Once BWE settles (~50 ms after start-up) probing stops, the
padding bursts stop, and every received packet from then on has the
extension -- but by then the WPT has already read its first failing
frame.
Why Chrome does not hit this
============================
Chrome's `cricket::WebRtcVoiceMediaSendChannel::WebRtcAudioSendStream::
UpdateAllowedBitrateRange` (third_party/webrtc, media/engine/
webrtc_voice_engine.cc:1105-1126) sets a fixed audio bitrate range:
const int kDefaultBitrateBps = 32000;
config_.min_bitrate_bps = kDefaultBitrateBps;
config_.max_bitrate_bps = kDefaultBitrateBps;
if (config_.send_codec_spec &&
config_.send_codec_spec->target_bitrate_bps) {
config_.min_bitrate_bps = *config_.send_codec_spec->target_bitrate_bps;
config_.max_bitrate_bps = *config_.send_codec_spec->target_bitrate_bps;
}
i.e. min == max, defaulting to 32 kbps, or whatever the negotiated
codec's `target_bitrate_bps` is. Combined with transport-sequence-
number being registered on the audio send stream, this satisfies the
`AudioSendStream::Start()` conditional in Chrome's libwebrtc copy
(which uses `TransportSeqNumId(config_) != 0` in place of
`include_in_congestion_control_allocation`, but is equivalent in
effect). Chrome therefore takes the "good" branch:
`AccountForAudioPacketsInPacedSender(true)` so audio packet sizes
charge against the pacer budget, `IncludeOverheadInPacedSender()` so
the RTP header (including the new transport-sequence-number bytes) is
charged too, `SetAsPartOfAllocation(true)`, and
`ConfigureBitrateObserver()` so audio is registered with the
`BitrateAllocator` with `enforce_min_bitrate = true`. BWE allocates the
fixed 32 kbps (or codec target) to audio, the pacer's budget reflects
what audio actually sends, and probing has real media to fill its
target with -- so padding-only packets on the audio SSRC are not
spuriously generated.
This matches Chrome's UpdateAllowedBitrateRange in webrtc_voice_engine.cc:1105–1126 (default 32 kbps, override with codec target if
set; min == max for a fixed audio rate). The include_in_congestion_control_allocation flag is gated on mTransportCCFbSet so when
transport-cc is not negotiated, behavior remains exactly as before (the conditional in AudioSendStream::Start() falls through to the
else branch).
With these three fields set + TWCC negotiated, AudioSendStream::Start() (audio_send_stream.cc:366-376) takes the path that calls
AccountForAudioPacketsInPacedSender(true) + IncludeOverheadInPacedSender() + SetAsPartOfAllocation(true) +
ConfigureBitrateObserver(). The pacer now sees audio bytes (and the new TWCC overhead), BWE allocates a real budget to audio, and
probing has actual media to send instead of falling back to padding-only packets on the audio SSRC.
All the debug logging from the prior turns has been reverted. Ready for you to build and rerun the test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment