Skip to content

Instantly share code, notes, and snippets.

@Oleksiy-Yakovenko

Oleksiy-Yakovenko/float32_rendering.md

Last active October 5, 2016 09:09
Show Gist options
  • Save Oleksiy-Yakovenko/ac3c143ef9d7e62ecbfddb0808881972 to your computer and use it in GitHub Desktop.
Save Oleksiy-Yakovenko/ac3c143ef9d7e62ecbfddb0808881972 to your computer and use it in GitHub Desktop.
Download ZIP
Floating point processing proposal
Raw
float32_rendering.md

The problem

On desktop / high performance CPUs, it is highly desirable to render audio stream in floating point from the decoder, apply DSP, do visualizations, apply the volume, replaygain, etc, and only then convert to the final output format, which could be configured in the Playback Output Settings.

We want to be able to get the original stream through the visualization and DSP for stuff like VU-meters, unified replaygain (in the streamer), etc.

We still want to present the user with the information about the "source" format, in the status bar.

So even if we're getting float32 for OggVorbis in the streamer - we want to show 16 bit in the status bar.

The risks

Sometimes we want to pass the unmodified stream all the way to the soundcard, for bit-perfect playback.

In the example case of 24 bit integer stream, there will be some unnecessary performance overhead, because we would need to convert the stream to float32, and then back to int24.

There's also a risk, that some distortion may occur, which we don't want.

But since float32 has enough precision in [-1,1] range, that's unlikely.

The scope

Technically, we only need to force float32 in those decoders, which may produce clipping. Most if not all of lossy decoders.

The rest may output whatever they need, and then we can convert to float32 in the streamer, without the information loss.

Requirements for the solution

  • The wave format displayed in the status bar needs to be obtained from track metadata, instead of decoder.
  • The playback options need to be configurable to allow outputting the sound in any supported format (int8, int16, int24, int32, float32)
  • An option must exist to force the decoders to switch to int16 mode, possibly with the loss of proper replaygain support (unless decoders are able to apply replaygain internally).

Verification of the solution

  • Visualization plugins must be able to process the source stream in float32, VU-meter plugin must be able to display overloads.
  • Replaygain must be able to prevent clipping from within the streamer, instead of from decoder.

Undecided

  • At which stage should DSP Plugins get the signal -- before or after replaygain?
  • The final format may either be decided (forced) by the user, or taken from the decoder settings / parameters.
  • Lossy decoders, like MP3, require dithering / noise shaping, when converting to int16/int24.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment