Notes on diving into the vcvrack source code.

mp-patch.md

All modules will implement a stepStream method that is called in the realtime callback. Instead of processing a single sample at a time, they will process a block of samples.

How to pass the sample output of one module to the sample input of the next?

After initializing the stream in openStream we allocate a buffer of blockSize and let modules write into this as we traverse the graph.

vcvnotes.md

Action Items

• No error handling callback set here.

Open Questions

Waiting to activate the module graph

Why have these two busy loops?

Ostensibly rtaudio only invokes our callback when one of those two conditions has occurred.

Could we do better and get rid of the busy loops?

rtaudio openStream options

Could we do better here?

Would RTAUDIO_MINIMIZE_LATENCY provide any performance improvement?

realtime code

AudioInterface

• Has three ring buffers (lock-free buffers that can be used in parallel safely by one writer thread and one reader thread): inputBuffer, outputBuffer, and inputSrcBuffer.
• Calls stepStream in the realtime callback.
• If the AudioInterface has inputs, then stepStream copies them to outputBuffer. If it has outputs then it copies samples from inputSrcBuffer to the outputs.
• stepStream passes input samples to the sample rate converter's output buffer, and output samples to the sample rate converter's input buffer.
• Where does the SRC's input buffer get copied to the rtaudio output buffer (sending the SRC output to the audio hardware)?
  • When AudioInterface is added as a module then its step method is being called here.

AudioInterface for multi-processor machines

Rack's AudioInterface module drives the input/output of audio signals to/from the machine.

It contains the realtime callback that is passed to the underlying system (e.g. rtaudio).

This callback ^ should drive the audio-rendering thread!

Proposal

Modules should be adapted to process audio buffers instead of processing a single frame at a time.

We could add this method to Module (notably, AudioInterface already has it):

virtual void stepStream(const float *input, float *output, int numFrames) {}

This would be implemented by modules with the expectation that it will be called in a realtime context and should follow Ross Bencina's guidelines.

To do this in a backwards compatible way we can provide a default implementation of stepStream which will call step to process the input and generate the output. This would give plugin authors time to update to the new API.

How to deal with cyclic graphs?

One idea is:

1. Detect the minimal set of connections that would have to be removed from a graph to make it acyclic. The resulting subgraph can then easily be processed in an orderly fashion.

2. The set of "feedback" connections gets a 1-cycle delay, so the output data at the tail end of each connection is cached and used as the input at the head of the connection on the next cycle. I think this may be the approach suggested by Stephane Letz here.

3. For a graph change that adds feedback connections, the input buffer at the head of the new feedback connections will be zeroes in the first cycle where the new graph is used.

Implementation

TODO: notes on how to move forward with an implementation.

Plugins

If we want plugins to expose a realtime callback, how easy would it be to get plugin authors to adapt their code?

Do most of them already adhere to realtime restrictions in their step implementations?

Here are the ones we will look at, which I chose because they happen to be some of my favorites:

• Fundamental
• Audible Instruments
• Bogaudio
• Befaco
• VCVRack-Simple
• Vult
• mhetrick
• Strum Mental

This is a simple patch:

Which generates the following flame graph:

This shows that there is a significant amount of CPU time spent in AudioInterface::step, VCO::step, and the speex function calls.

A rough estimate just from looking at the graph is that Rack spends about 2/3 of its CPU time on audio and 1/3 on graphics.