Cornerstone SDK provides an audio-processing engine called Resonant.
Resonant provides platform-independent access to audio devices using PortAudio library and a light-weight signal-processing engine.
By default, Cornerstone will use the default audio device available on the computer running the application. The used audio device can also be specified manually. This is done by setting the RESONANT_DEVICE environment variable to the index number or name of the desired audio device.
To get a list of available devices and the index numbers, use the ListPortAudioDevices application that is shipped with Cornerstone. Below is an example output from the application:
In the above example, to select the Saffire sound card we could set the RESONANT_DEVICE to either Saffire 6USB or 3 (the index number of the device). In practice, using the device name is preferred as the order of the audio devices is not guaranteed to be constant between reboots. You can use any (unique) substring of the device name to do the matching. For example, both Saffire and 6USB would map to device number 3 in the above example.
Cornerstone can utilize multiple synchronized audio devices simultaneously. The special requirements for the audio setup are described further below.
Multiple devices are used by mapping the logical audio channels used by Cornerstone to actual audio channels on the sound devices. This mapping is specified in an XML file that is passed to an application using –audio-devices command line parameter. This parameter is only required if you want to use multiple audio devices at the same time. Typically it is not needed.
For example, to map the first five logical audio channels onto two different audio devices, we would specify something like this:
In the above example, we map the first two channels (channels 0,1) to the HDA Intel sound device (channels 0,1) and the next three channels (channels 2,3,4) to pulse device channels 0,1,2.
The mapping is always done in incremental fashion in the same order as they appear in the configuration file. It is not possible to map individual channels arbitrarily.
As with RESONANT_DEVICE, the device name can be specified using any substring of the actual device name.
Using multiple devices only works if the audio setup fulfills the requirements below. Any setup that uses multiple devices needs to be tested carefully before releasing into production use.
A fundamental requirement for using multiple audio devices is that the devices must be tightly synchronized. This means that they rely on the same sample clock. If the devices are not tied to the same clock source, then their sampling rates usually drift, causing crackling of the output sound due to audio buffer underflows or overflows.
Also the audio devices must use identical buffer parameters. This means that the audio drivers of the different devices give the application buffers with identical number of samples to fill.
Usually the way to fulfill both of these requirements is to firstly use only type of sound card which unifies the drivers and buffering parameters. Secondly, the devices need to support external sample rate clocks so they share the same clock source. In practice only high-end professional sound cards support external clock sources.
A special use case is that you have a sound card that presents itself as multiple audio devices. For example some 8-channel sound cards appear as 4 stereo devices. These devices generally fulfill both requirements.
Cornerstone provides a built-in high-level API to configure speaker setups to implement audio panning. Typically the configuration is provided to applications using an XML file, but it can also be done in code using the Resonant classes. Using an XML file, the configuration is passed to an application using –audio-config parameter.
This is done by defining speaker locations in application coordinates and to which audio channel each speaker output is directed. The audio panning can be set up using two different modes: radial or rectangular.
In radial audio panning, only the speaker location is defined for each speaker. The audio volume is then attenuated based on the distance from the speaker. The attenuation is controlled by a global capture-radius parameter that defines the attenuation for all speakers. It is not possible to setup individual attenuation factors for each speaker.
An example setup for two speakers is shown below:
The audio channels are mapped incrementally starting from 0.
In rectangular audio panning, we define rectangular regions with each having a speaker on the left edge and the right edge. Rectangular panning mode offers more parameters to control how the audio is attenuated between speakers. For details, see the documentation of Resonant::SoundRectangle. An example configuration file for two rectangles is shown below:
Typically, there is no need to go deep into the internals of the audio processing engine. For those interested, below is information on the main principles of the audio engine.
Resonant is built using a modular approach where digital signal processing (DSP) is performed by linked DSP modules that form a graph. The graph supports hot-plugging, i.e. modules can be added and removed on the fly. The signal processing is performed in a separate thread because of the requirements for minimum latency in audio processing.
Modules perform all processing in the DSP graph. They have a set of inputs and outputs that can be connected to other modules. Modules can have a set of controls and parameters which are adjusted by sending events. All event passing must go through the Resonant::DSPNetwork class to guarantee thread-safety.
Audio signals are passed between the modules using sample buffers. Each buffer contains an array of floating point numbers which carry a mono audio signal from one module to another. The audio engine can handle any number of channels, by using multiple mono buffers to transfer multi-channel audio signals.
The modules are not aware of each other, it is the task of the DSPNetwork object to manage the modules and give the pointers to the signal buffers.
