Write a custom audio mixer

[nuju]

I'm going to be using PortAudio as a cross-platform audio player, and i'd like to write a custom mixer for use with it. The problem is, i have no idea how to do this. Does anyone know of any tutorials or examples i could look at? Is it as simple as adding the raw data together?

[Stewbond]

Yes actually it is. Since waveforms are above zero just as often as below zero, adding waves together doesn't really cause problems. In fact, that's exactly what happens in your ear.

However, the more waveforms you add, the more likely you are to hit the limit of your container. If it's a 16bit signed integer, then you want to ensure that if the sum of the two waveforms is greater than 2^16, that you limit the value to 2^16 instead of rolling over and getting a very low point. If you have a lot of this, you'll start to hear some distortion known as "clipping".

If that starts to happen, you'll need to normalize the entire waveform so that you bring the entire wave to a quieter volume. You may want to make that an extra control in your mixer. Normalizing is easy, just multiply each sample by a gain (less than 1 to reduce the volume).

So.... something like this I guess:
mixed_data[i] = (stream1[i] + stream2[i] + stream3[i] + ... ) * gain;

[firedraco]

You should probably multiply before adding because otherwise you might overflow, then multiply (which won't do what you want).

[Disch]

multiplying afterwards results in less rounding error (assuming you're using integral types for samples)

I would do as Stewbond suggests, but I would make sure you use a larger data type when mixing (ie, when dealing with 16-bit samples, use at least a 32-bit variable to mix them).

And be sure to clip afterwards (if output is higher/lower than the min/max value, output the min/max value instead). Clipping sounds bad, but wrapping sounds much, much worse.

[nuju]

Should i have some sort of inline function/macro that returns a multiplier given the number of samples and the total intensity (decreasing the multiplier as number of samples and intensity increase)?

[Stewbond]

If you want the multiplier to be automatically calculated instead of controlled by the user (which it is on things like guitar amplifiers or mixers with knobs), then you'll want to check out the RMS of the wave form. The RMS is a good approximation of the mean amplitude of a wave form. For a pure sine wave of amplitude 1, the RMS is 1/sqrt(2).

That means do something like this

__int16* wave1, wave2; // Each filled with 16bit signed data
__int32* output; // mixed data

double Gain1, Gain2; // Control the volume of each wave and of the output;

// Step 1: Add the waves
for (int i = 0; i < length; ++i)
  output[i] = Gain1 * (__int32)wave1[i] 
             + Gain2 * (__int32)wave2[i];

// Step 2: Find the RMS (root-mean-square) of the output
double RMS = 0;
for (int i = 0; i < length; ++i)
  RMS += output[i]*output[i];
RMS /= length;
RMS = sqrt(RMS);

// Step 3: Calculate the master gain so that the output RMS is reasonable
// 0.707 of the maximum is a good value for an RMS ( 1 / sqrt(2) )
double GainMaster = 0x7fff / (sqrt(2)*RMS);

// Step 4: Apply your gain: 
for (int i = 0; i < length; ++i)
  output[i] *= GainMaster;

// Step 5: Clipping (This method does not guarentee that there will be no clipping so this is for safety)
for (int i = 0; i < length; ++i)
  if (output[i] > 0x7fff)
    output[i] = 0x7fff;
  else if (output[i] < -0x7fff)
    output[i] = -0x7fff;

// Step 6: Stick the output into a container of the original size;
for (int i = 0; i < length; ++i)
  final[i] = (__int16)output[i];

I've split it into steps so it's easy to understand, but inline functions are great.

[nuju]

Thanks for all your help everyone. I'll look over this!