Mixing audio of different frequencies and mono/stereo?

[superfury]

I've build a mixer for mixing audio channels of different frequenties and mono/stereo samples, however in order to do that effeciently and fast I'm using one lookup tables per channel (which take quite a lot of memory) plus one lookup table for stereo support.

One stereo lookup table is used to convert the current sample position with l/r channel to the source sample position (precalcs contains the results of ((time*(1<<stereo)) + (stereo*rchannel)) ).

The samplerate lookuptable is used to convert the current sample position to the channel sample position, based on the samplerate of the used channel (effectively: (current sample position/hardware samplerate)*channel samplerate).

Is there a better way to do this?

[Disch]

I'm not sure why you'd need a lookup table.... or even what purpose one would serve.

Simple (linear) samplerate changes can be accomplished with a time scale:

// x = output sample time

scale = input_samplerate / output_samplerate;

fraction = modf( x*scale, &xx );

output_samples[x] = interpolate( input_samples[xx], input_samples[xx+1], fraction );

If the floating point math is no good... you can use fixed point.
And if linear interpolation is too slow, you can use nearest neighbor easily enough (although it'll sound much worse).




As for converting mono<->stereo.. that's pretty straightforward.

mono->stereo = output the sample twice
stereo->mono = average the 2 channels together.

[superfury]

My current mixing routine (mixing one output sample from all available channels):

//Simple macros for checking samples!
//Precalcs handling!
#define C_SAMPLEPOS(channel) (soundchannels[channel].sound.position)
#define C_BUFFERSIZE(channel) (soundchannels[channel].sound.numsamples)
//Use precalculated sample positions!
#define C_SAMPLERATE(channel,position) (convert_samplerate[soundchannels[(channel)].samplerate][(position)])
#define C_GETSAMPLEPOS(channel,rchannel,time) (samplepos[soundchannels[(channel)].stereo][(((word)time)<<1)|(rchannel)])
#define C_STEREO(channel) (soundchannels[channel].stereo)
#define C_VOLUMEPERCENT(channel) (soundchannels[channel].volume_percent)
#define C_SAMPLE(channel,samplepos) (soundchannels[channel].sound.samples[samplepos])

OPTINLINE void mixchannels(sample_stereo_t *res) //Mixes the channels with each other at a specific time!
{
	//Process multichannel!
	const float SHORTY = 1/USHRT_MAX; //Simple for easy multiplication of division!
	float result_l, result_r; //The result!
	float multi_l, multi_r; //Multiplication of all channels!
	uint_32 n,  relsample; //Current channel and relative sample!
	word channelsleft; //Reversed of above!
	//Channel specific data
	float volume,sample_l, sample_r;
	
	if (__HW_DISABLED) return; //Abort?

	//First, initialise our variables!
	channelsleft = soundchannels_used; //Ammount of channels left!
	if (!channelsleft)
	{
		res->l = res->r = 0.0f; //Nothin to process!
		return; //Done!
	}

	//Result and active counters!
	result_l = result_r = 0.0f; //Init to standard sum!
	multi_l = multi_r = 1.0f; //Init to standard factor!
	n = 0; //Current channel number counter and active channel counter!
	
	//Now process all channels!
	for (;;) //Process all channels!
	{
		if (soundchannels[n].soundhandler) //Active?
		{
			if (soundchannels[n].samplerate && memprotect(soundchannels[n].sound.samples,soundchannels[n].sound.length,"SW_Samples") && samplepos[soundchannels[n].stereo] && convert_samplerate[soundchannels[n].samplerate]) //Allocated all neccesary channel data?
			{
				relsample = C_SAMPLERATE(n,C_SAMPLEPOS(n)); //Get the sample rate!
				while (relsample>=C_BUFFERSIZE(n)) //Expired, we've reached the end of the buffer (sample overflow)?
				{
					#ifdef DEBUG_SOUNDSPEED
					dolog("Sound","Buffering %i @ %i/%i samples; extra data: %p",n,C_SAMPLEPOS(n),C_BUFFERSIZE(n),soundchannels[n].extradata);
					#endif
					//Buffer and update buffer position!
					soundchannels[n].soundhandler(soundchannels[n].sound.samples,C_BUFFERSIZE(n),soundchannels[n].extradata); /* Request next sample for this channel, also give our channel extra information! */
					C_SAMPLEPOS(n) -= C_BUFFERSIZE(n); //Reset position in the next frame!
					relsample = C_SAMPLERATE(n,C_SAMPLEPOS(n)); //Get the sample rate for the new buffer!
				}

				//Now process the buffered sound itself!
				
				volume = C_VOLUMEPERCENT(n); //Retrieve the current volume!
				sample_l = C_SAMPLE(n,C_GETSAMPLEPOS(n,0,relsample))*volume; //The composed sample, based on the relative position!
				sample_r = C_SAMPLE(n,C_GETSAMPLEPOS(n,1,relsample))*volume; //The composed sample, based on the relative position!
				++C_SAMPLEPOS(n); //Next position on each channel!
				
				//Next, add the data to the mixer!
				result_l += sample_l; //Mix the channels equally together based on volume!
				multi_l *= sample_l; //Multi based on volume!
				result_r += sample_r; //See above!
				multi_r *= sample_r; //See above!
			}
		}
		if (!--channelsleft) break; //Stop searching if nothing left!
		++n; //Next channel!
	}
	
	//Final calculations!
	multi_l *= SHORTY; //Set the limit!
	multi_r *= SHORTY; //Set the limit!
	result_l -= multi_l; //Final step in mixing: all channels equally!
	result_r -= multi_r; //Final step in mixing: all channels equally!
	//Now we have: (a+...)-((a*...)/MAXVAL), where MAXVAL is 256,65536 etc. depending on the type.
	res->l = result_l; //Left channel!
	res->r = result_r; //Right channel!
}

This is based upon the [sum of all channels]-([multiplication of all channels]/maximumsize), where for two channels is (a+b+...)-(a*b*.../256) with 8-bit mono sound. Replace 256 with 65536 with short instead of signed char.

For some reason it still has a little bug (i'm not 100% sure about the buffering part (the relsample calculation with the while loop). Also the first channel seems to override the other channels (I won't hear them through the speakers). Anyone knows what's going wrong here?

[Disch]

Wish I could help more, but straight C makes my eyes bleed. =x

This is based upon the [sum of all channels]-([multiplication of all channels]/maximumsize)

That's interesting... I never heard of that technique before.... I always just took the sum of all channels. The multiplication seems like it would heavily distort the sound.

[superfury]

Source of the technique used to mix (row 65-68&75-79 in the script): http://www.vttoth.com/CMS/index.php/technical-notes/68