Audio In Game

[spikebarnett]

Recently I've been mulling around the idea of making a Guitar Hero-esque game/trainer that will take input from an electric guitar. But before I dive into it head first, I want to get a feel for how deep the water is.

I have experience with C++/C# and would be coding for Windows or Linux.

My primary questions and concerns are:

What libs are recommended for such an application given the OS and language.

How over my head am I getting trying to rip useful information from an audio buffer? (Notes being played)

I do have a fairly strong mathematical background, but I imagine that determining pitch while two or more strings are ringing would be quite difficult to code.

[Disch]

What libs are recommended for such an application given the OS and language.

I've been a fan of SFML for a while: http://www.sfml-dev.org

And also an FFT lib (google, I don't have one to recommend)

How over my head am I getting trying to rip useful information from an audio buffer?

Not very. Run the audio data through an FFT (fast fourier transform) and you have the magnitude of all harmonics. Pick out the dominant tone(s) and you can tell more or less what the user is playing.

Of course it will take some tweaking and experimentation, but the overall process shouldn't be too difficult.

[cnoeval]

I do have a fairly strong mathematical background, but I imagine that determining pitch while two or more strings are ringing would be quite difficult to code.

Actually this is still a very black art. More than one note with almost ANY harmonic content will be hard to evaluate. Multiple simple tones (e.g. 2-3 flutes) would be possible but following along with even a mediocre guitarist playing chords and expecting reasonable accuracy is still a pipe-dream. That's why Guitar Hero guitars are just multi-button toys (plus, they're cheap of course).

http://www.intelliscore.net/

This company claims some success (and I have NOT tried their software) but everything else I've ever seen try this is a joke when faced with real music and not just very carefully plucked single-notes. Look into HEX pickups, they're used for guitar synths and would make your task MUCH easier.

[helios]

Or you could just use MIDI or similar program-based formats.

[spikebarnett]

I've been a fan of SFML for a while: http://www.sfml-dev.org

I gave it a look see and I like it so far. As far a an FFT, I think I'm going to go with FFTW. I'll use Ogre for graphics.
I think I'll start by making a simple tuning program to get my bearings, and then work up from there.

Look into HEX pickups, they're used for guitar synths and would make your task MUCH easier.

While that would simplify the input quite a bit, I'd like to make this program as accessible as possible. I intend to release it at no cost (at least the initial versions), so requiring people to have Hex pickups sort of defeats the purpose.

Or you could just use MIDI or similar program-based formats.

There are MIDI guitars being released soon on consoles, so this could become a better option as the percentage of guitarist who own MIDI enabled guitars increases. But until I get my hands on a MIDI guitar, I won't be able to test it.

Pic of a console MIDI Enabled Guitar for those who are interested.
http://www.blogcdn.com/www.joystiq.com/media/2010/03/forpressproduct2rev.jpg

[spikebarnett]

I tried using SFML with no success. I tried using it with Code::Blocks and VC++. It wouldn't find the headers after I added the folders to the search directories, so I rearranged the include folder in such a way that it would find the includes. Now it just errors on every line of SFML related code, claiming the identifiers are undefined. Am I missing something?

[NGen]

#pragma comment ( lib, "sfml-audio-s-d.lib" )

Will work in VC++, I don't know about other compilers.

[spikebarnett]

I was able to get it past the include lines, but now I get errors like the following.

audiointest.obj : error LNK2028: unresolved token (0A000429) "void __cdecl sf::Sleep(float)" (?Sleep@sf@@$$FYAXM@Z) referenced in function "int __cdecl main(void)" (?main@@$$HYAHXZ)

audiointest.obj : error LNK2019: unresolved external symbol "void __cdecl sf::Sleep(float)" (?Sleep@sf@@$$FYAXM@Z) referenced in function "int __cdecl main(void)" (?main@@$$HYAHXZ)

I'm getting these errors for every SFML function or class I use in the code. I have to be honest, I've never encountered this before, and I haven't the slightest idea what the problem is. I feel as though I've overlooked some small detail.

[spikebarnett]

OK. I got it to compile and run as expected after I copied all of the .lib to the project folder and all of the .dll to the debug folder. Apparently putting them in the lib directory and linking to them isn't good enough. What a pain in the ass. =P

Is this the norm?