SDL can be (and is) distributed as a pre-built binary so you don't have to worry about mismatching debug/release/compiler-version-x. That's one of the (only?) advantages to C over C++. It's a lot more well defined when it comes to distributed libraries. C++'s necessary name mangling kind of destroys that ability.
So yeah SFML being a C++ lib is both a blessing and a curse in that regard. And I agree it isn't entirely it's fault, but it certainly could handle the situation better than it does.
@Zereo
I just built SFML earlier today. I've uploaded it to my mediafire.
It work for me, so if it can get you going on Visual Studio, great. But i'll agree with you, VS can be a massive pain in the rear. If codeblocks works better for you, by all means, use it.