Pure mathematics? Not so much. Algorithms, yes. Obviously, they're based on mathematics, but only very specific parts.
Personally, I'd suggest picking up a/some book(s) on Algorithm Design and work top-down: Encounter some math you don't understand? Go down to the lower level. It's generally much easier to start that way, because mathematics is a big, big field and it's hard to know which parts will come in handy and which parts won't. (But I suppose any and all math is handy, if only for the "mathematical insight" it gives you.)
If you really want to get the math itself down, I'd suggest taking a good look at Graph Theory; it's very important in many "common" algorithms for games. For game design specifically, I'd say any kind of advanced geometry is important (Don't shoot me if I misclassified, but I'm thinking of Collision detection, Location analysis, etc.).