The problem is that AI is a massive field. So, you're going to need vastly different kinds of math for different things within AI. From what I've heard, stuff like computer vision is very math-heavy. You can use partial differential equations and Fourier analysis and lots of stuff.
My general gut reaction is to say that you need to learn discrete math, linear algebra, and prob/stat. Maybe some basic graph theory. But that might depend on what you are trying to do. If you are trying to do game programming, I'm guessing you would need stuff that's covered in a more standard algorithms course, like shortest path algorithms and that sort of thing. For that, you just need some discrete math/graph theory. Part of the "math" is going to be in the subject itself, too.
But there is a big disconnect, in that mathematics does not (yet) deal with the mechanics of computational requirements, and the most efficient way to calculate functions.
That IS math. P = NP is one of the millennium problems, a set of 7 problems (6 left) that you can win a million dollars for solving. The only disconnect is the same disconnect that exists between theoretical computer science and practical programming. In practice, it might not only be about big O. You might have to time different algorithms to see which is that fastest on the type of input data that you have and so on, and you might care about constant factors and so on.
I haven't seen a mathematical formalization of quicksort yet, anyway.
The way I see it, quicksort IS just math. Sorting is closely related to permutations of finite, totally ordered sets. And I'm sure someone has written it out in full rigor somewhere. The fact that people don't normally bother to do that just means it's non-rigorous, rather than that it's not math. There is actual algebra and math involved in analyzing the running time (best case, worst case, average case), too, even in CS classes.