I have a few recommendations:
* cheap way:
Prof. Nearing at University of Miami has a free book that is quite good that just might fit the bill:
http://www.physics.miami.edu/~nearing/mathmethods/"
Otherwise, for vectors in curvilinear coordinates, the Schaum's outline of Vector Analysis is pretty good, and includes a rough intro to tensors.
for complex analysis, get a used copy of an old edition of saff and snider. 2nd edition is $5 at amazon. Covers series, too. Perhaps schaum's outline is useful as well.
Use arfken for delta and special functions - it is fine. Fourier Jr's post links to the classic on complex analysis and special functions, from the point of view of 80 years ago or so. I own a hardcopy of that book and love it, but it is not for learning basics, as Fourier Jr states. The online edition is LEGAL! It is old so I think copyright has expired.
If you want a *reference* for special functions, ":handbook of mathematical functions" by Abramowitz and Stegun is the classic. It is available free online (legally!) if you google. Again I own a hardcopy that is almost worn out.
* other "math physics" books"
Mathematical Physics by Kusse and Westwig is quite good, in my opinion. Covers curvilinear, tensors, complex analysis, and delta functions. Not so good at special functions. Assumes you know the contents of Thomas' Calculus plus basic linear algebra.
Mathematical Methods for Physics and Engineering by Riley, Hobson and Bence is also a good general reference, that only assumes you know elementary calculus. Not too expensive for what you get.
* other books
For tensors, the best book I know is "a brief on tensor analysis" by Simmonds. Not so cheap.
A good semi-mathematical intro to distribution theory (and hence delta functions) is "a guide to distribution theory and Fourier tranforms" by strichartz. You do NOT need this book, but is just in case you are interested. Gauranteed not to help you in your class. I used this in a course I took, but some co-worker stole it from me so I am missing it!
Good luck,
jason