Hi cragwolf, I have this book and could understand it in my third year (of a mathematics degree).. I'm not sure how much mathematics the average physics graduate has? Judging from people I've met it must vary significantly..
The main prerequisites are basic real analysis (rigourous proofs, continuity, sequences, series and limits, various convergence theorems, possible Riemann-Stieljes integration [just how the notation works, not a proof that it exists for smooth functions etc.]) then complex analysis (you definitely need to know about holomorphic functions, countour integration, Cauchy's integral formula, intergration involving around simple poles).
I assume you probably have experience with some of the above. Edwards makes the (I think) deliberate point of missing out certain steps when manipulating equations - e.g. he says "upon applying Cauchy's integral formula we get..", and you will have to work out exactly how he applied it and the subsequent manipulations or simplifications.. but this makes it a more rewarding read when you can follow it.