If I want to learn about tensor mathematics - where do I need to start (i.e. what precursory mathematical knowledge will I need) and does anyone have any good site/book recommendations?
If you don't already have any, then you'll need to have some background in linear algebra (and for starting tensors, you'll also need familiarity with calculus in R^{n} space (multivariable calculus)). This book covers the multivariable calculus, but you'll need to learn linear algebra from some other source.
Besides linear algebra and multivariable calculus, it's useful to have at least some basic abstract algebra to understand the language and notation used in definitions.