I agree that Chen's book is a good place to start. In order to understand most of the book, you primarily need to know electrodynamics as the level of Griffiths (and you can skip special relativity if you wish). It also helps if you know elementary kinetic theory as can be found in basic "modern physics" books (Tippler, for example). Knowing fluid dynamics first isn't necessary, but wouldn't hurt.
For the math, vector calculus, differential equations and elementary linear algebra will allow you to understand most of Chen. Being comfortable with Fourier analysis would make the waves chapter easier to interpret, but you can get a basic understanding withouta lot of knowledge. Contour integration is used some in chapter 7, but you can get a basic understanding of elementary plasma plasma physics without it.
While Charles Link is correct in general, you can learn a lot of physics from Chen without really learning linear response formalism. If i were you i would just dive into Chen right after griffiths. Also, i don't recommend any of Ichimaru's books for an introduction of any aspect of plasma physics for a beginner. My kinetic theory class was loosely based on volume 1 of "statistical plasma physics", which is no good for a beginner; the earlier version of that book, titled "Basic principles of plasma physics" is much better, but still not a good introduction to the field; "plasma physics: ..." is mostly a collection of topics chapters that cover some advanced topics very early...
I hope you enjoy the journey.
Jason