Of course, how hard or easy it is depends completely on your university, this years' lecturer, the material used, your math skills and your knowledge / insight.
Quantum mechanics has a reputation of being weird and incomprehensible (hence famous quotes like "For those who are not shocked when they first come across quantum theory cannot possibly have understood it," and "I think I can safely say that nobody understands quantum mechanics."). However, if you defer the philosophical discussion, if you will, to later and first just learn how to work with it, then I don't think it is much harder than any other physics course. Personally I first learned QM from Griffiths' excellent book, which starts by teaching you how to calculate things, then goes on to formalise the mathematics, then applies it to some more complicated problems, and only goes deeper into some of the trickier interpretation stuff in the end.