I found my grad physics courses to be way more challenging, for one major reason. As an undergrad, if you struggle with a particular concept, there are plenty of other resources. If you struggle in Griffiths quantum, maybe you can find a different explanation (that might "click" more) in Sakurai, or by looking at Khan Academy, posting for homework help on this forum, etc. Once your education gets more "specialized", those resources become few and far between.
I think the content of the graduate courses was more difficult, but I was a better student in grad school than I had been as an undergrad. I was older, more mature/serious about studying; plus I had four years of undergrad experience in "learning how to study," if that makes sense.