No there is not a general consensus.
Linear algebra is for many people their first introduction to proof based and abstract mathematics, and if you are not prepared for it then that can be hard, and for some people very hard. If you are able to understand proof-based and abstract mathematics fairly well, then linear algebra may be a breeze for you, but for most people it is quite challenging.
Probability modelling (I'm assuming it is an introductory course in probability and possibly statistics) can be moderately difficult, but I have never seen anyone with the perspective that such a course is ridiculously hard (though of course it can be made so, but usually isn't). To a large extent you will learn some new concepts, compute integrals, sums, etc. Most concepts are fairly well-grounded in the real world. Sometimes it is taught together with a programmable statistics package like R, and in that case some people can find that a bit hard if they have never seen programming, but usually an introductory course is light on the programming.
Generally I would say that the concepts in linear algebra tends to be harder, but there are also fewer of them so you spend more time on each one.
Personally I found probability modelling harder because I had plenty of experience with proofs so Linear algebra was easy (the pace is obviously set so people who have little experience with proofs can follow).
In the end it depends on your preferences, previous experience, and how the courses are taught (they can be taught very differently depending on the professor's philosophy and the audience).