Linear Algebra is harder than calculus only in the fact that linear algebra is rigorous and elementary calculus is, for the most part, based on intuition
Analysis, which is pretty much just rigorous caluclus is, and I think everyone would agree, harder than linear algebra - imo as far as branches of maths go linear algebra is probably one of the nicer ones, I believe Strang said 'linear algebra is like analysis but everything is behaving nicely'
Perhaps you should learn some set theory and group theory before you start on linear algebra? That way you'll see how you build linear algebra up, starting with sets, adding a set of operations on the set, then adding an action of another set, called a field, on the set.
Or maybe try reading a basic proofs book, like 'How to Prove it - A Structured Approach', that might give you some kind of insight into how to do proofs in linear algebra and introduce you to a level of abstraction beyond the 'intuitive feel' that you're used to.
Either I'm going to have to go back to the beginning and write down every single thing I understand and ask questions about every single thing I don't understand
This is a good approach, if you know how to prove results then slowly they'll seep into you as if they were obvious facts
or just slog through it with only minimal understanding and hope I can get through QM without it.
Quantum Mechanics is pretty tricky on it own, without a solid background in linear algebra you're going to get lost before you even start. You'll also want to understand some basic analysis facts here too.
I'll also reccomend Gilbert Strangs "Introduction to Linear Algebra" if you haven't already read it.
The mistake they make is they assume the real basic of LA are too easy and they jump immediately to these abstruse problems that no one cares about
You'll care about those things one day, I think most people felt the same way about the isomorphism theorems when they first saw them but later on realized that it's quite a handy little fact to have with you. (I think a lot of authors don't point out enough when a theorem is going to turn out to be especially useful later on, so you end up wading through a sea of theorems and proofs without any idea why these are important, which is why it's a good idea to work with them for a while and try and apply them to a problem even if it is taylor made to require the use of your newly discovered theorem)
I can't even master the mechanics of LA or the notation and then they shove these hard problems in my face without teaching me how to do the real easy stuff.
If you're having trouble understanding notation then you just need to work with it for a while really. Usually when I start on a new textbook and the author introduces a whole new set of notations I need to stop for a second and look at the definitions, absorb what they mean and perhaps try and make a few examples.
I also agree 100% with Vandium50 and micromass, you've got to stop relying on there being numerous sources (especially lectures online, textbooks imo are far superior to any lecture you haven't personally attended) and you need to know how to derive the results given in the textbook/lectures.