Why Quantum Mechanics Feels Difficult — Mathematical Bridge
Table of Contents
QM’s formalism
Strangely enough, quantum mechanics (QM)’s formalism isn’t any more difficult than other areas of physics. The mathematics of the “standard” QM isn’t any worse than, for example, electromagnetism. Yet to many people—especially non-physicists—QM presents a daunting effort to understand.
Why QM often feels difficult
I strongly believe this difficulty comes down to how we expand our knowledge. When we teach students new topics, we normally build on existing understanding: basic mathematics, simple mechanics, and so on. That foundation lets us introduce forces, motion, energy, and other topics in introductory physics, and then gradually add complications.
The same approach applies to helping a student with a homework problem: we ask what the student already knows, identify the underlying principle, and find the most general relevant equation as a starting point. From that foundation we build toward the solution.
In both teaching cases there is a clear STARTING point—a reference foundation on which other “new” material is built. We learn new topics by extending what we already understand. This crucial link is what is missing for many learners of QM. For most non-physicists, this absence is the main reason QM seems puzzling, and why some people invoke QM inappropriately for metaphysics or mysticism.
The Disconnect
There is a pronounced disconnect between our classical conceptual understanding of the universe and the formalism of quantum mechanics. There is nothing in classical mechanics that we can directly build on to understand QM. We use identical words—particle, wave, spin, energy, position, momentum—but in QM these concepts often have very different meanings. You cannot always explain them using classical concepts.
Example: spin
Take “spin” as an example. How do you use the classical idea of a spinning object to explain a spin-1/2 particle that only regains its identical symmetry after two complete rotations? We simply have to accept that the word is reused: in QM “spin” primarily denotes an intrinsic angular-momentum-related property that produces a magnetic moment. It has nothing to do with a classical object physically spinning.
So while the terminology is shared between classical and quantum physics, the underlying concepts are not continuous in the intuitive, classical sense.
Mathematical formulation
Interestingly, the mathematical formulation of QM is familiar. The time-dependent Schrödinger equation has the same structure as standard wave equations. We call the energy operator the Hamiltonian because it mirrors the Hamiltonian approach in classical mechanics. The matrix formulation is also a familiar mathematical tool. In short: the conceptual foundation of QM is disconnected from classical intuition, yet the mathematics follows naturally from existing mathematical methods.
Teaching implications
For this reason I disagree with the idea that students should be taught QM’s conceptual interpretations first. There is little conceptual material they can reliably build on. Presenting postulates or interpretations in isolation risks telling students conclusions “out of thin air.” Instead, the mathematical formalism gives a stable framework on which to hang understanding.
If a student lacks command of the mathematical formalism, they will not truly understand QM. Without the math, QM can appear mysterious or mystical; people will draw unwarranted philosophical or metaphysical conclusions because they lack the formal connection to established physics.
The mathematical formalism defines the quantum description. Conceptual descriptions are secondary and arise because we try to translate the formalism into classical language. That is why different interpretations can coexist: they all refer back to the same formal mathematical structure.
QM as musical notes
This is not to say QM is “just mathematics.” That would be like saying musical notation is “just ink on paper.” The notes are a means to convey sound unambiguously. Likewise, mathematics is the only communication precise enough, so far, to describe our universe reliably. The math reflects and captures phenomena; it is not merely decoration.
Bridging the gap
You cannot rely on pre-existing classical intuition to fully grasp the quantum concepts. There is a conceptual discontinuity, and the only smooth bridge is the mathematical continuity of the formalism. Learning that formalism is the practical route to making QM “make sense.”
PhD Physics
Accelerator physics, photocathodes, field-enhancement. tunneling spectroscopy, superconductivity
Why should this be ugly?
Explaining equations in plain English does not mean there are no more equations. It means that the equations are written to be teaching tools themselves, rather than merely something to learn. I never thought it possible, but apparently they've been there for quite a while, hidden beneath a pile of Google trolls, and now the trolls have disappeared. (Obama's parting gift?) Here's an example:http://www.imperial.ac.uk/media/imperial-college/research-centres-and-groups/theoretical-physics/msc/dissertations/2012/Alec-Owens-Dissertation.pdfI suppose you think this math is ugly?
[QUOTE="houlahound, post: 5588490, member: 551046"]Define understand and how you know you understand something.”As far as explaining physics to the public, I'd define understanding as a form of economic utility. Math is the only reliable way to grapple with the "Big Questions". If people are to expend time, money, and effort to learn mathematical physics, they expect to see how that's done. If that's not what they see promoted, they'll take the low road and learn nothing or less.
Define understand and how you know you understand something.Personally I think the word understand is vague and rather useless.
[QUOTE="houlahound, post: 5588474, member: 551046"]Yes, learn the math and see the beauty in full.”There's nothing beautiful about having to observe everything projected onto a random choice among real-valued probabilities. But that's how the actual world works, and that's the mystery the public wants to understand.
[QUOTE="houlahound, post: 5588450, member: 551046"]Didnt Hawking famously say – every equation he puts in a book takes X amount of sales off.Paraphrased.”That's valid for popular science books. For textbooks on physics the opposite is true: The more formulae the author offers, the more steps are made explicit in a derivation, for the student the more simple it is to follow and understand the argument ;-).
Yes, learn the math and see the beauty in full.
[QUOTE="houlahound, post: 5588460, member: 551046"]I believe that would be most uncompelling and ugly.”Sic transit gloria. You have a better idea?
[QUOTE="bhobba, post: 5588378, member: 366323"]Then get copy of Landau – Mechanics where all of Classical mechanics is derived from this alone”For something worthy of this honor, it would need to be fully consistent with the applicable regimes of Newton, Hamilton, Lagrange, Schrodinger, Heisenberg, Einstein (at least as far as SR), and also all the various types of material physics. And on top of all that, it also has to be profinite, so it can be put to practical use.Are you saying that the path integral is up to that challenge?
I believe that would be most uncompelling and ugly.
Supposedly, it's absolutely impossible to put the equations of modern physics into plain English. I don't buy that. Has any group of physicists ever even attempted to "Gutenbergize" QM for the public?
Imo
[QUOTE="houlahound, post: 5588079, member: 551046"]The mystical stuff excites the general public and sells nontechnical books.”And conversely? That humble, honest, up-to-date explanations of what physicists have actually found out don't sell to the general public? More likely, those who could reach out this way aren't even trying!
[QUOTE="Dr. Courtney, post: 5588067, member: 117790"]Ahh, but figuring out how to arrive at classical physics from QM may well require pushing the boundaries or tweaking how the QM toolkit is used. Our understanding of QM may need to change to address all the issues in the "classical limit."”Not really.Derive the PLA which is fairly simple as I have posted a number of times. You start out with <x'|x> then you insert a ton of ∫|xi><xi|dxi = 1 in the middle to get ∫…∫<x|x1><x1|……|xn><xn|x> dx1…..dxn. Now <xi|xi+1> = ci e^iSi so rearranging you get∫…..∫c1….cn e^ i∑Si.Focus in on ∑Si. Define Li = Si/Δti, Δti is the time between the xi along the jagged path they trace out. ∑ Si = ∑Li Δti. As Δti goes to zero the reasonable physical assumption is made that Li is well behaved and goes over to a continuum so you get ∫L dt.Now Si depends on xi and Δxi. But for a path Δxi depends on the velocity vi = Δxi/Δti so its very reasonable to assume when it goes to the continuum L is a function of x and the velocity v.In this way you see the origin of the Lagrangian. And by considering close paths we see most cancel and you are only left with the paths of stationary action.Then get copy of Landau – Mechanics where all of Classical mechanics is derived from this alone – including the existence of mass and that its positive. Strange but true. Actually some other assumptions are also made, but its an interesting exercise first seeing what they are, and secondly their physical significance.You want to understand classical mechanics – this is how to do it. Honours students with calculus BC behind them and a course in multivariable calculus in parallel could do it. In fact it would probably be beest teaching an integrated course with chemistry along the lines of:Multivarible CalculusLinear AlgebraQuantum MechanicsClassical MechanicsChemistry taught as probability models like at Princetons integrated science.ThanksBill
I don't think your linguistic comment is useful nowadays. Adding polysemia (or simply erasing an old meaning to some words and replacing it with a new meaning) is not a scientific process anymore. New theories come with new words.
[QUOTE="Dr. Courtney, post: 5588067, member: 117790"] … “Dr Courtney – Thanks for the link in your signature to FFT source code.
We still use the term "planet" to reference those objects in the sky called planets, even though the word "planet", historically speaking, means "wandering god". But rather than change the word we've just changed the meaning of the word. And it was the invention of the telescope that altered the meaning. After looking through a telescope, at the god Saturn, we discovered Saturn was ball shaped and had a ring around his belly. A similar change in meaning has occurred in the use of words within quantum physics. An early model of the atom had the electrons orbiting the nucleus in the same the way the moon orbits the Earth. But it soon became apparent, from observations, that this model was inadequate. A better model was elaborated. One can still find use of the word "orbit" in relation to electrons but it has a different meaning.So we can see from this anecdotes that it's not a question of working out how to reconstruct wandering gods from planets, or moons orbiting the Earth from electron interference patterns. It's how to do the reverse.
The mystical stuff excites the general public and sells nontechnical books.
[QUOTE="Collin237, post: 5588003, member: 577701"]I think that's the heart of the problem. There's no such thing as physics "as of now" versus future physics. The laws of physics have existed long before us and will continue to exist long after. They don't change; only our understanding of them changes.There must be a continuity between quantum and classical physics, because classical matter is made of quantum stuff. That we haven't discovered it is no excuse for decreeing it doesn't exist. The idea of "Von Neumann's split" is untestable, unwarranted, and hands QM to mystics on a platter.QM is the produce of science, not faith; and it must be taught as such. A successful student of QM has the right and duty to question the answers just as much as they answer the questions, provided they stay within the QM toolkit and use it correctly.”Ahh, but figuring out how to arrive at classical physics from QM may well require pushing the boundaries or tweaking how the QM toolkit is used.Our understanding of QM may need to change to address all the issues in the "classical limit."
[QUOTE="Collin237, post: 5588003, member: 577701"]I think that's the heart of the problem. There's no such thing as physics "as of now" versus future physics. The laws of physics have existed long before us and will continue to exist long after. They don't change; only our understanding of them changes.There must be a continuity between quantum and classical physics, because classical matter is made of quantum stuff. That we haven't discovered it is no excuse for decreeing it doesn't exist. The idea of "Von Neumann's split" is untestable, unwarranted, and hands QM to mystics on a platter.QM is the produce of science, not faith; and it must be taught as such. A successful student of QM has the right and duty to question the answers just as much as they answer the questions, provided they stay within the QM toolkit and use it correctly.”You quoted it out of context. Here's the entire passage that are relevant to that statement:[quote]We use the identical words such as particle, wave, spin, energy, position, momentum, etc… but in QM, they attain a very different nature. You can’t explain these using existing classical concepts. The line between these two is not continuous, at least, not as of now. How does one use classical idea of a “spin” to explain a spin 1/2 particle in which one only regains the identical symmetry only upon two complete revolutions? We simply have to accept that we use the same word, but to ONLY mean that it produces a magnetic moment. It has nothing to do with anything that’s spinning classically. We can’t build the understanding of the QM spin using existing classical spin that we have already understood.[/quote]Notice that I did not say that QM cannot merge to explain classical observations. That wasn't what is "discontinuous". Rather, it is the USAGE of the terms such as "spin, position, etc. You cannot simply bring over the classical concepts of such things and expect them to be the same in QM. I used the concept of of "spin" as an example. We use the same word, but that word has a different meaning in QM.This is NOT the "Von Neumann split".Zz.
“The line between these two is not continuous, at least, not as of now.Reference https://www.physicsforums.com/insights/quantum-mechanics-difficult/“I think that's the heart of the problem. There's no such thing as physics "as of now" versus future physics. The laws of physics have existed long before us and will continue to exist long after. They don't change; only our understanding of them changes.There must be a continuity between quantum and classical physics, because classical matter is made of quantum stuff. That we haven't discovered it is no excuse for decreeing it doesn't exist. The idea of "Von Neumann's split" is untestable, unwarranted, and hands QM to mystics on a platter.QM is the produce of science, not faith; and it must be taught as such. A successful student of QM has the right and duty to question the answers just as much as they answer the questions, provided they stay within the QM toolkit and use it correctly.
[QUOTE="vanhees71, post: 5563803, member: 260864"]In highschool we had a long unit about "Bohr-Sommerfeld atoms". After that the teacher told us that we have to unlearn it and introduced the wave-mechanical picture a la Schrödinger (with the correct probability distribution). That was hard to digest. I never understood, why she had not introduced the wave-mechanical picture from the very beginning.”If nothing else, it teaches future scientists that theoretical physics is about constructing models that in the future can be superseded by better models. Some physicists attach too strongly to theories that were state of the art when they were young. The pedagogy of learning theories soon to be rejected and replaced by better theories may prevent this.
In highschool we had a long unit about "Bohr-Sommerfeld atoms". After that the teacher told us that we have to unlearn it and introduced the wave-mechanical picture a la Schrödinger (with the correct probability distribution). That was hard to digest. I never understood, why she had not introduced the wave-mechanical picture from the very beginning. That would have spared us from the hard "unlearning process". In university we had an "Introduction to quantum mechanics" before the theory-course lecture (QM1). There also they started with a historical introduction, and again the Bohr-Sommerfeld model was taught in some detail (this time of course using what we had learnt in the classical mechanics lecture, i.e., Hamilton and phasespace formulation). At the end of the lecture we had of course also covered Schrödinger wave mechanics. QM 1 was then the "true" representation free formalism a la Dirac, which was a revelation for me. All of a sudden one could understand the structure of the theory, the meaning of states, operators representing observables, their eigenvalues and eigenvectors. I think, it's better to go in a way as you described, i.e., a very brief discussion of only a very few lectures with history of the "old quantum theory", but not in much details but rather a description of the "development of ideas", which finally lead to modern quantum theory, which was invented independently by 3 groups (roughly): (a) Heisenberg (idea in his famous but imho quite enigmatic Helgoland paper), Born, Jordan (formalism), Pauli (applications, including the solution of the non-relativistic hydrogen atom) => "matrix mechanics"; (b) Schrödinger ("wave mechanics" including a lot of the standard methods and applications we still learn today in the more wave-mechanics oriented lectures/books; equivalence between wave and matrix mechanics); (c) Dirac ("transformation theory"; basically the representation free formalism we learn today still almost as Dirac formulated it as "bra-ket formalism"). Then there was of course the more mathematical branch starting from von Neumann, leading to the development of modern functional analysis. Already the book by Courant and Hilbert played an important role for Schrödinger in developing wave mechanics.
In my college days, some while back that is, we first had a „physics of atoms and molecules” course which indeed put forth in the first 2 lectures the history behind QM but without going into too many details. The only detailed historical models were: the blackbody radiation and Rutherford's scattering theory. The 3rd lecture had the Schrödinger equation already underway and that is it. Then, the next year, the proper QM course had as 1st lecture: „Prehilbert and Hilbert Space. Orthonormal systems”. No pre 1925 content whatsoever. I was happy with this. Still am.
I don't know, who first did the derivation I've posted in my insights article. Nowadays it's a standard exercise for 1st-order time-dependent perturbation theory.https://www.physicsforums.com/insights/sins-physics-didactics/
[QUOTE="vanhees71, post: 5563680, member: 260864"]Well, to answer your final demand, what's the significant physical difference between classical and quantum physics, I'd say it's the possibility of "entanglement".”But even entanglement can be viewed as an application of superposition (or maybe path-indistinguishability) to 2 objects (or a tensor product Hilbert space). So is it 'entanglement' that's fundamental or superposition?For me the essential difference boils down to the fact that in a classical phase space one can, in principle, distinguish between two arbitrarily close states (q,p) and (q',p') but in QM the very concept of distinguishability is tied into the notion of orthogonal states so that 2 non-orthogonal states can, with a certain probability, 'mimic' the other in a given experiment.[QUOTE="vanhees71, post: 5563680, member: 260864"]among which are Einsteins 1905 approach to the photoelectric effect, which in fact does not prove the necessity to quantize the electromagnetic field”Indeed – wasn't it Lamb (and Scully?) who showed this way back when, something like half a century ago?But I think the 'photoelectric' part of that paper is, for me, perhaps the least wonderful bit :-)
Well, to answer your final demand, what's the significant physical difference between classical and quantum physics, I'd say it's the possibility of "entanglement". Einstein called it the inseparabality of far-distant parts of a quantum system, and that's a very precise description. The possibility of the corresponding correlations between parts of a quantum system is not understandable from the point of view of a deterministic local theory, as has been shown by Bell. I agree, however, with your final statement that to start with this is not the best way to introduce QT. For that you need a good understanding of the formalism. I also tend to a mixture of the historic approach but telling from the very beginning that one should not get too much involved in old and partially inaccurate accounts of the history, among which are Einsteins 1905 approach to the photoelectric effect, which in fact does not prove the necessity to quantize the electromagnetic field (see my Insight article on this "sin" of physics didactics), the Bohr-Sommerfeld model of atoms, and inconsistent pictures like the wave-particle duality, which are all clarified and avoided by modern QT.
I have no idea what the 'best' way to teach QM is – and I oscillate between the historical approach and the formal. In one sense, since the maths can be quite easy (if you teach the basic principles and axioms using 2 state systems), it just becomes a matter of (initially) setting down the recipe and cooking and seeing what it tastes like at the end. After working out several meals in this fashion, so we become comfortable with just applying the recipe, one can then go back and try to figure out what the recipe actually means.QM, in its traditional 'textbook' form is such a massive disconnect from all the physics we learn up to that point – that this might indeed be the best way to do it, for most.But I also think there's merit in the historical approach too. Presenting the bare axioms, drawn from the magicians hat like some hapless white rabbit, and just working out the consequences has the potential to leave the student with the letters WTF indelibly tattooed on their forehead (my tattoo is fading a bit now).I wanted to know 'why'. Why did those amazing physicists like Planck, Einstein, Bohr, Heisenberg, Schrödinger, Dirac (and so on and so on) feel it necessary to change things so radically? Why were they forced to adopt this (on the surface) really bizarre set of axioms that looked like nothing that had gone before? Whatever possessed them?We get some insight into this by looking at the original papers. Einstein's 1905 paper on the photoelectric effect is, for me, a tour de force of theoretical physics. It's quite simply breath taking. A few years later (in 1909 I think, but not sure) he published a paper showing that the fluctuations in the black body spectrum are partly to due 'wave' fluctuations and partly due to 'particle' fluctuations – astounding stuff indeed – hitting the fundamental kookiness that was looming squarely on the head.Do we get a better insight into QM from this? I don't know – but you certainly get a better sense of why QM was needed at all – and why the classical approach that had been so successful just fell apart. I think it's important to 'understand' QM by looking at how it differs from classical thinking. We can make the difference seem 'small' by investigating the formalism (let's call QM a C* algebra but with non-commuting variables, to give a somewhat oversimplified example) – but ultimately there has to be a significant physical difference – and it's understanding that difference and why it's there is (probably) crucial for really grokking things.But is this the best way to present QM in the first place? Don't know – I suspect not.