Why observables are represented as operators in QM?

[MichPod]

TL;DR Summary

What are the (core) reasons observables are represented as operators in QM?

Can somebody provide an explanation why the dynamical variables/observables are represented in QM as linear operators with the measured values being eigenvalues of these operators?
For energy this is probably trivially and directly follows from the stationary Shrodinger equation which solutions are eigenvectors of the Hamiltonian operator with the eigenvalue being the energy, but why on Earth one would try to introduce the same machniery for any other possible dynamical variable and why does this works?

I know that an operator of an observable may be represented as a sum of eigenvalues multiplied by an outer product of the eigenvectors, so it may be considered that the operator just directly encodes the results of measurements and corresponding orthogonal eigenstates, but I wonder whether some more may be said about the nature and origin of observable operators formalism in QM and whether some additional physical meaning may be attributed to the measurement operators other than merely mechanically encoding the eigenvalues.

 

[Morbert]

In both classical and quantum physics, observables are technically operators. It's just they are more general in QM. The more general operators in QM permit a more general algebra of possible events, which is needed to make correct predictions.

 

[DarMM]

In both classical and quantum physics, observables are technically operators. It's just they are more general in QM. The more general operators in QM permit a more general algebra of possible events, which is needed to make correct predictions.

Indeed and furthermore of interest is that the event algebra is of such a form that noncontextual probabilities on it seem to require the Hilbert space formalism and the collapse postulate:
https://arxiv.org/abs/1702.01845
So the algebra of possible events of our world requires a probabilistic treatment based on Hilbert spaces intrinsically.

 

[kith]

Operators / matrices were first introduced in Heisenberg's 1925 paper where he tried to find the quantum analogue to the classical equation for the power radiated by an oscillating electron by incorporating the rules of old quantum mechanics (Bohr frequencies, Ritz combination principle). For the details, see Aitchison et al.'s take on Heisenberg's paper.

Heisenberg's guiding principle to use only observable quantities led him to replace the unobservable classical trajectory $x(t)$ of the electron by a collection of transition amplitudes $\langle m |\hat x | n \rangle \exp(i \omega_{mn}t)$ which we now recognize as the matrix elements of the position operator.

He notes that if we want to combine two such collections which represent different observables we get non-commutativity. This is an important ingredient for central aspects of QM like the Heisenberg uncertainty principle. So the simple story is that while numbers are sufficient for classical mechanics we need operators to get the non-commutativity of QM.

The more complicated story is that the line between classical theories and quantum theories isn't as clear cut as it may seem at a first glance which is what I think the other replies are referring to.

 

[Michael Price]

Indeed and furthermore of interest is that the event algebra is of such a form that noncontextual probabilities on it seem to require the Hilbert space formalism and the collapse postulate:
https://arxiv.org/abs/1702.01845
So the algebra of possible events of our world requires a probabilistic treatment based on Hilbert spaces intrinsically.

Yes, classical physics can be formulated in terms of matrices and operators, along with a non-controversial collapse postulate, because the classical wavefunction now really does just encapsulate our knowledge of the system.

The important difference is that all classical operators commute with each other.