The thing is that up until now, this has never been the correct approach. I mean what if Newton had said "well, there are these Kepler laws and maybe we can not understand them, we have reached the limit of our comprehension so let's just accept them as empirical laws. That's the way Nature is. They work well and that's all we should care about"
Instead he went "I'm sure I can figure this out"... and he did, to a certain extent.
We could repeat this pattern for many major discoveries. And so on.
So why somehow should we reach this point now and say "we can't understand this correlation business. That's just the way things are. It works well and that's all that matters" ?
Maybe QM is the end of the road. Maybe QM is fundamental the way it is and we'll never uncover a deeper principle. But I would not bet on it.
Well, classical E&M can be recovered from QED so this is clear.
But, if you are asking "why is QM fundamental", I would have to say that there is no reason to be sure of that! It might well be a kind of approximate theory to something else (maybe even more weird). I am my opinion (and that's just that, a very personal opinion), the nonlocality exhibited in Bohm-Aharonov, Bell's inequality type of experiments, which way experiments, etc are the strongest hints that we should think about some more fundamental principle. I am apparently almost alone in this

. It sounds as if most people just say "well, no information (in the usual sense) is transmitted, no energy is transmitted so everything is fine. End of story. Whereas I think that a more fundamental theory would present a more clear picture of the measurement process, of the collapse of the wavefunction, etc.
But it seems that people have got so used to the weirdness of QM that it does not elicit much desire to dig deeper.
Pat