Or is it posible that i am thinking about the magnetic force all wrong?
yes to the extent your ideas do not match current mathematical descriptions...but "right" and "wrong" in physics is relative.
but the original post gets into a lot of complicated issues very quickly. You are inferring conclusions not based on observations...like watching two cars go down the road with the red car passing the white car...what can you infer from that about the relative top speed of the two cars? nothing...it's simply a false conclusion to say the red car has the faster top speed. That would have to be tested.
No one to my knowledge has been able the describe the magnetic field of a magnet in terms of photons; and electromagnetic fields have not been successful at describing the photoelectric effect...Einstein got a Nobel prize for figuring out a quantum description...
Classicial electromganetic "fields and waves" has not been theoretically combined with the photons of quantum mechanics; different theories and mathematics have been used to describe different phenomena...I think the best we can do currently is assert a photon is a quanta, a "particle", of the electromagnetic field...
edit: a related way to think about the "meaning" of things is this: If you have two different sets of rules for different mathematical calculations, and follow those rules correctly in each case, you have two "right" answers. But what answers are subject to experimental verification and what they actually MEAN is subject to interpretation.
Neither answer may match physical observations or both may, in which case some other prediction must be used to weed out the "right" from the "wrong" calculation. Then scientists may argue for 100 years, as in quantum mechanics, about about what the answer(s) mean. In other cases, scientists may get different answers, as in string theory, only to later discover (as Ed Witten did) that the answers are actually consistent!