OMFG, thank you so much. It's finally working now.
btw amazing deductive reasoning there.When you render the spheres, you probably just call a transform, so you still make calls to render spheres as if no rotation happened, and the rotation matrix in place takes care of it. But not in physics. So your spheres end up moving on the screen, but not from perspective of simulation.