We are talking about the case where the magnet is fixed to the disk and rotates together with it. Surely magnetic field of the magnet will not separate from its source but will always be at the same location relative to the magnet and at the same location relative to every point on the disk.
As you said, the magnetic field is a vector field, an arrow at every point in space. Treat the magnet as the source of the field, as you say, and rotate the disk by a a fixed angle about its axis. Will the arrows in space change, given the symmetry of the magnetic source about the axis of rotation?