Without knowing any properites of the system like dimensions, mass and inertia, speed, and the deceleration rate for your braking, this is really just guessing possible causes - but they are both "real" effects that can occur with the right conditions.
1. The shear force of the shaft's own weight, plus the torque when it decelerates, combine to create an unsymmetrical stress distribution in the shaft with a higher stress on one side of the shaft than the other. This could be deforming the shaft so the braking force also becomes unsymmetrical around the circumference. In extreme conditions, you can finish up with plastic deformation of the shaft into a helix shape. If the plastic yielding is local rather than global, it may look like a "plastic hinge" between two straight sections of shaft.
2. Whirling. In most situations, the most interesting (i.e. troublesome) whirling modes are the synchronous forwards whirl modes, which are excited at specific RPMs because the shaft is not perfectly balanced. However, at any shaft speed there are potential non-synchronous whirling modes that can travel in either direction relative to the shaft rotation. (Forwards whirl means whirling in the same direction as the shaft rotation, backwards whirl means whirling in the opposite direction.) Friction forces can excite backwards whirl modes. These can be potentially unstable, if the deformed shape makes the friction force unevenly distributed round the disk. The unsymmetrical force distribution also gives a resultant shear force. A typical situation would be that the brakes "grab" or "bind", the rotor stops turning very quickly, and most of the energy goes into a cantilever vibration mode of the rotor.
To add an obvious (well, obvious with hindsight and past experience!) comment the dynamics of the system: the vibration and whirling modes may look very different when the brakes are on or off, since the brakes will form another "bearing" on the rotor system.