A free oscillating and decaying (R)LC circuit will have noise that's generated in the resistive element. For a resistor, the thermal noise power Pn = kTB. That suggests if you measure the Thermal noise power, then it will depend on the bandwidth (1/Q) of the circuit. That sort of justifies the above statement.
But Phase noise (and amplitude noise) will also originate in the Energy Sustaining circuit that drives an LC oscillator. The loop gain of such a circuit must be just right to compensate for the energy loss per cycle of the LC oscillation. The noise power admitted into the LC resonator from outside will also be proportional to the resonator bandwidth and that effect goes in the same direction as the first argument. The higher the natural Q, the lighter the sustaining power input needs to be so the looser the coupling.
A mechanical equivalent to this is a pendulum clock. Every swing of the pendulum allows the escapement to move by one click and energy is supplied by the escapement (a slight slope on the physical contact gives the pendulum a slight push to maintain its amplitude of swing). If the pendulum bearings are poor then more sustaining energy is required and variations in the power supply (gear train from dropping weight to escape fork) will affect the frequency and hence the clock timing. I remember an old boy pointing out to me that the tick on an expensive watch movement is much quieter than the tick from a cheapo Timex child's watch because the child's watch has more losses in the mechanism. (Other cheapo watches were available at the time and Timex was good value for a high risk instrument)