The other way around. If inlet pressure drops below discharge pressure, some regulators will allow flow in the reverse direction (from discharge to inlet) because essentially the poppet is a check valve pointing backwards. It checks in the forward direction (ie: shuts off flow) when inlet pressure is higher than discharge pressure but allows flow in the reverse direction (when discharge is higher than inlet pressure). It is only pushed open by the diaphragm which is how the downstream pressure is controlled.
Examine the second picture above. Note where the word "Poppet" is on the left side pointing to a small section of stem that sticks out of the poppet. Note the small section of stem doesn't touch the diaphragm. As the pressure on the diaphragm drops (outlet pressure or "set pressure" drops), the spring above the diaphragm pushes the diaphragm down which pushes on that small section of stem on the poppet, pushing the valve open. But if the pressure is too high for the diaphragm to be pushed down by the spring, it's held up as shown in the picture. And if the inlet pressure is lower than the discharge pressure, the pressure force on the poppet (pushing down) will overcome the spring force that's below the poppet (labeled "Valve Closing Spring") pushing up, and allow flow backwards through the regulator.