The diffuse reflector prevents the signal strength from varying with the position of the scintillation. The specular reflector increases the signal strength at the PMT.
The radiation causes flashes of light in the scintillator. Of course, the flash of light does not always occur at the same point on the scintillator. The scintillator has a finite volume. The radiation can stimulate photon emission anywhere in the crystal. The PMT would be closer to some parts of the scintillator than to other parts of the scintillator. The relative intensity of a flash of light will vary with the position of the flash in the crystal relative to the PMT.
The probability of the PMT registering the flash of light could vary with the position of the The strength of the signal may vary with the position of the flash of light relative to the PMT. By "randomizing" the direction of the light wave vectors, the geometrical bias is reduced. The diffuser also prevents vibrations from causing variation in the signal strength
If the wave vectors are randomized, then that means that photons are moving in random directions. If the light from the diffuse reflector isn't refocused on the detector aperture, most of the photons will be lost. Therefore, to make sure more photons are needed one needs a focusing element to collect the photons after the diffuser has randomized their directions. A specular reflector collects some of the photons from the diffuser that are going in the wrong direction for detection.
If my conjecture is correct, the path of the detected photon has to be from scintillator to diffuser to specular reflector.
Diffusers eliminate "geometric information" from the signal. If you don't use it, geometric information can only confuse you. Some things look better without your glasses!