Its an experimental fact photons are transparent to and interact weakly with things like air, which is the usual environment of the double slit, and are absorbed by other things like the barriers. Why is actually a bit advanced:
https://www.physicsforums.com/insights/do-photons-move-slower-in-a-solid-medium/
'A solid has a network of ions and electrons fixed in a “lattice”. Think of this as a network of balls connected to each other by springs. Because of this, they have what is known as “collective vibrational modes”, often called phonons. These are quanta of lattice vibrations, similar to photons being the quanta of EM radiation. It is these vibrational modes that can absorb a photon. So when a photon encounters a solid, and it can interact with an available phonon mode (i.e. something similar to a resonance condition), this photon can be absorbed by the solid and then converted to heat (it is the energy of these vibrations or phonons that we commonly refer to as heat). The solid is then opaque to this particular photon (i.e. at that frequency). Now, unlike the atomic orbitals, the phonon spectrum can be broad and continuous over a large frequency range. That is why all materials have a “bandwidth” of transmission or absorption. The width here depends on how wide the phonon spectrum is.'
Thanks
Bill