Not satisfied with the explanation for refraction I have got

[whyee]

I've basically read that refraction happens because when a part of a light wave enters a medium where it displaces faster or slower at an angle, the part in the new medium "bends" the light wave because of the change of velocity, basically "turning" the whole light wave. I've made two diagrams illustrating the idea:

I made the light wave flat because it's easier to draw, but the bending idea is the same. The part where I'm having trouble with that explanation is that a light wave is not actually solid, but made up of individual photons (of course because of wave-particle duality, they aren't really particles, but I've illustrated them as red dots to visualize the wave broken down into individual photons better).

In the 2nd diagram, half the light wave has crossed into the new medium, where it displaces faster. Each photon has gained more displacement the longer it has spent inside the second medium (so the furthest right photon has gained the most). The grey bar outlines the "solid wave" idea, and the grey arrow outlines the new direction of it.

But since the individual photons aren't actually connected, why don't they just continue traveling the same direction as before (black arrow), instead of traveling in a new direction (grey arrow)?

 

[TGlad]

You get diffraction even with a single photon. The short answer is that light is taking the shortest path. If if didn't bend then it would take longer as it would be going through more glass which is slower.
Roughly speaking, light takes the shortest path because longer paths cancel each other out on average.
That's a quantum mechanics answer to a general physics question. The classical physics answer would be to think of light as a wave in the diffraction situation, not as particles.