There seems to be a bottleneck of understanding here, combined with a fear of mirrors.
A mirror for an EM wave can be made from a flat conductive sheet. The reflection is from the surface layer of conductive atoms. If the wave penetrated more than one atom deep it would suffer multiple internal reflections which would increase energy losses within the mirror.
We can know that the zone of reflection must be spread over a depth of less than λ/4, or we would see destructive interference of reflected light.
An incident magnetic field causes a perpendicular current to flow on a conductive surface. That re-generates a perpendicular magnetic field, now opposite to the incident field. Turning left twice is the same as going back the way you came, i² = –1, reflect on that. The incident and reversed fields cancel into the mirror, so the incident energy must be carried away from the mirror in a reflected wave.
Since the incident and induced fields cancel into the mirror, the time needed to reverse must be very close to zero, or the phases into the mirror would not cancel, and the mirror would be lossy, making an inefficient reflector of energy.
This all suggests that the time delay of a mirror is less than the time needed to travel the ionic radius of a conductive atom. We do not know where the effective reflective surface of a mirror is, until we look near the face of the mirror, at standing waves formed between the incident and reflected rays . But it is the reflective surface we are interested in, so there is no problem, and no delay.