The weirdest and most puzzling problem I've ever encountered

[Eaglefrenzy]

Your problem is with your perception of the image you see. There also seems to be more than one lack of understanding going on as well.
First, your eye sees reflected light in a straight line to the surface of the mirror. There is no reversal in any direction. Anything moving left to right in relation to your eyes moves left to right in relation to your eyes in the mirror as well. An object moving up in relation to your eyes moves up in relation to your eyes in the mirror. You can do a simple but eerie experiment with three hand-held mirrors and a coin, in which you reflect a virtual image of the coin to a place on the table where it looks like you can just reach down and touch it. When you try its not there even though your brain sees it!
Second, you seem to believe that there are two actions. The first being you moving your hand, the second being the image moving in the mirror. This is probably the one confusing you the most. Ther is ONLY ONE moving part (your hand), and you are only seeing it's reflected light in the mirror. Once again notice that where things are in relation to your eyes is where they will appear to you in reflection. Extra mirrors would change this effect.
Third is an "Oh By The Way". A cyclops would see a one demensional image. No second eye means no depth perception, and objects would not appear to have or change depth.

 

[rcgldr]

There is no reversal in any direction.

Except there is a reversal from front to back. Move an object forwards in front of you away from you, and the image of the object appears to be moving towards you.

 

[Defennder]

I don't see how the explanation of the mirror reversing front to back instead of left-right helps here. If the mirror were instead to reverse up-down instead of left-right, it would still be reversing front-back, wouldn't it? Look at the concave part of a spoon for example.

 

[jtbell]

If you hold up a piece of clear glass or plastic with text written on the piece of glass or plastic, so that you can read the text on the glass, you'll also be able to see the same text in the refelction, left and right are not swapped.

Bingo!

If you write the text on paper, and hold it in front of you so you can read it directly, then you cannot see the text reflected in the mirror, because the wrong side of the paper faces the mirror. In order to see the text reflected in the mirror, you have to flip the paper. It makes a difference whether you flip the paper left-to-right or top-to-bottom!

 

[Eaglefrenzy]

There is absolutely no reversal from front to back. Once again this is a perception problem. If an object gets closer to the surface of the mirror (in relation to your eyes) the image appears to get closer to your real eyes because you are seeing its light reflected off the surface, and there is now less distance to the surface of the mirror. The virtual image you are looking at appears as if that object is getting closer to the surface of the mirror in relation to it's (virtual) eyes. That's not a reversal unless you are caught up with the image being a tangible thing. Don't be confused because it "looks so real".