For instance, when I look in the mirror, my right hand is to my right. In fact, the image I see in the mirror is inverted horizontally compared to what other people see when they look at me. In fact, I am not seeing me as I am, but a "mirror image" of me. Isn't that where the term mirror image comes from in the first place?
It doesn't surprise me though...take every point my body and map it to the opposite wall using straight lines perpendicular to the wall. It's easy to see why the image on the wall is necessarily inverted. Only if the observer could go through the wall and view the image formed by the points from the other side, would what he see match with what I look like to him. Is this making any sense?
For instance, when I look in the mirror, my right hand is to my right.
Yes, but your right hand looks like the left hand of a person standing in front of you.
The point is nothing is "flipped." Everything that was to your right in the "real" world is to the right in the "image" world, everything that is up in the real world is up in the image world and so on. That's just simple reflection which is essentially what you are attempting to express in your "model."
If you draw lines from points in the real world to corresponding points in the mirror, the lines will never cross, neither vertically, nor horizontally.