I would say it may happen for following reason.
The first is that the rifle's crack is not produced in the chamber, it's going to be largely produced as the bullet exits the barrel. This comes from the sonic boom of the round and the pressure from the expanding gase that follow behind the bullet. When you fire the rifle, your right ear is going to be held down a little because you need to cock your head a bit to bring it over the barrel along the sights. So this could bring your left ear into more exposure to the pressure wave from the barrel. Finally, your right ear is going to be more along the line of the barrel than the left. I would expect that the pressure wave is perturbed more along the barrel than offset from it and so the pressure may be a bit softer on the right than the left.
Or it could be that your stance lessens the effectiveness of your hearing protection over your left ear. I never could stand using muffs because I usually shifted the muff a bit when I got into position so I always wore plugs. I don't know though, my hearing loss is in my right ear though I don't know if that comes from shooting or if was sourced from other problems (I know I started to get tinnitus in that ear after a really bad three day bout of migraines.).
