The force that causes the balloon to move the other way is thrust, the same force pushing rockets into orbit. The particular thrust here comes ultimately from the elastic force of the balloon trying to compress back to its original shape, but having to work against the pressure of the gas inside.
Thrust itself comes from conservation of momentum.
If you throw something away from you (for the balloon, this would be a stream of air), you experience a kickback force (i.e., thrust), because the total momentum of you plus the thrown object must add up to the same value before and after the throwing event (assuming no other external forces).