Grandfather paradox is not a proof of impossibility. There can be solutions to this problem, such as foliation of space-time.

An analogy is this: if you are moving in complex plane and are evaluating complex logarithm of your current location, the value of the logarithm always changes smoothly as you move.
However, if you move along a closed loop around zero, the evaluated value, while it is still always smooth, does not return to the same value: you have "returned to the same place", but this is not really the same place.