This is true, there are no doubt galaxies whose 'future light cone' (the region of space that light from them can reach) will never intersect our position.
However the mass of these galaxies is included in our model of the universe. On large enough scales we see that the universe is homogeneous, that is to same, the same density on average everywhere. The scale on which we see the homogeneity is much smaller than the size of the observable universe. This means that the total gravitational acceleration on each part of the universe is zero (so we are moving towards Andromeda, while the whole local groups moves towards the Great Attractor but this is because the universe is not homogenous on these smaller scales). Hence the mass of these galaxies won't be pulling nearby galaxies away, since there is as much mass pulling them the opposite way such that the total acceleration is zero.
Interestingly though, there are theories that suggest that even though the universe appears homogeneous, it may be possible that on scales larger than the observable universe the universe is not homogeneous and perhaps we are in some over or (more commonly suggested) an underdense region. This has been put forward as an explanation for the accelerated expansion of the universe. The theory is problematic since it is hard to explain why the universe would be lumpy on small scales, very smooth on large scales but then get lumpy again on even larger scales. It also isn't supported well by any observational evidence (which would be hard, since by definition we can't see beyond the observable universe!) but I believe it hasn't been ruled out, in that I don't think we can point to something and say 'it would look like this if the theory was true but it doesn't'.
If anyone is interested in these theories, the technical name they go by is "super horizon sized perturbations".