Matterwave said:
So how does one propose to take the difference of two undefined vectors?
I came in late, but as others have pointed out, displacement vectors live naturally in an affine space, not a manifold.
Wiki has some discussion of affine spaces, which may or may not be helpful.
One can more or less think of infinitesimal displacements in General relativity as vectors, though this isn't rigorous and may cause confusion. Hopefully it will be more helpful than confusing, but be warned it isn't rigorous and my description isn't totally textbook standard.
It turns out there are two types of vectors important in General relativity, called covariant and contravariant vectors. Under the right conditions, an infinitesimal displacement
dx can be regarded as being one of those types of vectors. The other type vector is related to the first by the duality principle, which basically states that if you have a vector space, the map from that vector space to a scalar (a number) is also a vector space. Infinitesimal displacements can be vectors, however finite displacements cannot be regarded as vectors. This is because finite displalcements don't commute - the order of the displacements matters. If you go x miles north and y miles east on a sphere, you don't get to the same point as if you went y miles east and x miles north, taking the displacements in the opposite order. Details of where you do wind up depend on what you mean by "go north" and "go east", but in general the order of the displacements matter. Vectors must commute, the order cannot matter, therefore finite displacements are not vectors for a general manifold.
A sphere is used in the above example to represent a more general manifold - a sphere is a particular sort of manifold that's not flat that is familiar, so it's a good visual aid to use a sphere to illustrate some of the more general principles of more general manifolds.
The set of vectors at any single point on the manifold form a vector space, which is also an affine space since any vector space is an affine space.
The name for the vector space formed by the set of all vectors at a given point on the manifold is called the "tangent space". To illustrate this, one usually visualizes the manifold as a sphere, and the tangent space as a plane tangent to said sphere. The vectors are usually visualized as little arrows drawn in the tangent plane/tangent space. With this visual technique, one can see that a small displacement on the sphere r would be the arc of some curve on the manifold such as a great circle, while the vector in the tangent space are represented by a little arrow drawn on a flat plane.