The usual example might help. To make the problem less abstract, let's talk about vectors not in a curved 4-d space=time, but on the curved 2-d surface of a 3-d sphere. The sphere is 3 dimensional, it's surface is 2 dimensional.
We can imagine a 2-d plane tangent to the sphere at some point. P, as in the image below which I borrowed from google images:
We can imagine and manipulate vectors in the tangent plane as we would on any other plane. The abstract property of these vectors is that they can be multiplied by scalars, and that they add, and that both processes are commutative. We'll focus on the addition property in particular - if ##\vec{A}## and ##\vec{B}## are vectors, ##\vec{A} + \vec{B} = \vec{B} + \vec{A}##.
This is true for the vectors in the tangent plane, but it's not exactly true for displacments on the surface of the sphere. I'm not sure whether to go into more detail or not - my judgment call is that it's best to go into a little more detail with an example, but not a lot of detail at this point.
If our displacements are "small" compared to the diameter of the sphere, the vectors "almost commute". But this sort of fuziness is not suitable for a rigorous mathematical treatment. To define vectors that always commute, no matter how large, we need to introduce the tangent space.
For a very crude example, consider a displacement so large that it wraps halfway around the surface of the sphere, to the antipodal point. If the displacement were a vector, we would say ##\vec{V}## + ##\vec{V}## = ##\vec{0}##, where ##\vec{0}## is the identity element of a vector space. If displacments were vectors, simple algebra would allow us to say that ##\vec{V} = 0##. But the antipodal displacement operator is not zero, it's not an identity element, so we have mathematical inconsistencies.
So, the bottom line is that we regard vectors on curved manifolds as being defined in the tangent space. Consiering the example above hopefully demonstrates why this is necessary.