Exactly. In an "affine space" you have points and some kind of linear structure but no "zero" point and so can't add or subtract points as you can vectors.
For example, in R2, once we have set up a coordinate system, you can associate each point with the vector represented by an arrow from the origin to that point (exactly the kind of thing you do in Calculus). That gives you a "vector space". But that depends on the coordinates system-an there are an infinite number of different choices for a coordinate system. Without the coordinate system you just have R2 as an "affine space". You can calculate the distance between two points but you can't add or subtract points.
a nice way to compare the two is this i think: imagine a flat affine space, everywhere homogeneous but no origin or coordinates. then consider the family of all translations of this space. those form a vector space of the same dimension, and the zero translation is the origin.
given any point of the affine space, any translation takes it to another point such that those two ordered points form a vector that determines the translation. vice versa, given two ordered points of affine space, i.e. a vector, there is a unique translation taking the foot of the vector to the head.
so there is a natural way to construct an associated vector space from an affine space, such that the vector space acts on the affine space by translation.
and if we fix any one point of affine space, i.e. an "origin", then this sets up a 1-1 correspondence between points of the affine space and elements of the vector space.
so this is a special case of a group acting on a set, and here the action is fre and transitive, so the set is a homogeneous space for the group.