I've often thought about that. Seems to me you cannot have a 'point A and a point B' alone. Because as points there's no way to measure their separation.
A dimensionless point has no dimensions in itself. In a surroundings lacking dimensions there's nothing we can say in terms of dimension.
Given a third point we can start measuring. That third point could be part of the first point if it is given dimensions, thickness, height, width...
Then the distance from A to B becomes in terms of A the number of times you can divide the width of A into the interval.
Hence the need for three. Three is the smallest number.