They are perpendicular because that's how the Cartesian coordinate system is defined. In general, they don't have to be, in the sense that one can define a coordinate system in N dimensions consisting of N linearly independent basis vectors that can be at any angle w/r to each other. So for example in 2 dimension you could have a coordinate system defined by two vectors at 45° to each other, and still be able to uniquely name any position on this 2D plane as well as convert to the Cartesian coordinates.
But the Cartesian coordinate system is conventionally used in much of physics, and it requires the axes to be at 90° by definition.
The topic of vector bases is covered by linear algebra, so if you are going to be taking those classes, it'll all be explained there.