Restated from a crystallographic point of view, what is important is that A and B have the correct atomic sizes and electron configurations so that they can exist in a stable structure with a lower energy than that of the two parent structures.
For instance, if they have very similar atomic sizes, it is often easy for say, A, to replace the B atoms in the B-lattice. This may be feasible up to a point, beyond which adding more A to the mixture causes too much strain in the lattice, and a separation of phases occurs. Alternatively, if A and B have vastly dissimilar sizes, it may be possible for A (assuming it is the smaller atom) to nicely occupy some interstitial voids in the B-lattice. Again, adding too much A could increase the strain energy beyond a point where the excess A would rather form a separate phase.