If two excited states have different energies, they will be orthogonal because eigenvectors with different eigenvalues are always orthogonal.
However, if the two excited states have the same energy, they can be different, but not orthogonal.
In the standard version of QM, the explanation is that one has two sorts of things, states and operators. The states are members of a Hilbert space. The operators operate on the Hilbert space.
There is another way of describing the same situation, "Schwinger's measurement algebra". In this version, the states are converted into operators, so that instead of the states and operators living in different things, they become objects that are both operators. One presumes that they operate on a Hilbert space, but you don't have to actually specify the Hilbert space, or choose the arbitrary complex phases that show up when you put a state in that form.
In the measurement algebra, two states with different eigenvalues are represented by two different measurements. Since the measurements are different, we can distinguish between the two states.
Think of a Stern-Gerlach apparatus, which separates two states based on their spin. Now think of the analogous machine that separates different energy particles. Since the two energies are different, we can distinguish between them in an experiment.
In the measurement algebra, one computes the probability of transition between two states by multiplying their operators together and taking the trace. Two different energies are different, so their probability of transition is zero. (Atoms don't just change energy.) Therefore their trace will be zero. Working out the mathematics, this is the same as their states being orthogonal. The orthogonality is retained when you write the states as spinors.
tr ( |e1><e1| |e2><e2| ) = 0 so
tr ( <e1|e2><e2|e1> ) = 0 so
|<e1|e2>|^2 = 0
so |e1> and |e2> are orthogonal.
One more little lesson. In standard QM, two states are orthogonal or are not, no matter which order they are considered. In the measurement algebra, one can entertain the possibility that order matters. This happens when one considers non Hermitian states, which are not possible in standard QM. (In standard QM, only operators are Hermitian or not.) Non Hermitian states are of interest because they violate the usual time symmetry. This has to do with CPT being a symmetry of physics, but not C, P, T, CP, CT, or PT. (Do a search on "non Hermitian operator" for more info.)
An example of a non Hermitian state is the nonzero product of two different Hermitian states. To have the product be nonzero, you have to deal with "incompatible measurements", for example, spin in two directions 90 degrees apart, rather than energy. Let "x", "y", and "z" stand for the Pauli spin matrices. Then (1+x)/2 is the measurement algebra state for spin 1/2 in the x direction. It is a projection operator in the x direction for spin, and is also the density operator state for spin-1/2 in the x direction.
Let A = (1+x)(1+y)/2, and B = (1-y)(1+x)/2, that is, twice the products of the states in the +x, +y, and -y and +x directions. Then A and B are non Hermitian density operator states. They satisfy AA = A, BB = B, and have trace 1, which means that they are "normalized" states. AB is zero, but BA is not. Instead, BA corresponds to a sequence of Stern-Gerlach filters arranged to reverse the polarization of an electron. This is similar to how you can take pieces of polarizing film and arrange them to rotate horizontally polarized light over to vertical polarization.
Just to complete CarlB's answer: excited states with the same energy don't have to be orthogonal, but we can always choose a basis for them which are orthogonal.