The change-of-basis identity:
<br />
\rho_{k' l'} = \sum_{m n}{\langle k' \vert m \rangle \, \rho_{m n} \, \langle n \vert l' \rangle}<br />
with the identification of the matrix:
<br />
U_{n l'} \equiv \langle n \vert l'\rangle<br />
can be rewritten as:
<br />
\mathbf{\rho}' = \mathbf{U}^{\dagger} \cdot \mathbf{\rho} \cdot \mathbf{U}<br />
where we used the fact that:
<br />
\hat{U}^{\dagger}_{k' m} = U^{\ast}_{m k'} = (\langle m \vert k' \rangle)^{\ast} = \langle k' \vert m \rangle<br />
The unitarity of the similarity transformation matrix U is an expression of the orthonormality of the old and new bases:
<br />
\langle m \vert n \rangle = \sum_{k'} {\langle m \vert k' \rangle \, \langle k' \vert n \rangle} = \sum_{k'} {U_{m k'} \, U^{\dagger}_{k' n}} = \left[\mathbf{U} \cdot \mathbf{U}^{\dagger}\right]_{m n} = \delta_{m n} \Rightarrow \mathbf{U} \cdot \mathbf{U}^{\dagger} = \mathbf{1}<br />
<br />
\langle k' \vert l' \rangle = \sum_{m} \langle k' \vert m \rangle \, \langle m \vert l' \rangle = \sum_{m} U^{\dagger}_{k' m} \, U_{m l'} = \left[ \mathbf{U}^{\dagger} \cdot \mathbf{U} \right]_{k' l'} = \delta_{k' l'} \Rightarrow \mathbf{U}^{\dagger} \cdot \mathbf{U} = \mathbf{1}<br />