For finite dimensional vector spaces, a "diagonal matrix" is something like
[tex]\begin{bmatrix}a & 0 & 0 \\ 0 & b & 0 \\ 0 & 0 & c\end{bmatrix}[/tex]
having non-zero entries only on the main diagona. But I suspect you already knew that!
More generally, a square matrix represents a linear operator on some vector space. If that that vector space has finite dimension, n, then we can represent the operator as a n by n matrix. If the vector space is infinite dimensional is, as is typically the case in Quantum theory, we can't really write it as a "matrix" but the same ideas work.
The matrix is "diagonal" in a particular basis, [itex]\{v_1, v_2, ..., v_n\}[/itex] then the basis vectors are eigenvectors: [itex]Av_i= a_iv_i[/itex] where [itex]a_i[/itex] is the number, on the diagonal, at the ith row and column. To say that an operator in Quantum Mechanics is "diagonal" also means that the basis vectors are all eigenvectors (eigenstates). Physically, "eigenstates" are those that give specific values to whatever the property is associated to the state while vectors that are not eigenstates can be written as linear combinations of the eigenstates and then give "mixtures" of those values.