No. K-by-K is correct. Moreover it is symmetric. Each term in the matrix is the estimated covariance between two of the variables.http://en.wikipedia.org/wiki/Covariance
I read from the above link that-
The sample covariance of N observations of K variables is the K-by-K matrix.
I am wondering-
Should "K-by-K" be "N-by-K" or not?