What is Covariance matrix: Definition and 46 Discussions

In probability theory and statistics, a covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of a given random vector. Any covariance matrix is symmetric and positive semi-definite and its main diagonal contains variances (i.e., the covariance of each element with itself).
Intuitively, the covariance matrix generalizes the notion of variance to multiple dimensions. As an example, the variation in a collection of random points in two-dimensional space cannot be characterized fully by a single number, nor would the variances in the


{\displaystyle x}


{\displaystyle y}
directions contain all of the necessary information; a


{\displaystyle 2\times 2}
matrix would be necessary to fully characterize the two-dimensional variation.
The covariance matrix of a random vector


{\displaystyle \mathbf {X} }
is typically denoted by




{\displaystyle \operatorname {K} _{\mathbf {X} \mathbf {X} }}


{\displaystyle \Sigma }

View More On Wikipedia.org
  1. BWV

    Negative eigenvalues in covariance matrix

    Trying to run the factoran function in MATLAB on a large matrix of daily stock returns. The function requires the data to have a positive definite covariance matrix, but this data has many very small negative eigenvalues (< 10^-17), which I understand to be a floating point issue as 'real'...
  2. F

    MATLAB Can I calculate the covariance matrix of a large set of data?

    Hello everyone. I want to calculate the covariance matrix of a stochastic process using Matlab as cov(listOfUVValues) being the dimensions of listOfUVValues 211302*50. I get the following error: Requested 211302x211302 (332.7GB) array exceeds maximum array size preference. Creation of...
  3. M

    I Weird covariance matrix for linear fit

    Hello! I really don't know much about statistics, so I am sorry if this questions is stupid or obvious. I have this data: ##x = [0,1,2,3]##, y = ##[25.885,26.139,27.404,30.230]##, ##y_{err}=[1.851,0.979,2.049,6.729]##. I need to fit to this data the following function: $$y = a (x+0.5)/4.186 +...
  4. C

    I Correlations vs. negligence of correlations in a covariance matrix

    Suppose I have a model composed of two parameters ##(a,b)## that I want to describe a set of data points with. In CASE A, I fit the model taking into consideration the correlations between the data points (that is, in the chi square formulation I use the full covariance matrix for the data) and...
  5. F

    Python Invert a matrix from a 4D array : equivalence or difference with indexes

    I have a 4D array of dimension ##100\text{x}100\text{x}3\text{x}3##. I am working with `Python Numpy. This 4D array is used since I want to manipulate 2D array of dimensions ##100\text{x}100## for the following equation (it allows to compute the ##(i,j)## element ##F_{ij}## of Fisher matrix) ...
  6. F

    A Covariance matrix size: 3x3 or 4x4?

    Hello, I follow the post https://www.physicsforums.com/threads/cross-correlations-what-size-to-select-for-the-matrix.967222/#post-6141227 . It talks about the constraints on cosmological parameters and forecast on futur Dark energy surveys with Fisher's matrix formalism. Below a capture of...
  7. F

    I Cross-correlations: what size to select for the matrix?

    Hello, I am working on Fisher's formalism in order to get constraints on cosmological parameters. I am trying to do cross-correlation between 2 types of galaxy populations (LRG/ELG) into a total set of 3 types of population (BGS,LRG,ELG). From the following article...
  8. F

    I Fisher matrix - equivalence or not between sequences

    I am currently studying Fisher's formalism as part of parameter estimation. From this documentation : They that Fisher matrix is the inverse matrix of the covariance matrix. Initially, one builds a matrix "full" that takes into account all the parameters. 1) Projection : We can then do...
  9. Jonas Hall

    I Conf.intervals for fitted parameters: divide by sqrt(n)?

    If you fit a parametrized model (i.e. y = a log(x + b) + c) to some data points the output is typically the optimized parameters (i.e. a, b, c) and the covariance matrix. The squares of the diagonal elements of this matrix are the standard errors of the optimized parameters. (i.e. sea, seb...
  10. Ken Gallock

    I Covariance Matrices and Standard form

    Hi. I have a question about covariance matrices (CMs) and a standard form. In Ref. [Inseparability Criterion for Continuous Variable Systems], it is mentioned that CMs ##M## for two-mode Gaussian states can be symplectic transformed to the standard form ##M_s##: ## M= \left[ \begin{array}{cc}...
  11. S

    I Uncertainty of coefficients after a least square fit

    Fitting data to a linear function (y=a0+a1*x) with least square gives the coefficients a0 and a1. I am having trouble with calculating the uncertainty of a0. I understand that the diagonal elements of the covariance matrix C is the square of the uncertainty of each coefficient if there are no...
  12. mfb

    A Covariance matrix for transformed variables

    This sounds like a common application, but I didn't find a discussion of it. Simple case: I have 30 experimental values, and I have the full covariance matrix for the measurements (they are correlated). I am now interested in the sum of the first 5 measured values, the sum of the following 5...
  13. S

    I Covariance in fitting function

    Hello! I have to calculate the covariance between 2 parameters from a fit function. I found this package in Python called iminuit that did a good fit and also calculate the covariance matrix of the parameters. I tested the package on a simple function and I am not sure I understand the result...
  14. Daaavde

    Covariance matrix with asymmetric uncertainties

    Hello everyone, I'm currently building the covariance matrix of a large dataset in order to calculate the Chi-Squared. The covariance matrix has this form: \begin{bmatrix} \sigma^2_{1, stat} + \sigma^2_{1, syst} & \rho_{12} \sigma_{1,syst} \sigma_{2, syst} & ... \\ \rho_{12} \sigma_{1,syst}...
  15. R

    Kalman filter a posteriori estimate covariance matrix

    I am having some trouble deriving the a posteriori estimate covariance matrix for the linear Kalman filter. Below I have shown my workings for two methods. Method one is fine and gives the expected result. Method two is the way I tried to derive it initially before further expanding out terms to...
  16. W

    What Are the Constraints of a Valid Covariance Matrix?

    I'm trying to understand what makes a valid covariance matrix valid. Wikipedia tells me all covariance matrices are positive semidefinite (and, in fact, they're positive definite unless one signal is an exact linear combination of others). I don't have a very good idea of what this means in...
  17. A

    MHB Proof that covariance matrix is positive semidefinite

    Hello, i am having a hard time understanding the proof that a covariance matrix is "positive semidefinite" ... i found a numbe of different proofs on the web, but they are all far too complicated / and/ or not enogh detailed for me. Such as in the last anser of the link : probability -...
  18. P

    Covariance matrix does not always exist?

    Hey guys. I am going through the PRM (risk manager) material and there is a sample question that is bugging me. The PRM forum is relatively dead, and they don't usually go that deep into the theory anyway. So wanted to ask you guys. Shouldn't a random vector always have a covariance matrix? Why...
  19. T

    Calculating a covariance matrix with missing data

    Consider a co-variance matrix A such that each element ai,j = E(Xi Xj) - E(Xi) E(Xj) where Xi,Xj are random variables. Consider the case that each variable X has a different sample size. Let's say that Xi contains the elements xi,1, …, xi,N, and Xj contains the elements xj,1, ..., xj,n where...
  20. D

    Covariance Matrix

    if X= (3, 5, 7) & Y = (2, 4, 1) What is the 3x3 covariance matrix for X & Y?
  21. dexterdev

    MATLAB How to find covariance matrix of 3 or more vectors in matlab?

    Hi all, I know how to find covariance of 2 vectors and variance too. If covariance matrix is to be found of 3 vectors x,y and z, then then the cov matrix is given by cov_matrix(x,y,z) =[var(x) cov(x,y) cov(x,z); cov(x,y) var(y) cov(y,z); cov(x,z) cov(y,z) var(z) ]; Is this...
  22. dexterdev

    Physical significance of eigen vectors of Covariance matrix

    Hi all, I have a doubt regarding the physical significance of eigen vectors of the covariance matrix. I came to know that eigen vectors of covariance matrix are the principal components for dimensionality reduction etc, but how to prove it?
  23. C

    Covariance Matrix of a Vector Random Variable w/ Components Related

    Homework Statement Given X=ZU+Y where (i) U,X,Y, and Z are random variables (ii) U~N(0,1) (iii) U is independent of Z and Y (iv) f(z) = \frac{3}{4} z2 if 1 \leq z \leq 2 , f(z)=0 otherwise (v) fY|Z=z(y) = ze-zy (i.e. Y depends conditionally on...
  24. G

    Block of a covariance matrix

    Hello everybody, I’d like to present this math problem that I’ve trying to solve… This matter is important because the covariance matrix is widely use and this leads to new interpretations of the cross covariance matrices. Considering the following covariance block matrix ...
  25. O

    MHB Significance of the Eigenvalues of a covariance matrix

    Hello everyone! I'm curious to know what is the significance of the Eigenvalues of a covariance matrix. I'm not interested to find an answer in terms of PCA (as you of you may be familiar with the term). I'm thinking of a Gaussian vector, whose variance represent some notion of power or...
  26. T

    Dimensions of Covariance matrix (multiple observations)

    Suppose we have a mxn matrix, where each row is an observation and each column is a variable. The (i,j)-element of its covariance matrix is \mathrm{E}\begin{bmatrix}(\vec{X_i} - \vec{\mu_i})^t*(\vec{X_j} - \vec{\mu_j})\end{bmatrix}, where \vec{X_i} is the column vector corresponding to a...
  27. O

    Mahalanobis Distance using Eigen-Values of the Covariance Matrix

    Given the formula of Mahalanobis Distance: D^2_M = (\mathbf{x} - \mathbf{\mu})^T \mathbf{S}^{-1} (\mathbf{x} - \mathbf{\mu}) If I simplify the above expression using Eigen-value decomposition (EVD) of the Covariance Matrix: S = \mathbf{P} \Lambda \mathbf{P}^T Then, D^2_M =...
  28. S

    Question about sample covariance matrix

    Suppose vectors X1, X2, ... , Xn whose components are random variables are mutually independent(I mean Xi's are vectors of components with constants which are possible values of random variables labeled by the component indice, and all these labeled random variables are organized as a vector X...
  29. S

    What to do with variances in a covariance matrix?

    How to get a covariance matrix is well defined, but I don't really know how to use it once obtained. I'm trying to find the best parameters for a data set with a given function. I'm having four parameters a1,a2,a3,a4 and from these parameters I have the covariance matrix. I'm supposed to get...
  30. B

    Calculate covariance matrix of two given numbers of events

    Hi, I am trying to follow this paper: (arXiv link). On page 18, Appendix A.1, the authors calculate a covariance matrix for two variables in a way I cannot understand. Homework Statement Variables N_1[\itex] and [itex]N_2[\itex], distributed on [itex]y \in [0, 1][\itex] as follows...
  31. M

    Covariance matrix in barycentric coordinates

    Hi folks, I know the covariance matrix and the location of a point, both of which are expressed in Cartesian coordinates. I am going to represent the point in barycentric coordinates, and I would like to represent the covariance matrix for the point in barycentric coordinates as well. Does...
  32. F

    Calculating covariance matrix

    So I need to calculate the square root of the covariance matrix \sqrt{\Sigma_tR\Sigma_t} (the matrix square root, not the element-wise square root). \Sigma_t is a diagonal matrix with the square root of the variance on the diagonal (these values are time dependent) and R is the correlation...
  33. W

    Covariance matrix of 2 matrices?

    I have an (m \times n) complex matrix, \textbf{N}, whose elements are zero-mean random variables. I have a sort of covariance expression: \mathcal{E}\left\{\textbf{N}\textbf{N}^H\right\} = \textbf{I} where \mathcal{E}\left\{\right\} denotes expectation, \{\}^H is conjugate transpose and...
  34. W

    Simple Covariance Matrix Question

    I have a time-varying random vector, \underline{m}(t), whose elements are unity power and uncorrelated. So, its covariance matrix is equal to the identity matrix. Now, if I separate \underline{m}(t) into two separate components (a vector and a scalar)...
  35. M

    Prediction covariance matrix with Kalman filter

    Hello all. I have set up a model using the Kalman filter to estimate automobile prices. I'm having difficulty in figuring out how to formulate a prediction covariance matrix based on the model, i.e. given a set y_{new} = y_1, \ldots, y_N of N cars, finding the covariance matrix based on the...
  36. R

    Mean and covariance matrix?

    My professor sucks she hasnt gone over mean vector and she expects up to solve this let z1, z2, z3 be the random variables with mean vector and covariance matrix given below mean vector = [1 2 3]T. T = transpose covariance vector 3 2 1 2 2 1 1 1 1 Define the new variables...
  37. L

    Specifying a covariance matrix

    I have a Gaussian distribution. I know the variance in the directions of the first and second eigenvectors (the directions of maximum and minimum radius of the corresponding ellipse at any fixed mahalnobis distance), and the direction of the first eigenvector. Is there a simple closed form...
  38. P

    Mean centering of the covariance matrix in PCA

    Hi all, I thought I posted this last night but have received no notification of it being moved or can't find it the thread I have started list. I was wondering if you could help me understand how PCA, principal component analysis, works a little better. I have read often that it to get the...
  39. F

    Why Covariance Matrix of Complex Random Vector is Hermitian Positive Definite

    I've been reading everywhere, including wikipedia, and I can't seem to find a prove to the fact that the covariance matrix of a complex random vector is Hermitian positive definitive. Why is it definitive and not just simple semi-definitive like any other covariance matrix? Wikipedia just...
  40. C

    Given Cont.Joint PDF function find Covariance MATRIX

    Hello Buddies, I need to calculate "covariance matrix" of the given joint PDF function. Joint PDF is fx(x1,x2,x3)=2/3(x1+x2+x3) over S (x1,x2,x3), 0<xi<1, i=1,2,3 How can I calculte the Covariance Matrix? Thanks
  41. N

    Find Covariance Matrix for c1,c2 Given x,y=7,4

    if you've got c1=2x+3x, and c2=x-y, with cov matrix x,y = [7 4].how do you find C = (c1,c2)transpose?
  42. F

    Rank of sample covariance matrix

    I was reading Turk and Pentland paper 'Eigenfaces for recognition' and they assert that, if M < N, the maximum rank of a covariance matrix is M - 1, being M the number of samples and NxN the size of the covariance matrix. Is there any simple demonstration of this fact? Thanks in advance...
  43. S

    Determining the covariance matrix of a multivariate normal distribution

    Hi all, I have a stats problem I'm trying to figure out. Suppose I have a very large population (~millions) of colored balls with exactly 50% red, 30% green, 20% blue. If I take a random sample of 1000 of these balls, the distribution of colors I end up with can be modeled as a multivariate...
  44. Y

    Rotation of Elipse axes from covariance matrix

    Im writing some java code and need help with some matrix math... :confused: Basically I am trying to figure out how to rotate an ellipse given the std deviations, means, and covariance matrix such that the major and minor axes are along the direction that has the greatest variance. This is just...
  45. L

    Negative values in covariance matrix

    Hello I had measured luminescence decay profile. Then I want to fit a function which would approximate my experimental date. For that I make a simple program in LabWiev. The problem is that, that program give me out a negative values in covariance matrix. Why that? P.S. Sorry for my English