Mahalanobis Distance using EigenValues of the Covariance Matrixby orajput Tags: eigenvalues, eigenvectors, linearalgebra, mahalanobis 

#1
Jul2312, 04:14 AM

P: 1

Given the formula of Mahalanobis Distance:
[itex]D^2_M = (\mathbf{x}  \mathbf{\mu})^T \mathbf{S}^{1} (\mathbf{x}  \mathbf{\mu})[/itex] If I simplify the above expression using Eigenvalue decomposition (EVD) of the Covariance Matrix: [itex]S = \mathbf{P} \Lambda \mathbf{P}^T[/itex] Then, [itex]D^2_M = (\mathbf{x}  \mathbf{\mu})^T \mathbf{P} \Lambda^{1} \mathbf{P}^T (\mathbf{x}  \mathbf{\mu})[/itex] Let, the projections of [itex](\mathbf{x}\mu)[/itex] on all eigenvectors present in [itex]\mathbf{P}[/itex] be [itex]\mathbf{b}[/itex], then: [itex]\mathbf{b} = \mathbf{P}^T(\mathbf{x}  \mathbf{\mu})[/itex] And, [itex]D^2_M = \mathbf{b}^T \Lambda^{1} \mathbf{b}[/itex] [itex]D^2_M = \sum_i{\frac{b^2_i}{\lambda_i}}[/itex] The problem that I am facing right now is as follows: The covariance matrix [itex]\mathbf{S}[/itex] is calculated on a dataset, in which no. of observations are less than the no. of variables. This causes some zerovalued eigenvalues after EVD of [itex]\mathbf{S}[/itex]. In these cases the above simplified expression does not result in the same Mahalanobis Distance as the original expression, i.e.: [itex](\mathbf{x}  \mathbf{\mu})^T \mathbf{S}^{1} (\mathbf{x}  \mathbf{\mu}) \neq \sum_i{\frac{b^2_i}{\lambda_i}}[/itex] (for nonzero [itex]\lambda_i[/itex]) My question is: Is the simplified expression still functionally represents the Mahalanobis Distance? P.S.: Motivation to use the simplified expression of Mahalanbis Distance is to calculate its gradient wrt [itex]b[/itex]. 



#2
Feb413, 05:48 AM

P: 19

Hello,
In order to be invertible, S mustn't have zero eigen values, that is , must be positive definite or negative definite. Apart from that , that expression must work... All the best GoodSpirit 



#3
Feb413, 08:31 PM

P: 4,570

Hey orajput and welcome to the forums.
For your problem, if you do have a singular or illconditioned covariance matrix, I would try and do something like Principal Components, or to remove the offending variable from your system and redo the analysis. 


Register to reply 
Related Discussions  
Regarding Eigen values Of a Matrix  Calculus & Beyond Homework  5  
Eigen Values of Positive Definite Matrix  Linear & Abstract Algebra  8  
Eigen Values of a 2x2 Matrix  Calculus & Beyond Homework  4  
Finding eigen values of a 2x2 matrix  Calculus & Beyond Homework  2  
Negative values in covariance matrix  Calculus  2 