Register to reply

Mahalanobis Distance using Eigen-Values of the Covariance Matrix

Share this thread:
orajput
#1
Jul23-12, 04:14 AM
P: 1
Given the formula of Mahalanobis Distance:

[itex]D^2_M = (\mathbf{x} - \mathbf{\mu})^T \mathbf{S}^{-1} (\mathbf{x} - \mathbf{\mu})[/itex]

If I simplify the above expression using Eigen-value decomposition (EVD) of the Covariance Matrix:

[itex]S = \mathbf{P} \Lambda \mathbf{P}^T[/itex]

Then,

[itex]D^2_M = (\mathbf{x} - \mathbf{\mu})^T \mathbf{P} \Lambda^{-1} \mathbf{P}^T (\mathbf{x} - \mathbf{\mu})[/itex]

Let, the projections of [itex](\mathbf{x}-\mu)[/itex] on all eigen-vectors present in [itex]\mathbf{P}[/itex] be [itex]\mathbf{b}[/itex], then:

[itex]\mathbf{b} = \mathbf{P}^T(\mathbf{x} - \mathbf{\mu})[/itex]

And,

[itex]D^2_M = \mathbf{b}^T \Lambda^{-1} \mathbf{b}[/itex]

[itex]D^2_M = \sum_i{\frac{b^2_i}{\lambda_i}}[/itex]

The problem that I am facing right now is as follows:

The covariance matrix [itex]\mathbf{S}[/itex] is calculated on a dataset, in which no. of observations are less than the no. of variables. This causes some zero-valued eigen-values after EVD of [itex]\mathbf{S}[/itex].

In these cases the above simplified expression does not result in the same Mahalanobis Distance as the original expression, i.e.:

[itex](\mathbf{x} - \mathbf{\mu})^T \mathbf{S}^{-1} (\mathbf{x} - \mathbf{\mu}) \neq \sum_i{\frac{b^2_i}{\lambda_i}}[/itex] (for non-zero [itex]\lambda_i[/itex])

My question is: Is the simplified expression still functionally represents the Mahalanobis Distance?

P.S.: Motivation to use the simplified expression of Mahalanbis Distance is to calculate its gradient wrt [itex]b[/itex].
Phys.Org News Partner Science news on Phys.org
Experts defend operational earthquake forecasting, counter critiques
EU urged to convert TV frequencies to mobile broadband
Sierra Nevada freshwater runoff could drop 26 percent by 2100
GoodSpirit
#2
Feb4-13, 05:48 AM
P: 19
Hello,

In order to be invertible, S mustn't have zero eigen values, that is , must be positive definite or negative definite. Apart from that , that expression must work...

All the best

GoodSpirit
chiro
#3
Feb4-13, 08:31 PM
P: 4,573
Hey orajput and welcome to the forums.

For your problem, if you do have a singular or ill-conditioned covariance matrix, I would try and do something like Principal Components, or to remove the offending variable from your system and re-do the analysis.


Register to reply

Related Discussions
Regarding Eigen values Of a Matrix Calculus & Beyond Homework 5
Eigen Values of Positive Definite Matrix Linear & Abstract Algebra 8
Eigen Values of a 2x2 Matrix Calculus & Beyond Homework 4
Finding eigen values of a 2x2 matrix Calculus & Beyond Homework 2
Negative values in covariance matrix Calculus 2