Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Mahalanobis Distance using Eigen-Values of the Covariance Matrix

  1. Jul 23, 2012 #1
    Given the formula of Mahalanobis Distance:

    [itex]D^2_M = (\mathbf{x} - \mathbf{\mu})^T \mathbf{S}^{-1} (\mathbf{x} - \mathbf{\mu})[/itex]

    If I simplify the above expression using Eigen-value decomposition (EVD) of the Covariance Matrix:

    [itex]S = \mathbf{P} \Lambda \mathbf{P}^T[/itex]


    [itex]D^2_M = (\mathbf{x} - \mathbf{\mu})^T \mathbf{P} \Lambda^{-1} \mathbf{P}^T (\mathbf{x} - \mathbf{\mu})[/itex]

    Let, the projections of [itex](\mathbf{x}-\mu)[/itex] on all eigen-vectors present in [itex]\mathbf{P}[/itex] be [itex]\mathbf{b}[/itex], then:

    [itex]\mathbf{b} = \mathbf{P}^T(\mathbf{x} - \mathbf{\mu})[/itex]


    [itex]D^2_M = \mathbf{b}^T \Lambda^{-1} \mathbf{b}[/itex]

    [itex]D^2_M = \sum_i{\frac{b^2_i}{\lambda_i}}[/itex]

    The problem that I am facing right now is as follows:

    The covariance matrix [itex]\mathbf{S}[/itex] is calculated on a dataset, in which no. of observations are less than the no. of variables. This causes some zero-valued eigen-values after EVD of [itex]\mathbf{S}[/itex].

    In these cases the above simplified expression does not result in the same Mahalanobis Distance as the original expression, i.e.:

    [itex](\mathbf{x} - \mathbf{\mu})^T \mathbf{S}^{-1} (\mathbf{x} - \mathbf{\mu}) \neq \sum_i{\frac{b^2_i}{\lambda_i}}[/itex] (for non-zero [itex]\lambda_i[/itex])

    My question is: Is the simplified expression still functionally represents the Mahalanobis Distance?

    P.S.: Motivation to use the simplified expression of Mahalanbis Distance is to calculate its gradient wrt [itex]b[/itex].
  2. jcsd
  3. Feb 4, 2013 #2

    In order to be invertible, S mustn't have zero eigen values, that is , must be positive definite or negative definite. Apart from that , that expression must work...

    All the best

  4. Feb 4, 2013 #3


    User Avatar
    Science Advisor

    Hey orajput and welcome to the forums.

    For your problem, if you do have a singular or ill-conditioned covariance matrix, I would try and do something like Principal Components, or to remove the offending variable from your system and re-do the analysis.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook