- #1
orajput
- 1
- 0
Given the formula of Mahalanobis Distance:
[itex]D^2_M = (\mathbf{x} - \mathbf{\mu})^T \mathbf{S}^{-1} (\mathbf{x} - \mathbf{\mu})[/itex]
If I simplify the above expression using Eigen-value decomposition (EVD) of the Covariance Matrix:
[itex]S = \mathbf{P} \Lambda \mathbf{P}^T[/itex]
Then,
[itex]D^2_M = (\mathbf{x} - \mathbf{\mu})^T \mathbf{P} \Lambda^{-1} \mathbf{P}^T (\mathbf{x} - \mathbf{\mu})[/itex]
Let, the projections of [itex](\mathbf{x}-\mu)[/itex] on all eigen-vectors present in [itex]\mathbf{P}[/itex] be [itex]\mathbf{b}[/itex], then:
[itex]\mathbf{b} = \mathbf{P}^T(\mathbf{x} - \mathbf{\mu})[/itex]
And,
[itex]D^2_M = \mathbf{b}^T \Lambda^{-1} \mathbf{b}[/itex]
[itex]D^2_M = \sum_i{\frac{b^2_i}{\lambda_i}}[/itex]
The problem that I am facing right now is as follows:
The covariance matrix [itex]\mathbf{S}[/itex] is calculated on a dataset, in which no. of observations are less than the no. of variables. This causes some zero-valued eigen-values after EVD of [itex]\mathbf{S}[/itex].
In these cases the above simplified expression does not result in the same Mahalanobis Distance as the original expression, i.e.:
[itex](\mathbf{x} - \mathbf{\mu})^T \mathbf{S}^{-1} (\mathbf{x} - \mathbf{\mu}) \neq \sum_i{\frac{b^2_i}{\lambda_i}}[/itex] (for non-zero [itex]\lambda_i[/itex])
My question is: Is the simplified expression still functionally represents the Mahalanobis Distance?
P.S.: Motivation to use the simplified expression of Mahalanbis Distance is to calculate its gradient wrt [itex]b[/itex].
[itex]D^2_M = (\mathbf{x} - \mathbf{\mu})^T \mathbf{S}^{-1} (\mathbf{x} - \mathbf{\mu})[/itex]
If I simplify the above expression using Eigen-value decomposition (EVD) of the Covariance Matrix:
[itex]S = \mathbf{P} \Lambda \mathbf{P}^T[/itex]
Then,
[itex]D^2_M = (\mathbf{x} - \mathbf{\mu})^T \mathbf{P} \Lambda^{-1} \mathbf{P}^T (\mathbf{x} - \mathbf{\mu})[/itex]
Let, the projections of [itex](\mathbf{x}-\mu)[/itex] on all eigen-vectors present in [itex]\mathbf{P}[/itex] be [itex]\mathbf{b}[/itex], then:
[itex]\mathbf{b} = \mathbf{P}^T(\mathbf{x} - \mathbf{\mu})[/itex]
And,
[itex]D^2_M = \mathbf{b}^T \Lambda^{-1} \mathbf{b}[/itex]
[itex]D^2_M = \sum_i{\frac{b^2_i}{\lambda_i}}[/itex]
The problem that I am facing right now is as follows:
The covariance matrix [itex]\mathbf{S}[/itex] is calculated on a dataset, in which no. of observations are less than the no. of variables. This causes some zero-valued eigen-values after EVD of [itex]\mathbf{S}[/itex].
In these cases the above simplified expression does not result in the same Mahalanobis Distance as the original expression, i.e.:
[itex](\mathbf{x} - \mathbf{\mu})^T \mathbf{S}^{-1} (\mathbf{x} - \mathbf{\mu}) \neq \sum_i{\frac{b^2_i}{\lambda_i}}[/itex] (for non-zero [itex]\lambda_i[/itex])
My question is: Is the simplified expression still functionally represents the Mahalanobis Distance?
P.S.: Motivation to use the simplified expression of Mahalanbis Distance is to calculate its gradient wrt [itex]b[/itex].