Intuitive explanations for Gaussian distribution function and mahalanobis distance

jjepsuomi

Hello

I was wondering If anyone could give intuitive explanations for the multivariate Gaussian distribution function and mahalanobis distance? My professor didn't explain these in probability class, they were only defined...

Where did the formula come from? Why is the Gaussian function the way it is? Is there a way to intuitively explain mahalanobis distance?

Thank you for any support

Stephen Tashi

Do you understand the 1-dimensional case? Do you know about the use of "Z-scores" in statistics?

I myself don't have a good understanding of why the 1-dimensional Gaussian distribution has its particular formula - to get that intuition, one would need to study why the gaussian shape is a limiting shape for binomial distributions You could get intuition in the way that a person who has inutition for algebra and calculus gets inutition. As to the man-in-the-street type of intuition, have you seen those demonstrations where balls are dropped into a pyramid of dividers and land in a bell shaped pattern?

jjepsuomi

Answers to both of your questions are: NO
I know what Z-score is "= How many standard deviations from the mean" but that's all I know about it.

And no I have not seen the pyramid divider, I'll try to look for it. Maybe you know where to look? Thank you! =)

Stephen Tashi

The video of a conceptual normal distribution machine http://www.youtube.com/watch?v=PM7z_03o_kk is clearer than the videos of the physical machines that I found.

Stephen Tashi

People measuring things that have normal distributions would work with a variety of units. They might be measuring things in dollars, Ohms, grams etc. It would be inconvenient to use different formulae for each different unit of measure. The use of the Z-score is a way to convert all such measurements to a standard reference whose units are dimensionless.

Think about what you must do to convert two histograms to the "same scale". If one is measuring dollars on the x-axis and the other is measuring Ohms, there is no law of economics or physics that establishes a definite relation between dollars and Ohms. So you can't convert the measurements to a common unit. Even if distribution were both measuring dollars on the x-axis, there is no law of that tells you that $1000 dollars and $5000 dollars are measurements "a long ways" apart. If you're talking about car prices, they might be. If you're talking about the national GNP, they aren't.

Next, think about how you would convert two 2-dimensional histograms to the same scale. Suppose one histogram has measurement of ordered pairs (weight of person, blood glucose level of person) and the other has measurements of (mileage on car, price of car).

broccoli7

The roots of the gaussian distribution lies in the method of least squares, used widely in the 17th and 18th century for navigation, astronomy. To explain the method of least squares, arises the Gaussian distribution.

ImaLooser

A Gaussian dist arises when you have the sum of a large number of independent factors, all of which are small. It is the limit as the number of factors become infinitely large, and the size of each factor becomes infinitely small. In actual practice many situations converge to this limit quite quickly.

The basic Gaussian is one dimensional, but it is possible to have any number of dimensions. You can think of a large number of tiny insects flying around at random in an empty space.

Mandelbroth

The probability density function for the multivariate case of the Gaussian distribution is basically the formula for the univariate Gaussian distribution expanded to multiple variables.

You can see that [itex]f_{\mathbf x}(x_1,\ldots,x_k)\, =
\frac{1}{(2\pi)^{k/2}|\boldsymbol\Sigma|^{1/2}}
\exp\left(-\frac{1}{2}({\mathbf x}-{\boldsymbol\mu})^T{\boldsymbol\Sigma}^{-1}({\mathbf x}-{\boldsymbol\mu})
\right)[/itex] becomes [itex]f(x) = \frac{1}{\sigma \sqrt{2\pi}}\exp\left(-\frac{1}{2} (\frac{x-\mu}{\sigma})^2\right)[/itex] if Ʃ is a real number (1 X 1 matrix).

Mahalanobis distance is Mahalanobis distance. Tautology is always intuitive.

ImaLooser

Imagine a swarm of insects. Choose one insect. How can you tell whether or not it is in the swarm? The first thing you do is find the distance of that insect from the center of the swarm. Then you consider the standard deviation of the swarm, which shows how big the swarm is. If the swarm is big then the insect does not need to be all that close to the center. That's the mahalanobis distance. So if the insect is more than three standard deviations from the center then the insect is very likely not in the swarm.