Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Determining the Likelihood function

  1. Nov 7, 2008 #1
    I was under the impression that the likelihood function was simply the probability density function but viewing the parameter theta as the variable instead of the observations x. Ie
    p(x|theta) = L(theta|x)

    However, the likelihood function is no longer a probability function

    See Example 1 here:

    Can anyone explain this a little bit?

  2. jcsd
  3. Nov 7, 2008 #2
    Right. The observations are fixed in the context of a likelihood function.

    Yeah, it doesn't typically integrate to 1 (and sometimes doesn't integrate at all). Note that in most pdf's, the data and parameters play very different roles, so we shouldn't expect the function to have the same integral over each of them. Also, in maximum likelihood estimation, the parameter is assumed to be a fixed, unknown constant (and NOT a random variable), so there is no distribution assigned to it.

    I could probably give a better answer if you could explain why you think the likelihood function should be a probability function.
  4. Nov 8, 2008 #3
    maybe I used a bad example

    I considered a Gaussian random variable with variance 1,

    so f(x|u=3) is a Gaussian distribution centered at 3.

    Then L(u|x=5) is a Gaussian distribution centered at 5.

    Is that correct? That must just be a "special" case and hence a bad example?

  5. Nov 10, 2008 #4
    Yeah, that's a special case, as the Gaussian mean and the data play interchangeable roles in the pdf. Try the same thing with a fixed mean and considering the variance to be the parameter, and the resulting likelihood will not be a probability function.

    The problem of estimating the mean from data drawn from a unit-variance Gaussian distribution is one of the most pervasive "special cases" in statistics. The MLE for such a situation is equivalent to the MVUE, for example.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook