Determining the Likelihood function

[daviddoria]

I was under the impression that the likelihood function was simply the probability density function but viewing the parameter theta as the variable instead of the observations x. Ie
p(x|theta) = L(theta|x)

However, the likelihood function is no longer a probability function

See Example 1 here:
http://en.wikipedia.org/wiki/Likelihood_function

Can anyone explain this a little bit?

Thanks,
Dave

 

[quadraphonics]

I was under the impression that the likelihood function was simply the probability density function but viewing the parameter theta as the variable instead of the observations x.

Right. The observations are fixed in the context of a likelihood function.

However, the likelihood function is no longer a probability function

Yeah, it doesn't typically integrate to 1 (and sometimes doesn't integrate at all). Note that in most pdf's, the data and parameters play very different roles, so we shouldn't expect the function to have the same integral over each of them. Also, in maximum likelihood estimation, the parameter is assumed to be a fixed, unknown constant (and NOT a random variable), so there is no distribution assigned to it.

I could probably give a better answer if you could explain why you think the likelihood function should be a probability function.

 

[daviddoria]

maybe I used a bad example

I considered a Gaussian random variable with variance 1,

so f(x|u=3) is a Gaussian distribution centered at 3.

Then L(u|x=5) is a Gaussian distribution centered at 5.

Is that correct? That must just be a "special" case and hence a bad example?

Dave

 

[quadraphonics]

maybe I used a bad example

I considered a Gaussian random variable with variance 1,

so f(x|u=3) is a Gaussian distribution centered at 3.

Then L(u|x=5) is a Gaussian distribution centered at 5.

Is that correct? That must just be a "special" case and hence a bad example?

Yeah, that's a special case, as the Gaussian mean and the data play interchangeable roles in the pdf. Try the same thing with a fixed mean and considering the variance to be the parameter, and the resulting likelihood will not be a probability function.

The problem of estimating the mean from data drawn from a unit-variance Gaussian distribution is one of the most pervasive "special cases" in statistics. The MLE for such a situation is equivalent to the MVUE, for example.