Why is likelihood function defined as such?

[CantorSet]

Hi everyone,

This is not a homework question but something I thought of while reading.

In the method of maximum likelihood estimation, they're trying to maximize the likelihood function
$f(\vec{x}| \theta )$ with respect to $\theta$. But shouldn't the likelihood function be defined as $f(\theta| \vec{x} )$ since we are GIVEN the data vector $\vec{x}$ while $\theta$ is the unknown parameter?

 

[Stephen Tashi]

But shouldn't the likelihood function be defined as $f(\theta| \vec{x} )$ since we are GIVEN the data vector $\vec{x}$ while $\theta$ is the unknown parameter?

Philosophically, I think you're right. But in frequentist statistics ( the usual kind taught in introductory statistics courses) no probability distribution is ever assumed for the parameters of a distribution. When we don't know these parameters, we say they have "definite but unknown values". If you don't have a "prior" probability distribution for the parameters, you can't compute the "posterior" distribution $f(\theta|\vec{x})$.

My personal view of frequentist statistics is this: The commonsense person's outlook is "I have certain ideas. Given the data, what is the probability that my ideas are true?" Frequentist statistics answers "I assumed certain ideas and calculated the probability of the data. Based on that probability, I will stipluate some decisions." This evades the question! Things are done backwards in comparison to what people naturally want to know.

The authoritative terminology used ("statistical significance" "rejection" of a hypothesis, "confidence" intervals) makes many laymen think that they are getting information about the probability that some idea is true given the data. But if you look under the hood at what's happening, what your are getting is a quantification of the probability of the data based on the assumption that certain ideas are true.

I'm not trying to say that frequentist statistics isn't effective. But how to apply it requires empirical trial and error. People observe that certain methods work in certain types of situations.

If you want to compute $f(\theta|\vec{x})$ you must become a Bayesian. That's my favorite kind of statistics.

 

[SW VandeCarr]

Hi everyone,

This is not a homework question but something I thought of while reading.

In the method of maximum likelihood estimation, they're trying to maximize the likelihood function
$f(\vec{x}| \theta )$ with respect to $\theta$. But shouldn't the likelihood function be defined as $f(\theta| \vec{x} )$ since we are GIVEN the data vector $\vec{x}$ while $\theta$ is the unknown parameter?

Generally that would be written: $L(\vec{x}|\theta) = f(\theta|\vec{x})$.

The expression $f(\vec{x}| \theta)$ is just a conditional probability when f = P.

 

[CantorSet]

Thanks for the responses, guys.