Maximum Likelihood Estimation: Exploring Solutions

[Chen]

Hi,

I'm taking a basic course in statistical methods, and we recently learned of maximum likelihood estimation. We defined the likelihood as a function of some parameter 'a', and found the estimator of 'a' by requiring a maximum likelihood with respect to it.

As an example, we took the results of some fictional experiment, in which the measurements 't' are distributed according to exp(-at), with a statistical error that is normally distributed with a known variance. We defined the likelihood function and maximized it, numerically, in order to find the estimated value of 'a'.

However, in this example we assumed that we know the variance of the statistical error. If this value of 'sigma' is not known, then we must maximize the likelihood function with respect to both 'a' and 'sigma'. And now comes my question - isn't it possible that we will get more than one solution for this extremum problem? Couldn't it be that two different sets of 'a' and 'sigma' will give rise to the same distribution that we saw in the experiment? If that happens, what are the "real" estimators? And besides, even if we deal only with one parameter 'a', couldn't the likelihood function have maxima for two or more values of it?

Sorry about the lengthy post...

Thanks,
Chen

 

[EnumaElish]

All points you have raised are good points; it all depends on the lkhd function. If the lkhd is everywhere concave, then you won't have these problems.

 

[tacman]

Hi Chen,

Great question! You are absolutely correct that in some cases, maximum likelihood estimation can result in multiple solutions. This is known as the issue of "identifiability." In your example, if we only have data on the measurements 't' and not on the variance of the statistical error, then there are multiple combinations of 'a' and 'sigma' that could result in the same likelihood function. In this case, we cannot say for certain which combination is the "real" estimator.

There are a few ways to address this issue. One approach is to use additional information or assumptions to narrow down the possible solutions. For example, if we have prior knowledge or beliefs about the values of 'a' and 'sigma', we can incorporate them into our analysis and potentially eliminate some of the solutions. Another approach is to use a different estimation method, such as Bayesian estimation, which can handle cases of non-identifiability more effectively.

Regarding your question about the possibility of multiple maxima for the likelihood function, this is certainly possible. However, the goal of maximum likelihood estimation is to find the global maximum, which represents the most likely value for the parameter of interest. In cases where there are multiple local maxima, it is important to carefully examine the data and the model to determine which maximum is the most appropriate to use as the estimator.

I hope this helps clarify your understanding of maximum likelihood estimation! It is a powerful tool in statistics, but as with any method, it is important to consider potential limitations and explore alternative solutions if necessary. Best of luck in your studies!