Estimation of parameters using maximum likelihood method

In summary, the maximum likelihood method is a statistical approach for estimating probability distribution parameters by maximizing the likelihood of observed data. It involves calculating the likelihood function and assumes data follows a specific distribution with independent and continuous parameters. The method is widely used and provides unbiased estimates, but is sensitive to outliers and assumptions, and may have multiple local maxima.
  • #1
twoflower
368
0

Homework Statement


Let's have random value X defined by its density function:

[tex]
f(x; \beta) = \beta^2x \mbox{e}^{-\beta x}
[/tex]

where [itex]\beta > 0[/itex] for [itex]x > 0[/itex] and [itex]f(x) = 0[/itex] otherwise.

Expected value of X is [itex]EX = \frac{2}{\beta}[/itex] and variance is [itex]\mbox{var } X = \frac{2}{\beta^2}.[/itex]

Next, suppose we have random sample [itex]X_1, X_2, ..., X_n[/itex] from distribution defined by the given density function.

Infer estimate of the parameter [itex]\beta[/itex] using the method of maximum likelihood.

The Attempt at a Solution



[tex]
L(\beta; X) = \prod_{i=1}^{n} f(X_i; \beta)
[/tex]

[tex]
\mbox{lik}(\beta; X) = \log L(\beta; X) = \sum \log f(X_i; \beta) = ... = 2n\log \beta + \sum \left( \log X_i - \beta X_i\right)
[/tex]

Now we'll find derivative of [itex]\mbox{lik}(\beta; X)[/itex] and find point where it's equal to zero (to find maximum of likelihood function).

[tex]
\frac{\partial \mbox{lik} (\beta; X)}{\partial \beta} = \frac{2n}{\beta} - \sum X_i
[/tex]

[tex]
\frac{2n}{\widehat{\beta}} - \sum X_i = 0
[/tex]

[tex]
\widehat{\beta} = \frac{2n}{\sum X_i} = \frac{2}{\overline{X_{n}}}
[/tex]

So [itex]\widehat{\beta}[/itex] is estimate of [itex]\beta[/itex].

Next task is the following:

Infer asymptotical distribution of [itex]\widehat{\beta}[/itex]. Hint: Use Fisher information [itex]\mathcal{F}_n = -\mbox{E} \frac{

\partial^2 \mbox{lik}(\beta) }{\partial \beta^2}[/itex].

Using infered asymptotical distribution [itex]\widehat{\beta} \sim \mathbf{N}(\beta, \mathcal{F}_{n}^{-1})[/itex], try to designate a confidence

interval for parameter [itex]\beta[/itex] (possible unknown parameters replace with their estimates).


With this I'm having little troubles. I compute Fisher information:

[tex]
\mathcal{F}_n = -\mbox{E} \frac{ \partial^2 \mbox{lik}(\beta) }{\partial \beta^2} = -\mbox{E} \left( -\frac{2n}{\beta^2} \right)
[/tex]

And now what? Can I continue like this?

[tex]
\mathcal{F}_n = -\mbox{E} \left( -\frac{2n}{\beta^2} \right) = \frac{2n}{\beta^2}
[/tex]

or am I supposed to expand it and continue like this?

[tex]
\mathcal{F}_n = -\mbox{E} \left( -\frac{2n}{\beta^2} \right) = \mbox{E} \left{ \frac{2n}{\widehat{\beta}}^2} \right} = \mbox{E} \left{

\frac{2n\left( \overline{X_n}^2 \right)}{4} \right} = \mbox{E} \left{ \frac{1}{2} n \left( \overline{X_n}^2 \right) \right}
[/tex]

I don't know how to continue..
 
Physics news on Phys.org
  • #2




Thank you for your post and for sharing your attempt at a solution. It seems like you are on the right track in finding the maximum likelihood estimate for the parameter \beta. However, for the next task of inferring the asymptotic distribution of \widehat{\beta}, you can use the following steps:

1. Find the second derivative of the log likelihood function, \mbox{lik} (\beta; X), with respect to \beta.

2. Substitute the value of \widehat{\beta} that you found earlier into the second derivative.

3. Take the expectation of this expression to get the Fisher information, \mathcal{F}_n.

4. Once you have the Fisher information, you can use it to find the asymptotic distribution of \widehat{\beta} as \widehat{\beta} \sim \mathbf{N}(\beta, \mathcal{F}_{n}^{-1}).

5. To designate a confidence interval for the parameter \beta, you can use the standard error of \widehat{\beta}, which is the square root of the diagonal element of \mathcal{F}_{n}^{-1}. A 95% confidence interval can then be calculated as \widehat{\beta} \pm 1.96 \times \mbox{SE}(\widehat{\beta}).

I hope this helps. Keep up the good work in your research and analysis. Best of luck!


 

What is the maximum likelihood method?

The maximum likelihood method is a statistical approach used to estimate the parameters of a probability distribution by finding the values that maximize the likelihood of the observed data.

How does the maximum likelihood method work?

The maximum likelihood method involves calculating the likelihood function, which is a measure of how likely the observed data is to occur for a given set of parameter values. The parameter values that result in the highest likelihood are considered to be the best estimates.

What are the assumptions of the maximum likelihood method?

The maximum likelihood method assumes that the data follows a specific probability distribution and that the observations are independent and identically distributed. It also assumes that the parameters being estimated are continuous.

What are the advantages of using the maximum likelihood method?

The maximum likelihood method is a widely used and well-established approach to parameter estimation. It provides unbiased estimates and is relatively easy to compute. It also has good properties, such as consistency and efficiency, when the sample size is large.

What are the limitations of the maximum likelihood method?

The maximum likelihood method can be sensitive to outliers and is often not robust to violations of its assumptions. It also requires knowledge of the underlying probability distribution, which may not always be known. Additionally, in some cases, the likelihood function may have multiple local maxima, making it difficult to determine the global maximum.

Similar threads

  • Calculus and Beyond Homework Help
Replies
18
Views
1K
  • Calculus and Beyond Homework Help
Replies
1
Views
642
  • Programming and Computer Science
Replies
6
Views
2K
  • Calculus and Beyond Homework Help
Replies
6
Views
2K
  • Calculus and Beyond Homework Help
Replies
12
Views
1K
  • Calculus and Beyond Homework Help
Replies
2
Views
723
  • Calculus and Beyond Homework Help
Replies
4
Views
997
  • Calculus and Beyond Homework Help
Replies
3
Views
1K
  • Calculus and Beyond Homework Help
Replies
3
Views
358
  • Calculus and Beyond Homework Help
Replies
5
Views
194
Back
Top