1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Finding maximum likelihood estimator

  1. Mar 22, 2014 #1
    1. The problem statement, all variables and given/known data

    The independent random variables [itex]X_1, ..., X_n[/itex] have the common probability density function [itex]f(x|\alpha, \beta)=\frac{\alpha}{\beta^{\alpha}}x^{\alpha-1}[/itex] for [itex]0\leq x\leq \beta[/itex]. Find the maximum likelihood estimators of [itex]\alpha[/itex] and [itex]\beta[/itex].

    2. Relevant equations

    log likelihood (LL) = n ln(α) - nα ln(β) + (α-1) ∑(ln xi)

    3. The attempt at a solution
    When I take the partial derivatives of log-likelihood (LL) with respect to α and β then set them equal to zero, I get:
    (1) d(LL)/dα = n/α -n ln(β) + ∑(ln xi) = 0 and
    (2) d(LL)/dβ = -nα/β = 0

    I am unable to solve for α and β from this point, because I get α=0 from equation (2), but this clearly does not work when you substitute α=0 into equation (1). Can someone please help me figure out what I should be doing?
     
    Last edited: Mar 22, 2014
  2. jcsd
  3. Mar 22, 2014 #2
    So there might be some mistakes in the way you computed the log (LL) function. The term premultiplying log(β) should probably be reworked. Hint: log(β^y) = ylog(β). But what is y? It is not αn.
     
  4. Mar 23, 2014 #3

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    Your expression for LL is correct, but condition (2) is wrong. Your problem is
    [tex] \max_{a,b} LL = n \ln(a) - n a \ln(b) + (a-1) \sum \ln(x_i) \\
    \text{subject to } b \geq m \equiv \max(x_1, x_2, \ldots ,x_n)[/tex]
    Here, I have written ##a,b## instead of ##\alpha, \beta##. The constraint on ##b## comes from your requirement ##0 \leq x_i \leq b \; \forall i##. When you have a bound constraint you cannot necesssarily set the derivative to zero; in fact, what replaces (2) is:
    [tex] \partial LL/ \partial b \leq 0, \text{ and either } \partial LL/ \partial b = 0 \text{ or } b = m [/tex]

    For more on this type of condition, see, eg.,
    http://en.wikipedia.org/wiki/Karush–Kuhn–Tucker_conditions


    In the notation of the above link, you want to maximize a function ##f = LL##, subject to no equalities, and an inequality of the form ##g \equiv m - b \leq≤ 0##. The conditions stated in the above link are that
    [tex] \partial LL/ \partial a = \mu \partial g / \partial a \equiv 0 \\
    \partial LL / \partial b = \mu \partial g \partial b \equiv - \mu [/tex]
    Here. ##\mu \geq 0## is a Lagrange multiplier associated with the inequality constraint, and the b-condition above reads as ##\partial LL / \partial b \leq 0##, as I already stated. Furthermore, the so-called "complementary slackness" condition is that either ##\mu = 0## or ##g = 0##, as already stated.

    Note that if ##a/b \geq 0## you have already satisfied the b-condition, and if ##a/b > 0## you cannot have ##\partial LL / \partial b = 0##, so you must have ##b = m##
     
  5. Mar 23, 2014 #4
    Ray, log((b^a)^N) = a^N*log(b) ≠ aNlog(b)?
     
  6. Mar 23, 2014 #5

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    We have ## (b^a)^2 = b^a \cdot b^a = b^{2a},## etc.
     
  7. Mar 23, 2014 #6
    Ah..ok, my bad. This problem is harder than I thought...KKT coming in a statistics problem. Thanks.
     
  8. Mar 23, 2014 #7

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    It's not that complicated in this case. For ##a,b > 0## the function ##LL(a,b)## is strictly decreasing in ##b##, so for any ##a > 0## its maximum over ##b \geq m \,(m > 0)## lies at ##b = m##. You don't even need calculus to conclude this.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted



Similar Discussions: Finding maximum likelihood estimator
Loading...