1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Stuck on obtaining a closed form for parameter using MLE

  1. Mar 3, 2016 #1
    1. The problem statement, all variables and given/known data
    We have a Markov Random Field with the log likelihood as such:
    $$ l(\theta) = \sum\limits_{i=1}^L \log p(x^{(i)}|\theta) = \sum\limits_{i=1}^L \left( \sum\limits_{s \in V} \theta_{s} x_{s}^{(i)} - \log \sum\limits_{x} \exp \left\lbrace \sum\limits_{s \in V} \theta_{s} x_{s} \right\rbrace \right) $$

    Note L is the number of data examples.
    Also each x(i) is a vector where each component xs(i) is a binary variable taking on a value 0 or 1.
    The set V denotes the set of vertices/nodes in the Markov Random Field. The nodes are the components xs of the vector x. Each node or xs has a parameter denoted θs.
    And the sum over x, is the sum over every possible arrangement of the values in the vector x.

    I then take the derivative with respect to θs to determine the ML estimates.
    \frac{\partial l(\theta)}{\partial \theta_{s}} &= \sum\limits_{i=1}^L \left( x_{s}^{(i)} - \frac{\partial}{\partial \theta_{s}} \log \sum\limits_{x} \exp \left\lbrace \sum\limits_{s \in V} \theta_{s} x_{s} \right\rbrace \right)
    \\ &= \sum\limits_{i=1}^L x_{s}^{i} - \dfrac{L \sum\limits_{x} \exp \left( \sum\limits_{s \in V} \theta_{s} x_{s} \right)x_{s}}{\sum\limits_{x} \exp \left( \sum\limits_{s \in V} \theta_{s} x_{s} \right)}

    At this point I'm not really sure how to approach the problem. I'm having difficulty directly obtaining a closed form for θs. It has been suggested to consider cases when xs = 0 and 1 separately. But I don't understand what that entails.
  2. jcsd
  3. Mar 9, 2016 #2
    Thanks for the post! This is an automated courtesy bump. Sorry you aren't generating responses at the moment. Do you have any further information, come to any new conclusions or is it possible to reword the post?
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted