Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Sampling from normalized and un-normalized posterior

  1. Feb 15, 2015 #1
    Help me understand something.

    I get that the posterior ##p(\theta|y) \propto p(y|\theta)p(\theta)## should be normalized by ##\frac{1}{p(y)}## for the probability to sum to 1, but what about the mean and variance?

    Am I not right understanding that sampling from the un-normalized posterior gives the same mean and variance as sampling from the normalized posterior?

    Can I prove it mathematically?

    Can't find it and can't figure it out.
  2. jcsd
  3. Feb 15, 2015 #2

    Stephen Tashi

    User Avatar
    Science Advisor

    How do you define "sampling" from a distribution that doesn't integrate to 1.0 ?
  4. Feb 15, 2015 #3
    I don't know.

    What exactly is the point with MCMC and not normalizing?

    Isn't that what the Bayesians are doing?
  5. Feb 15, 2015 #4

    Stephen Tashi

    User Avatar
    Science Advisor

    You should explain where in the Markov Chain Monte Carlo method that you think sampling is done from a non-normalized distribution. As far as I know, that technique is never defined or used.
  6. Feb 15, 2015 #5
    Ok, let me rephrase the question.

    if ##p(\theta|y)## is the distribution of interest, then what good is ##p(y|\theta)p(\theta)## if the mean and variance aren't the same?
  7. Feb 15, 2015 #6

    Stephen Tashi

    User Avatar
    Science Advisor

    You are speaking as if you've read that one distribution is of some "good" in answering questions about the other, but until you say exactly what the "good" is, it isn't possible to understand what you are asking.

    In general, if [itex] W [/itex] is a random variable and [itex] X = k\ W [/itex] for some constant [itex]k [/itex] then you can figure out the mean and variance of [itex] X [/itex] if you know the mean and variance of [itex] W [/itex]. If we treat [itex] y [/itex] a fixed event then [itex] k = \frac{1}{p(y)} [/itex].
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook