- #1

Jatex

- 1

- 0

Theorem 1.31 (Bayes' theorem):

Suppose that ##X## has a parametric family ##\mathcal{P}_0## of distributions with parameter space ##\Omega##.

Suppose that ##P_\theta \ll \nu## for all ##\theta \in \Omega##, and let ##f_{X\mid\Theta}(x\mid\theta)## be the conditional density (with respect to ##\nu##) of ##X## given ##\Theta = \theta##.

Let ##\mu_\Theta## be the prior distribution of ##\Theta##.

Let ##\mu_{\Theta\mid X}(\cdot \mid x)## denote the conditional distribution of ##\Theta## given ##X = x##.

Then ##\mu_{\Theta\mid X} \ll \mu_\Theta##, a.s. with respect to the marginal of ##X##, and the Radon-Nikodym derivative is

$$

\frac{d\mu_{\Theta\mid X}}{d\mu_\Theta}(\theta \mid x)

= \frac{f_{X\mid \Theta}(x\mid \theta)}{\int_\Omega f_{X\mid\Theta}(x\mid t) \, d\mu_\Theta(t)}

$$

for those ##x## such that the denominator is neither ##0## nor infinite.

The prior predictive probability of the set of ##x## values such that the denominator is ##0## or infinite is ##0##, hence the posterior can be defined arbitrarily for such ##x## values.

I tried to derive the right hand side of the Radon-Nikodym derivative above but I got different result, here is my attempt:

\begin{equation} \label{eq1}

\begin{split}

\frac{\mathrm d\mu_{\Theta\mid X}}{\mathrm d\mu_\Theta}(\theta \mid x) &= f_{\Theta\mid X}(\theta\mid x) \mathrm \space \space[1]\\

&=\frac{f_{X\mid \Theta}(x\mid \theta) \cdot f_{\Theta}(\theta)}{f_X(x)}\\

&=\frac{f_{X\mid \Theta}(x\mid \theta) \cdot f_{\Theta}(\theta)}{\int_\Omega f_{X\mid\Theta}(x\mid t) \, \cdot f_{\Theta}(t) \space \mathrm dt}\\

&=\frac{f_{X\mid \Theta}(x\mid \theta) \cdot f_{\Theta}(\theta)}{\int_\Omega f_{X\mid\Theta}(x\mid t) \, \mathrm d\mu_\Theta(t)}

\end{split}

\end{equation}

but now, where does ##f_{\Theta}(\theta)## go?

for ##[1]## see slide ##10## of the following document: http://mlg.eng.cam.ac.uk/mlss09/mlss_slides/Orbanz_1.pdf

Thanks in advance.

Last edited by a moderator: