Radon-Nikodym Derivative and Bayes' Theorem

Click For Summary
SUMMARY

The discussion centers on the Radon-Nikodym derivative in the context of Bayes' theorem, specifically Theorem 1.31. It establishes that the Radon-Nikodym derivative can be expressed as the ratio of the conditional density of X given Θ and the integral of that density over the parameter space. A participant attempts to derive this expression but encounters confusion regarding the presence of the prior distribution f_Θ(θ) in their calculations. They seek clarification on whether their interpretation of the Radon-Nikodym derivative is correct and if there are any typos in the referenced notes.

PREREQUISITES
  • Understanding of Bayes' theorem and its applications
  • Familiarity with Radon-Nikodym derivatives
  • Knowledge of conditional densities and prior distributions
  • Basic proficiency in measure theory and probability theory
NEXT STEPS
  • Study the derivation of the Radon-Nikodym theorem in measure theory
  • Learn about the implications of Bayes' theorem in statistical inference
  • Examine conditional probability distributions and their properties
  • Review advanced topics in Bayesian statistics, focusing on prior predictive checks
USEFUL FOR

Statisticians, data scientists, and researchers in Bayesian statistics who are looking to deepen their understanding of the Radon-Nikodym derivative and its application in probabilistic models.

Jatex
Messages
1
Reaction score
0
Theorem 1.31 (Bayes' theorem):
Suppose that ##X## has a parametric family ##\mathcal{P}_0## of distributions with parameter space ##\Omega##.
Suppose that ##P_\theta \ll \nu## for all ##\theta \in \Omega##, and let ##f_{X\mid\Theta}(x\mid\theta)## be the conditional density (with respect to ##\nu##) of ##X## given ##\Theta = \theta##.
Let ##\mu_\Theta## be the prior distribution of ##\Theta##.
Let ##\mu_{\Theta\mid X}(\cdot \mid x)## denote the conditional distribution of ##\Theta## given ##X = x##.
Then ##\mu_{\Theta\mid X} \ll \mu_\Theta##, a.s. with respect to the marginal of ##X##, and the Radon-Nikodym derivative is
$$
\frac{d\mu_{\Theta\mid X}}{d\mu_\Theta}(\theta \mid x)
= \frac{f_{X\mid \Theta}(x\mid \theta)}{\int_\Omega f_{X\mid\Theta}(x\mid t) \, d\mu_\Theta(t)}
$$
for those ##x## such that the denominator is neither ##0## nor infinite.
The prior predictive probability of the set of ##x## values such that the denominator is ##0## or infinite is ##0##, hence the posterior can be defined arbitrarily for such ##x## values.

I tried to derive the right hand side of the Radon-Nikodym derivative above but I got different result, here is my attempt:

\begin{equation} \label{eq1}
\begin{split}
\frac{\mathrm d\mu_{\Theta\mid X}}{\mathrm d\mu_\Theta}(\theta \mid x) &= f_{\Theta\mid X}(\theta\mid x) \mathrm \space \space[1]\\
&=\frac{f_{X\mid \Theta}(x\mid \theta) \cdot f_{\Theta}(\theta)}{f_X(x)}\\
&=\frac{f_{X\mid \Theta}(x\mid \theta) \cdot f_{\Theta}(\theta)}{\int_\Omega f_{X\mid\Theta}(x\mid t) \, \cdot f_{\Theta}(t) \space \mathrm dt}\\
&=\frac{f_{X\mid \Theta}(x\mid \theta) \cdot f_{\Theta}(\theta)}{\int_\Omega f_{X\mid\Theta}(x\mid t) \, \mathrm d\mu_\Theta(t)}
\end{split}
\end{equation}

but now, where does ##f_{\Theta}(\theta)## go?

for ##[1]## see slide ##10## of the following document: http://mlg.eng.cam.ac.uk/mlss09/mlss_slides/Orbanz_1.pdf

Thanks in advance.
 
Last edited by a moderator:
Physics news on Phys.org
Jatex said:
but now, where does ##f_{\Theta}(\theta)## go?

The ##f_{\Theta}(\theta)## is present in the expression for ##f_{\Theta|X}(t|x)## according to http://web.stanford.edu/class/stats200/Lecture20.pdf eq. 20.1.

Is the error in saying ##\frac{ d \mu_{\Theta|X}}{d \mu_\Theta} (t,x)= f_{\Theta|X}(t,x) ##? What is he definition of ##\frac{ d \mu_{\Theta|X}}{d \mu_\Theta}##?

Or is there a typo in the notes you quoted?
 

Similar threads

  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 1 ·
Replies
1
Views
1K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 4 ·
Replies
4
Views
4K
  • · Replies 11 ·
Replies
11
Views
9K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 8 ·
Replies
8
Views
2K
Replies
5
Views
2K