- #1

- 128

- 0

## Main Question or Discussion Point

Hello,

I am trying to understand what makes estimating the posterior distribution such a hard problem.

So, imagine I need to estimate the posterior distribution over a set of parameters given the data y, so a quantity [itex]P(\theta|y)[/itex] and [itex]\theta[/itex] is generally high dimensional.

The prior over [itex]\theta[/itex] is a multivariate Gaussian i.e. [itex]P(θ)∼N(θ;0,Σ)[/itex]

The likelihood i.e. [itex]P(y|θ)[/itex] can be written down as product over Gaussian likelihoods.

Now, it seems to be that the posterior distribution will also be Gaussian. Is that correct?

Secondly, going through Bishop's book, it seems that the conditional posterior distributions and the marginal distributions will be Gaussian as well (assuming that the joint distribution over the parameters and data is Gaussian) and should have a closed form solution. If that is the case, why is this problem intractable?

If I need to find the parameters of this posterior distribution, can this not be set as an optimisation problem where I estimate the mean and covariance of the posterior Gaussian? I am basically having trouble visualising why this problem is complicated?

I am trying to understand what makes estimating the posterior distribution such a hard problem.

So, imagine I need to estimate the posterior distribution over a set of parameters given the data y, so a quantity [itex]P(\theta|y)[/itex] and [itex]\theta[/itex] is generally high dimensional.

The prior over [itex]\theta[/itex] is a multivariate Gaussian i.e. [itex]P(θ)∼N(θ;0,Σ)[/itex]

The likelihood i.e. [itex]P(y|θ)[/itex] can be written down as product over Gaussian likelihoods.

Now, it seems to be that the posterior distribution will also be Gaussian. Is that correct?

Secondly, going through Bishop's book, it seems that the conditional posterior distributions and the marginal distributions will be Gaussian as well (assuming that the joint distribution over the parameters and data is Gaussian) and should have a closed form solution. If that is the case, why is this problem intractable?

If I need to find the parameters of this posterior distribution, can this not be set as an optimisation problem where I estimate the mean and covariance of the posterior Gaussian? I am basically having trouble visualising why this problem is complicated?