Tractability of posterior distributions

[pamparana]

Hello,
I am trying to understand what makes estimating the posterior distribution such a hard problem.

So, imagine I need to estimate the posterior distribution over a set of parameters given the data y, so a quantity P(\theta|y) and \theta is generally high dimensional.

The prior over \theta is a multivariate Gaussian i.e. P(θ)∼N(θ;0,Σ)

The likelihood i.e. P(y|θ) can be written down as product over Gaussian likelihoods.

Now, it seems to be that the posterior distribution will also be Gaussian. Is that correct?

Secondly, going through Bishop's book, it seems that the conditional posterior distributions and the marginal distributions will be Gaussian as well (assuming that the joint distribution over the parameters and data is Gaussian) and should have a closed form solution. If that is the case, why is this problem intractable?

If I need to find the parameters of this posterior distribution, can this not be set as an optimisation problem where I estimate the mean and covariance of the posterior Gaussian? I am basically having trouble visualising why this problem is complicated?

 

[Stephen Tashi]

If I need to find the parameters of this posterior distribution, can this not be set as an optimisation problem where I estimate the mean and covariance of the posterior Gaussian?

I don't understand how you will set up the problem. If we have a multivariate gaussian we can estimate its parameters from observations of the variates. If you have a Gaussian posterior distribution where the variables are \theta, are you assuming you have data that gives direct observations of \theta ?