Tractability of posterior distributions

  • Thread starter pamparana
  • Start date
  • #1
128
0

Main Question or Discussion Point

Hello,
I am trying to understand what makes estimating the posterior distribution such a hard problem.

So, imagine I need to estimate the posterior distribution over a set of parameters given the data y, so a quantity [itex]P(\theta|y)[/itex] and [itex]\theta[/itex] is generally high dimensional.

The prior over [itex]\theta[/itex] is a multivariate Gaussian i.e. [itex]P(θ)∼N(θ;0,Σ)[/itex]

The likelihood i.e. [itex]P(y|θ)[/itex] can be written down as product over Gaussian likelihoods.

Now, it seems to be that the posterior distribution will also be Gaussian. Is that correct?

Secondly, going through Bishop's book, it seems that the conditional posterior distributions and the marginal distributions will be Gaussian as well (assuming that the joint distribution over the parameters and data is Gaussian) and should have a closed form solution. If that is the case, why is this problem intractable?

If I need to find the parameters of this posterior distribution, can this not be set as an optimisation problem where I estimate the mean and covariance of the posterior Gaussian? I am basically having trouble visualising why this problem is complicated?
 

Answers and Replies

  • #2
Stephen Tashi
Science Advisor
6,963
1,208
If I need to find the parameters of this posterior distribution, can this not be set as an optimisation problem where I estimate the mean and covariance of the posterior Gaussian?
I don't understand how you will set up the problem. If we have a multivariate gaussian we can estimate its parameters from observations of the variates. If you have a Gaussian posterior distribution where the variables are [itex] \theta [/itex], are you assuming you have data that gives direct observations of [itex] \theta [/itex] ?
 

Related Threads for: Tractability of posterior distributions

Replies
6
Views
1K
  • Last Post
Replies
0
Views
1K
Replies
8
Views
1K
Replies
5
Views
1K
Replies
1
Views
602
  • Last Post
Replies
1
Views
847
Top