View Single Post
kasraa is offline
Apr20-10, 10:15 AM
P: 16
Quote Quote by SW VandeCarr View Post
I think this article may help.

I take it that P(Z) is your unconditional probability density and p(Z|x) is your likelihood function. Then taking the joint density p(x)p(Z|x) you can use Bayes Theorem to for the posterior density which is the conditional p(x|Z)=p(Z|x)p(x)/p(Z).

I'm not sure why you think the unconditional and conditional probability densities would be equal unless, of course, the prior density and the posterior density were equal. It appears that the MMSE estimate applies to the posterior density p(x|Z).

EDIT: The link is a bit slow, but works as of my testing at the edit time.
Part one:

The posterior [tex] p \left( x|Z \right) [/tex], has a mean and a (co)variance. Its mean is the MMSE estimator, [tex] E \left[ x|Z \right] [/tex], and its variance (or the trace of its covariance matrix, if it's a random vector) is the minimum mean squared error. Am I right?

So the trace of conditional (co)variance ((co)variance of conditional pdf), that is the trace of
[tex] E \left[ \left( x - E \left[ x|Z \right] \right) \left( x - E \left[ x|Z \right] \right)^{T} | Z \right] [/tex]
is the minimum MSE (and
[tex] E \left[ \left(x-E \left[ x|Z \right] \right)^2 | Z \right] [/tex]
for the case of scalar RV).
Is it correct?

And then what is the trace of
[tex] E \left[ \left( x - E \left[ x|Z \right] \right) \left( x - E \left[ x|Z \right] \right)^{T}\right] [/tex]
[tex] E \left[ \left(x-E \left[ x|Z \right] \right)^2 \right] [/tex]
for the case of scaler RV).

Part Two:

As I know MMSE estimation is about finding [tex] h \left( . \right) [/tex] that minimizes the
[tex] E \left[ \left( x - h \left( Z \right) \right)^2 \right] [/tex] (MSE).
And the answer is [tex] h \left( Z \right) = E \left[ x | Z \right] [/tex].

So the MMSE is
[tex] E \left[ \left(x-E \left[ x|Z \right] \right)^2 \right] [/tex].

Can you see the problem?

And a new one :D Maybe it's the answer.

Orthogonality principle implies [tex] E \left[ \left( x - E \left[ x|Z \right] \right)Z \right] = 0 [/tex], which implies
[tex] E \left[ \left( x - E \left[ x|Z \right] \right)| Z \right] = E \left[ \left( x - E \left[ x|Z \right] \right) \right] [/tex].

Does it also imply:
[tex] E \left[ \left( x - E \left[ x|Z \right] \right) ^2 | Z \right] = E \left[ \left( x - E \left[ x|Z \right] \right)^2 \right] [/tex]?
Is it correct?