Proof of the MMSE Estimator of $\mathbf{\theta}\left[n\right]$

  • Thread starter prhzn
  • Start date
  • Tags
    Proof
In summary, the MMSE estimator for the parameter \mathbf{\theta}\left[n\right] is \mathbf{\hat{\theta}}\left[n\right] =\mathbf{A}^n\mathbf{\hat{\theta}}\left[0\right] or \mathbf{\hat{\theta}}\left[n\right] = \mathbf{A}\mathbf{\hat{\theta}}\left[n-1\right], where \mathbf{\hat{\theta}}\left[0\right] is the MMSE estimator of \mathbf{\theta}\left[0\right]. This is proven by showing that \mathbf{\theta}\left[n\right] can
  • #1
prhzn
9
0

Homework Statement



Consider a parameter [tex]\mathbf{\theta}[/tex] which changes with time according to the deterministic relation

[tex]\mathbf{\theta}\left[n\right] = \mathbf{A}\mathbf{\theta}\left[n-1\right]\; n\geq 1[/tex],

where [tex]\mathbf{A}[/tex] is a known [tex]p\times p[/tex] matrix and [tex]\mathbf{\theta}\left[0\right][/tex] is an unknown parameter which is modeled as a random ([tex]p\times 1[/tex]) vector. Note that once [tex]\mathbf{\theta}\left[0\right][/tex] is specified, so is [tex]\mathbf{\theta}\left[n\right][/tex] for [tex]n\geq 1[/tex].

Prove that the MMSE estimator of [tex]\mathbf{\theta}\left[n\right][/tex] is

[tex]\mathbf{\hat{\theta}}\left[n\right] =\mathbf{A}^n\mathbf{\hat{\theta}}\left[0\right][/tex],

where [tex]\mathbf{\hat{\theta}}\left[0\right][/tex] is the MMSE estimator of [tex]\mathbf{\theta}\left[0\right][/tex], or equivalently,

[tex]\mathbf{\hat{\theta}}\left[n\right] = \mathbf{A}\mathbf{\hat{\theta}}\left[n-1\right][/tex]

Homework Equations



MMSE: [tex]\hat{\theta} = \mathbb{E}\left[\theta|\mathbf{x}\right] = \int p\left(\theta\right)\ln\left(p\left(\theta|\mathbf{x}\right)\right)\mathrm{d}\theta[/tex]

The Attempt at a Solution



So far I haven't got any good attempt as my main problem is how to start. Until now, all exercises about MMSE that I've done have specified information about the PDF's to some of the variables or some other information that has made it more obvious how to start; however, with this I feel like I'm a bit lost. So mainly I'm just looking for a hint on how to start, such that I can do an fair attempt on my own.
 
Last edited:
Physics news on Phys.org
  • #2
Not sure if this is correct, maybe someone can tell or not?

We know that [tex]\mathbf{\theta}[n] = \mathbf{A}\mathbf{\theta}[n-1][/tex] for [tex]n\geq 1[/tex].

Then [tex]\mathbf{\theta}[n] = \mathbf{A}\left(\mathbf{A}\mathbf{\theta}[n-2]\right)[/tex] and so on, resulting in [tex]\mathbf{\theta}[n] = \mathbf{A}^n\mathbf{\theta}[0][/tex].

The MMSE is then

[tex]\mathbf{\hat{\theta}}[n] = \mathbb{E}\left[\mathbf{\theta}[n]|\mathbf{x}\right][/tex]

Doing the same "trick" as above we get

[tex]\mathbf{\hat{\theta}}[n] = \mathbf{A}^n\mathbb{E}\left[\mathbf{\theta}[0]|\mathbf{x}\right][/tex].

We already know that [tex]\mathbf{\hat{\theta}}[0][/tex] is the MMSE estimator of [tex]\mathbf{\theta}[0][/tex]; hence, the proof is complete.
 
Last edited:

What is the MMSE estimator of θ[n] and how is it used in scientific research?

The MMSE (Minimum Mean Square Error) estimator is a mathematical tool used in statistics and signal processing to estimate the value of a random variable or signal. It is used in scientific research to analyze and quantify data, make predictions, and test hypotheses.

What is the difference between the MMSE estimator and other estimation methods?

The MMSE estimator is a Bayesian method that takes into account prior knowledge about the variable being estimated. It minimizes the expected value of the squared error between the estimated value and the true value. Other estimation methods, such as maximum likelihood or least squares, do not take into account prior knowledge and focus solely on minimizing the error.

How is the MMSE estimator calculated?

The MMSE estimator is calculated using the conditional expectation formula, where the estimated value is equal to the expected value of the random variable given the observations. This involves calculating the mean and covariance of the random variable and the observations.

What are the assumptions and limitations of the MMSE estimator?

The MMSE estimator assumes that the data is normally distributed and that the prior knowledge is accurate. If these assumptions are not met, the results of the estimator may not be reliable. Additionally, the MMSE estimator can only estimate linear functions of the variable, so it may not be suitable for nonlinear relationships.

Can the MMSE estimator be used for any type of data?

The MMSE estimator can be used for any type of data as long as the assumptions and limitations are met. However, it is most commonly used for continuous and normally distributed data, such as in signal processing and financial analysis.

Similar threads

Replies
33
Views
3K
  • Calculus and Beyond Homework Help
Replies
11
Views
1K
  • Calculus and Beyond Homework Help
Replies
4
Views
1K
Replies
4
Views
143
  • Calculus and Beyond Homework Help
Replies
4
Views
1K
Replies
2
Views
1K
Replies
4
Views
544
  • Calculus and Beyond Homework Help
Replies
2
Views
1K
  • Introductory Physics Homework Help
Replies
3
Views
955
  • Advanced Physics Homework Help
Replies
1
Views
865
Back
Top