Solving Minimization Problem Involving Variance & Covariance

[OhMyMarkov]

Hello Everyone!

What $b$ minimizes $E[(X-b)^2]$ where $b$ is some constant, isn't it $b=E[X]$? Is it right to go about the proof as follows:

$E[(X-b)^2] = E[(X^2+b^2-2bX)] = E[X^2] + E[b^2]-2bE[X]$, but $E = b$, we differentiate with respect to $b$ and set to zero, we obtain that $b=E[X]$. Is this proof correct? I was thinking it was until I got this problem:

What $Y$ minimizes $E[(Y-aX-b)^2]$? The given expression contains variances and covariances, but all I get was $Y=aE[X]+b$.

What am I doing wrong here?

Any help is appreciated! :D

 

[CaptainBlack]

Hello Everyone!

What $b$ minimizes $E[(X-b)^2]$ where $b$ is some constant, isn't it $b=E[X]$? Is it right to go about the proof as follows:

$E[(X-b)^2] = E[(X^2+b^2-2bX)] = E[X^2] + E[b^2]-2bE[X]$, but $E = b$, we differentiate with respect to $b$ and set to zero, we obtain that $b=E[X]$. Is this proof correct?

Correct

I was thinking it was until I got this problem:

What $Y$ minimizes $E[(Y-aX-b)^2]$? The given expression contains variances and covariances, but all I get was $Y=aE[X]+b$.

What am I doing wrong here?

Any help is appreciated! :D

The problem with this second question is that with normal naming conventions \(Y\) is a random variable not a constant, if it were a constant what you get would be correct. If it is a RV then it leaves you with a minimisation problem where the variables are \( \overline{Y}\), \(Var(Y)\), \( Covar(Y,aX+b)\). This is a constrained minimisation problem as \( | Covar(Y,aX+b)| \le \sqrt{Var(Y)Var(aV+b)}\)

CB

 

[OhMyMarkov]

Thank you CaptainBlack!

But, you mentioned [FONT=MathJax_Math-italic]E[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]V[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]r[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]C[FONT=MathJax_Math-italic]o[FONT=MathJax_Math-italic]v[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main],[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]+[FONT=MathJax_Math-italic]b[FONT=MathJax_Main]] , what about [FONT=MathJax_Math-italic]E[FONT=MathJax_Main][[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]V[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]r[FONT=MathJax_Main][[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]] , can we sub them for [FONT=MathJax_Math-italic]C[FONT=MathJax_Math-italic]o[FONT=MathJax_Math-italic]v[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main],[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]+[FONT=MathJax_Math-italic]b[FONT=MathJax_Main]] ? Or are the variables intentionally used in this fashion so that the hand calculation becomes easier?

 

[CaptainBlack]

Thank you CaptainBlack!

But, you mentioned [FONT=MathJax_Math-italic]E[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]V[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]r[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]C[FONT=MathJax_Math-italic]o[FONT=MathJax_Math-italic]v[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main],[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]+[FONT=MathJax_Math-italic]b[FONT=MathJax_Main]] , what about [FONT=MathJax_Math-italic]E[FONT=MathJax_Main][[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]V[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]r[FONT=MathJax_Main][[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]] , can we sub them for [FONT=MathJax_Math-italic]C[FONT=MathJax_Math-italic]o[FONT=MathJax_Math-italic]v[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main],[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]+[FONT=MathJax_Math-italic]b[FONT=MathJax_Main]] ? Or are the variables intentionally used in this fashion so that the hand calculation becomes easier?

\(E(X)\) and \(Var(X)\) are constants for this problem, while \(u=Covar(Y,aX+b)\) is one of the variable in the optimisation problem.

CB