Solving Minimization Problem Involving Variance & Covariance

Click For Summary

Discussion Overview

The discussion revolves around minimizing expressions involving variance and covariance, specifically focusing on the constants and random variables involved in the minimization problems. Participants explore the mathematical derivation of minimizing expected squared differences and the implications of using random variables versus constants in these contexts.

Discussion Character

  • Technical explanation
  • Mathematical reasoning
  • Debate/contested

Main Points Raised

  • One participant asserts that the value of $b$ that minimizes $E[(X-b)^2]$ is $b=E[X]$ and provides a proof involving differentiation.
  • Another participant agrees with the proof but raises a second question about minimizing $E[(Y-aX-b)^2]$, suggesting that their earlier reasoning may not apply due to the nature of $Y$ as a random variable.
  • A participant points out that if $Y$ is a random variable, the minimization problem becomes constrained and involves variances and covariances, indicating a more complex relationship than initially considered.
  • Further clarification is sought regarding the use of expectations and variances of $X$ in relation to the covariance term, questioning whether substitutions can be made without losing generality.
  • Another participant emphasizes that $E(X)$ and $Var(X)$ are constants in the context of the problem, while the covariance term is a variable in the optimization process.

Areas of Agreement / Disagreement

Participants generally agree on the minimization of $E[(X-b)^2]$ leading to $b=E[X]$, but there is disagreement and uncertainty regarding the second minimization problem involving $Y$, with multiple interpretations of how to handle random variables versus constants.

Contextual Notes

The discussion highlights the complexity of handling random variables in minimization problems, particularly in terms of constraints and the relationships between expectations, variances, and covariances. There are unresolved aspects regarding the assumptions made about the nature of the variables involved.

OhMyMarkov
Messages
81
Reaction score
0
Hello Everyone!

What $b$ minimizes $E[(X-b)^2]$ where $b$ is some constant, isn't it $b=E[X]$? Is it right to go about the proof as follows:

$E[(X-b)^2] = E[(X^2+b^2-2bX)] = E[X^2] + E[b^2]-2bE[X]$, but $E = b$, we differentiate with respect to $b$ and set to zero, we obtain that $b=E[X]$. Is this proof correct? I was thinking it was until I got this problem:

What $Y$ minimizes $E[(Y-aX-b)^2]$? The given expression contains variances and covariances, but all I get was $Y=aE[X]+b$.

What am I doing wrong here?

Any help is appreciated! :D
 
Physics news on Phys.org
OhMyMarkov said:
Hello Everyone!

What $b$ minimizes $E[(X-b)^2]$ where $b$ is some constant, isn't it $b=E[X]$? Is it right to go about the proof as follows:

$E[(X-b)^2] = E[(X^2+b^2-2bX)] = E[X^2] + E[b^2]-2bE[X]$, but $E = b$, we differentiate with respect to $b$ and set to zero, we obtain that $b=E[X]$. Is this proof correct?


Correct


I was thinking it was until I got this problem:

What $Y$ minimizes $E[(Y-aX-b)^2]$? The given expression contains variances and covariances, but all I get was $Y=aE[X]+b$.

What am I doing wrong here?

Any help is appreciated! :D

The problem with this second question is that with normal naming conventions \(Y\) is a random variable not a constant, if it were a constant what you get would be correct. If it is a RV then it leaves you with a minimisation problem where the variables are \( \overline{Y}\), \(Var(Y)\), \( Covar(Y,aX+b)\). This is a constrained minimisation problem as \( | Covar(Y,aX+b)| \le \sqrt{Var(Y)Var(aV+b)}\)

CB
 
Last edited:
Thank you CaptainBlack!

But, you mentioned [FONT=MathJax_Math-italic]E[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]V[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]r[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]C[FONT=MathJax_Math-italic]o[FONT=MathJax_Math-italic]v[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main],[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]+[FONT=MathJax_Math-italic]b[FONT=MathJax_Main]] , what about [FONT=MathJax_Math-italic]E[FONT=MathJax_Main][[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]V[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]r[FONT=MathJax_Main][[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]] , can we sub them for [FONT=MathJax_Math-italic]C[FONT=MathJax_Math-italic]o[FONT=MathJax_Math-italic]v[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main],[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]+[FONT=MathJax_Math-italic]b[FONT=MathJax_Main]] ? Or are the variables intentionally used in this fashion so that the hand calculation becomes easier?
 
Last edited:
OhMyMarkov said:
Thank you CaptainBlack!

But, you mentioned [FONT=MathJax_Math-italic]E[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]V[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]r[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]C[FONT=MathJax_Math-italic]o[FONT=MathJax_Math-italic]v[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main],[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]+[FONT=MathJax_Math-italic]b[FONT=MathJax_Main]] , what about [FONT=MathJax_Math-italic]E[FONT=MathJax_Main][[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]][FONT=MathJax_Main],[FONT=MathJax_Math-italic]V[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]r[FONT=MathJax_Main][[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]] , can we sub them for [FONT=MathJax_Math-italic]C[FONT=MathJax_Math-italic]o[FONT=MathJax_Math-italic]v[FONT=MathJax_Main][[FONT=MathJax_Math-italic]Y[FONT=MathJax_Main],[FONT=MathJax_Math-italic]a[FONT=MathJax_Math-italic]X[FONT=MathJax_Main]+[FONT=MathJax_Math-italic]b[FONT=MathJax_Main]] ? Or are the variables intentionally used in this fashion so that the hand calculation becomes easier?

\(E(X)\) and \(Var(X)\) are constants for this problem, while \(u=Covar(Y,aX+b)\) is one of the variable in the optimisation problem.

CB
 

Similar threads

MHB Distribution, expected value, variance, covariance and correlation
  • · Replies 14 ·
Replies
14
Views
2K
MHB How Do You Minimize Mean Square Deviation in Dice Roll Predictions?
  • · Replies 4 ·
Replies
4
Views
1K
Undergrad On Jensen's inequality for conditional expectation
  • · Replies 1 ·
Replies
1
Views
2K
Graduate Computing a variance in astrophysics context
  • · Replies 1 ·
Replies
1
Views
1K
MHB Cdf, expectation, and variance of a random continuous variable
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad Help understanding conditional expectation identity
  • · Replies 1 ·
Replies
1
Views
2K
MHB No problem! Happy to help. (Smile)
  • · Replies 9 ·
Replies
9
Views
2K
Undergrad Expected value of X~Geom(p) given X+Y=z
  • · Replies 5 ·
Replies
5
Views
2K
MHB Counterfactual Expectation Calculation
  • · Replies 1 ·
Replies
1
Views
2K
MHB Calc Expected Value & Variance of Multivar. Func.
  • · Replies 1 ·
Replies
1
Views
2K