# Minimum mean square error for two random variables

## Homework Statement

Determine the minimum mean square error for the joint PMF. You will need to evaluate ##E_{X, Y}[(Y - 14/11\cdot X - 1/11)^2]##. ## The Attempt at a Solution

The answer is ##\frac{3}{22}##, but when I work it out, I get ##\frac{203}{484}##. From my values, I have the same expected values, covariance, and variance of X as the book. Therefore, since my numbers are on point, I cant figure out why my solution is so off.

The ##mse(a, b) = E_{X, Y}[(Y - \hat{Y})^2] = E_{X, Y}[(Y - aX - B)^2]## since ##\hat{Y} = aX + b##. Expanding out ##mse(a, b)## and taking the derivatives of ##\frac{\partial mse(a, b)}{\partial a} = 0## and ##\frac{\partial mse(a, b)}{\partial b} = 0##, we get that
\begin{alignat*}{2}
a_{opt} &= \frac{cov(X, Y)}{var(X)} &&={} \frac{14}{11}\\
b_{opt} &= E[Y] - \frac{cov(X, Y)}{var(X)}E[X] &&={} \frac{1}{11}
\end{alignat*}
Then the ##mse(a, b)## is
\begin{align*}
mse(a, b) &= E[Y^2] - 2aE[XY] + a^2E[X^2] - 2bE[Y] + 2abE[X] + b^2\\
&= E[Y^2] - \frac{28}{11}E[XY] + \frac{196}{121}E[X^2] -
\frac{2}{11}E[Y] + \frac{28}{121}E[X] + \frac{1}{121}
\end{align*}
In order to find ##mse(a, b)##, we need to know the expected value of ##X##, ##Y##, ##E[X^2]##, and ##E[X^2]## as well as ##E[XY]##.
\begin{alignat*}{4}
E[X] &= \sum_ix_ip_X[x_i] &\quad &
E[Y] &&={} \sum_jy_jp_Y[y_j]\\
&= 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{2} & &
&&={} 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{4} + 3\frac{1}{4}\\
&= \frac{5}{4} & &
&&={} \frac{3}{2}\\
E[X^2] &= \sum_ix_i^2p_X[x_i] & &
E[Y^2] &&={} \sum_jy_j^2p_Y[y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{2} & &
&&={} 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} + 3^2\frac{1}{4}\\
&= \frac{9}{4} & &
&&={} \frac{15}{4}\\
E[XY] &= \sum_{i, j}x_iy_jp_{X, Y}[x_i, y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} +
2\cdot 3\frac{1}{4}\\
&= \frac{11}{4}
\end{alignat*}
Now, that we have remaining components, we can obtain ##mse(a, b)##.
\begin{align*}
&= \frac{15}{4} - \frac{28}{11}\frac{11}{4} +
\frac{196}{121}\frac{9}{4} - \frac{2}{11}\frac{3}{2} +
\frac{28}{121}\frac{5}{4} + \frac{1}{121}\\
&= \frac{203}{484}
\end{align*}

Last edited:

Ray Vickson
Science Advisor
Homework Helper
Dearly Missed

## Homework Statement

Determine the minimum mean square error for the joint PMF. You will need to evaluate ##E_{X, Y}[(Y - 14/11\cdot X - 1/11)^2]##.

## The Attempt at a Solution

The answer is ##\frac{3}{22}##, but when I work it out, I get ##\frac{203}{242}##. From my values, I have the same expected values, covariance, and variance of X as the book. Therefore, since my numbers are on point, I cant figure out why my number is so off.

The ##mse(a, b) = E_{X, Y}[(Y - \hat{Y})^2] = E_{X, Y}[(Y - aX - B)^2]## since ##\hat{Y} = aX + b##. Expanding out ##mse(a, b)## and taking the derivatives of ##\frac{\partial mse(a, b)}{\partial a} = 0## and ##\frac{\partial mse(a, b)}{\partial b} = 0##, we get that
\begin{alignat*}{2}
a_{opt} &= \frac{cov(X, Y)}{var(X)} &&={} \frac{14}{11}\\
b_{opt} &= E[Y] - \frac{cov(X, Y)}{var(X)}E[X] &&={} \frac{1}{11}
\end{alignat*}
Then the ##mse(a, b)## is
\begin{align*}
mse(a, b) &= E[Y^2] - 2aE[XY] + a^2E[X^2] - 2bE[Y] + 2abE[X] + b^2\\
&= E[Y^2] - \frac{28}{11}E[XY] + \frac{196}{121}E[X^2] -
\frac{2}{11}E[Y] + \frac{28}{121}E[X] + \frac{1}{121}
\end{align*}
In order to find ##mse(a, b)##, we need to know the expected value of ##X##, ##Y##, ##E[X^2]##, and ##E[X^2]## as well as ##E[XY]##.
\begin{alignat*}{4}
E[X] &= \sum_ix_ip_X[x_i] &\quad &
E[Y] &&={} \sum_jy_jp_Y[y_j]\\
&= 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{2} & &
&&={} 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{4} + 3\frac{1}{4}\\
&= \frac{5}{4} & &
&&={} \frac{3}{2}\\
E[X^2] &= \sum_ix_i^2p_X[x_i] & &
E[Y^2] &&={} \sum_jy_j^2p_Y[y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{2} & &
&&={} 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} + 3^2\frac{1}{4}\\
&= \frac{9}{4} & &
&&={} \frac{15}{4}\\
E[XY] &= \sum_{i, j}x_iy_jp_{X, Y}[x_i, y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} +
2\cdot 3\frac{1}{4}\\
&= \frac{11}{4}
\end{alignat*}
Now, that we have remaining components, we can obtain ##mse(a, b)##.Y - (14/11)X -
\begin{align*}
&= \frac{15}{4} - \frac{28}{11}\frac{11}{4} +
\frac{196}{121}\frac{9}{4} - \frac{2}{11}\frac{3}{2} +
\frac{28}{121}\frac{5}{4} + \frac{1}{121}\\
&= \frac{203}{242}
\end{align*}

I do not fully understand the notation ##(Y - 14/11\cdot X - 1/11)^2##. Do you mean
$$\left( Y - \frac{14}{11} X - \frac{1}{11} \right)^2 ?$$
If so, use the "\frac" command in TeX. If you mean something else, try using appropriate parentheses. Or, you could/should even use parentheses without the "\frac" command---for example, write ##(Y - (14/11)X - 1/11)^2## if that is what you mean. Alternatively, you could write ##(Y - aX - b)^2,## where ##a = 14/11, \, b = 1/11##. Personally, I like this last way the best, as it helps keep different factors and influences separate.

I do not fully understand the notation ##(Y - 14/11\cdot X - 1/11)^2##. Do you mean
$$\left( Y - \frac{14}{11} X - \frac{1}{11} \right)^2 ?$$
If so, use the "\frac" command in TeX. If you mean something else, try using appropriate parentheses. Or, you could/should even use parentheses without the "\frac" command---for example, write ##(Y - (14/11)X - 1/11)^2## if that is what you mean. Alternatively, you could write ##(Y - aX - b)^2,## where ##a = 14/11, \, b = 1/11##. Personally, I like this last way the best, as it helps keep different factors and influences separate.

I have no idea what to say if you dont understand the meaning of ##(Y - 14/11\cdot X - 1/11)^2##. I am not trying to be rude, but I don't need an education in LaTeX.

Mark44
Mentor
I have no idea what to say if you dont understand the meaning of ##(Y - 14/11\cdot X - 1/11)^2##. I am not trying to be rude, but I don't need an education in LaTeX.
Dustin, I'm pretty sure what you wrote is exactly what you intended to write. The reason for Ray's comment is that we get so many people who write x - 2/y - 3 when they really mean (x - 2)/(y - 3).

Dustin, I'm pretty sure what you wrote is exactly what you intended to write. The reason for Ray's comment is that we get so many people who write x - 2/y - 3 when they really mean (x - 2)/(y - 3).

Even if one is unsure, look past the question into the attempt. If that was done, one would see the proper definition Y - aX - B and realize X is not in the denominator. Critiquing a question in which you didn't fully read is pointless since the reader was premature in their analysis when the clarity was already present and verified later.

haruspex
Science Advisor
Homework Helper
Gold Member
2020 Award
I get a different number for E[Y2], but it still doesn't get me to 3/22.

I get a different number for E[Y2], but it still doesn't get me to 3/22.

Did you get 7/2 and then 41/242 for the mean square error? I see my addition error in E[Y^2].

LCKurtz
Science Advisor
Homework Helper
Gold Member
I have no idea what to say if you dont understand the meaning of ##(Y - 14/11\cdot X - 1/11)^2##. I am not trying to be rude, but I don't need an education in LaTeX.

Even if one is unsure, look past the question into the attempt. If that was done, one would see the proper definition Y - aX - B and realize X is not in the denominator. Critiquing a question in which you didn't fully read is pointless since the reader was premature in their analysis when the clarity was already present and verified later.

Sorry, but I agree with Ray here. It isn't our job to look past what is written to see if we can figure out what was meant. When posters neglect needed parentheses we suggest they use them just as we make other suggestions to help posters solve their problems or how to write their solutions up properly. Often their misuse of parentheses is the cause of later algebra mistakes. Instead of telling us you don't need an education in LaTeX perhaps you could say something like "Thanks, I will try to be more careful." If you aren't "trying to be rude", then don't be rude.

Sorry, but I agree with Ray here. It isn't our job to look past what is written to see if we can figure out what was meant. When posters neglect needed parentheses we suggest they use them just as we make other suggestions to help posters solve their problems or how to write their solutions up properly. Often their misuse of parentheses is the cause of later algebra mistakes. Instead of telling us you don't need an education in LaTeX perhaps you could say something like "Thanks, I will try to be more careful." If you aren't "trying to be rude", then don't be rude.

For me, this is the 2nd time Ray hasn't read and made a mistake.

On this post: https://www.physicsforums.com/threads/cov-w-z-where-w-x-and-z-ax-y.775444/ he mentioned and identity to use that I already listed. So he has a recorded with me of not reading the question in full and making a suggestion mistake.

Additionally, I know how to format my questions and wouldnt make the error of saying times when I want to divide that as a quantity. I wasn't being careless and didnt need to be more careful since that format is what I meant and is correct.

haruspex
Science Advisor
Homework Helper
Gold Member
2020 Award
I get a different number for E[Y2], but it still doesn't get me to 3/22.
The other error is in your calculation of B.
As you probably realise, it is very easy to get the 3/22 result once you have A and B. Just calculate Y errors.

• Dustinsfl
The other error is in your calculation of B.
As you probably realise, it is very easy to get the 3/22 result once you have A and B. Just calculate Y errors.

Thanks sign error. All is good now.