- #1

- 699

- 5

## Homework Statement

Determine the minimum mean square error for the joint PMF. You will need to evaluate ##E_{X, Y}[(Y - 14/11\cdot X - 1/11)^2]##.

## Homework Equations

## The Attempt at a Solution

The answer is ##\frac{3}{22}##, but when I work it out, I get ##\frac{203}{484}##. From my values, I have the same expected values, covariance, and variance of X as the book. Therefore, since my numbers are on point, I cant figure out why my solution is so off.

The ##mse(a, b) = E_{X, Y}[(Y - \hat{Y})^2] = E_{X, Y}[(Y - aX - B)^2]## since ##\hat{Y} = aX + b##. Expanding out ##mse(a, b)## and taking the derivatives of ##\frac{\partial mse(a, b)}{\partial a} = 0## and ##\frac{\partial mse(a, b)}{\partial b} = 0##, we get that

\begin{alignat*}{2}

a_{opt} &= \frac{cov(X, Y)}{var(X)} &&={} \frac{14}{11}\\

b_{opt} &= E[Y] - \frac{cov(X, Y)}{var(X)}E[X] &&={} \frac{1}{11}

\end{alignat*}

Then the ##mse(a, b)## is

\begin{align*}

mse(a, b) &= E[Y^2] - 2aE[XY] + a^2E[X^2] - 2bE[Y] + 2abE[X] + b^2\\

&= E[Y^2] - \frac{28}{11}E[XY] + \frac{196}{121}E[X^2] -

\frac{2}{11}E[Y] + \frac{28}{121}E[X] + \frac{1}{121}

\end{align*}

In order to find ##mse(a, b)##, we need to know the expected value of ##X##, ##Y##, ##E[X^2]##, and ##E[X^2]## as well as ##E[XY]##.

\begin{alignat*}{4}

E[X] &= \sum_ix_ip_X[x_i] &\quad &

E[Y] &&={} \sum_jy_jp_Y[y_j]\\

&= 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{2} & &

&&={} 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{4} + 3\frac{1}{4}\\

&= \frac{5}{4} & &

&&={} \frac{3}{2}\\

E[X^2] &= \sum_ix_i^2p_X[x_i] & &

E[Y^2] &&={} \sum_jy_j^2p_Y[y_j]\\

&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{2} & &

&&={} 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} + 3^2\frac{1}{4}\\

&= \frac{9}{4} & &

&&={} \frac{15}{4}\\

E[XY] &= \sum_{i, j}x_iy_jp_{X, Y}[x_i, y_j]\\

&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} +

2\cdot 3\frac{1}{4}\\

&= \frac{11}{4}

\end{alignat*}

Now, that we have remaining components, we can obtain ##mse(a, b)##.

\begin{align*}

&= \frac{15}{4} - \frac{28}{11}\frac{11}{4} +

\frac{196}{121}\frac{9}{4} - \frac{2}{11}\frac{3}{2} +

\frac{28}{121}\frac{5}{4} + \frac{1}{121}\\

&= \frac{203}{484}

\end{align*}

Last edited: