How Do You Minimize the Squared Distance to a Line Using Linear Algebra?

[Mathman23]

The following formula $$E(a,b) = \sum_{j=1} ^{N} (y_{i} - (a + b_{i})^2$$ is used to messure the distance between the points $$(x_1,y_1),(x_2,y_2), \ldots, (x_n, y_n) \in \mathbb{R}^2$$ and the line $y=a+bx$

I need to find a set of points $$(a,b) \in \mathbb{R}^2$$ where E(a,b) using tools of linear Algebra.

I'm given the following vectors:

$$x = \left [ \begin{array}{c} x_{1} \\ \vdots \\ x_{N} \end{array} \right ], v = \left [ \begin{array}{c} y_{1} \\ \vdots \\ y_{N} \end{array} \right ], \ \ \overline{x} = \frac{1}{N} \sum_{j=1} ^N x_{i}, \ \ \overline{y} = \frac{1}{N} \sum_{j=1} ^N y_{i}$$

I'm tasked with calculating $$\mathb{\frac{dE}{da}} , \mathb{\frac{dE}{db}}$$ and showing that if $$\mathb{\frac{dE}{da}}=0 , \mathb{\frac{dE}{db}}= 0$$ leads to the linear system.

$$A \left [ \begin{array}{c} a \\ b \end{array} \right ] = c$$, where $$A = \left [ \begin{array}{cc} \frac{1}{x} \ \ \overline{x} \\ \ \ \frac{x \cdot x}{N} \end{array} \right ]$$ and $$c = \left [ \begin{array}{c} \overline{y} \\ \frac{x \cdot y}{N}\end{array} \right ]$$

Anybody have any hits/idear for to solve this assignment ??

Many thanks in advance

Sincerley Fred

 

[OlderDan]

The following formula $$E(a,b) = \sum_{j=1} ^{N} (y_{i} - (a + b_{i})^2$$ is used to messure the distance between the points $$(x_1,y_1),(x_2,y_2), \ldots, (x_n, y_n) \in \mathbb{R}^2$$ and the line $y=a+bx$

I need to find a set of points $$(a,b) \in \mathbb{R}^2$$ where E(a,b) using tools of linear Algebra.

I'm given the following vectors:

$$x = \left [ \begin{array}{c} x_{1} \\ \vdots \\ x_{N} \end{array} \right ], v = \left [ \begin{array}{c} y_{1} \\ \vdots \\ y_{N} \end{array} \right ], \ \ \overline{x} = \frac{1}{N} \sum_{j=1} ^N x_{i}, \ \ \overline{y} = \frac{1}{N} \sum_{j=1} ^N y_{i}$$

I'm tasked with calculating $$\mathb{\frac{dE}{da}} , \mathb{\frac{dE}{db}}$$ and showing that if $$\mathb{\frac{dE}{da}}=0 , \mathb{\frac{dE}{db}}= 0$$ leads to the linear system.

$$A \left [ \begin{array}{c} a \\ b \end{array} \right ] = c$$, where $$A = \left [ \begin{array}{cc} \frac{1}{x} \ \ \overline{x} \\ \ \ \frac{x \cdot x}{N} \end{array} \right ]$$ and $$c = \left [ \begin{array}{c} \overline{y} \\ \frac{x \cdot y}{N}\end{array} \right ]$$

Anybody have any hits/idear for to solve this assignment ??

Many thanks in advance

Sincerley Fred

First, let's make sure the problem is understood. This looks to me like the problem of linear regression. E(a,b) is the sum of the squared deviations in the y (vertical) direction of the data points from the line y = a + bx. Your task is to minimize E(a,b) by finding the values of a and b that result in that minimum. That point is found by taking the partial derivatives of E with respect to a and b and setting each to zero. Your derivatives should be

$$\mathb{\frac{\partial E}{\partial a}}=0 , \mathb{\frac{\partial E}{\partial b}}= 0$$

and your first equation should read

$$E(a,b) = \sum_{j=1} ^{N} (y_{j} - (a + bx_{j}))^2$$

What is missing in the matrix?

$$A = \left [ \begin{array}{cc} \frac{1}{x} \ \ \overline{x} \\ \ \ \frac{x \cdot x}{N} \end{array} \right ]$$

Looks like is should be this

$$A = \left[ {\begin{array}{*{20}c} 1 & {\overline x } \\ {\overline x } & {\frac{{ x \cdot x }}{N}} \\ \end{array}} \right]$$

 

[OlderDan]

Actually, his first equation should read:
$$E(a,b)=\sum_{j=1}^{N}(y_{j}-(a+bx_{j}))^{2}$$
EDIT:
I see OlderDan edited his reply; the index must agree too, of course.

Yep.. I missed the index. good catch.. I will fix that in my previous post also.

I missed the bottom half of those partial derivatives too . I fixed those also.

And then there are the other index problems from the OP

$$\overline{x} = \frac{1}{N} \sum_{j=1} ^N x_{j}, \ \ \overline{y} = \frac{1}{N} \sum_{j=1} ^N y_{j}$$

 

[Mathman23]

Hi

The matrix is suppose to look like this.

$$A = \left [ \begin{array}{cc} 1 & \overline{x} \\ \overline{x} & \frac{x \cdot x}{N} \end{array} \right ]$$

/Fred

 

[OlderDan]

Hi

I'm not sure what is missing in the matrix. Whats how my professor presented it.

/Fred

I think you copied it wrong. I just added what I think it should be to my first post.

 

[Mathman23]

Hi Dan and thanks for Your answer,

Then taking the partial derivative of E. First step is that to write the complete sum-formula ??

/Fred

First, let's make sure the problem is understood. This looks to me like the problem of linear regression. E(a,b) is the sum of the squared deviations in the y (vertical) direction of the data points from the line y = a + bx. Your task is to minimize E(a,b) by finding the values of a and b that result in that minimum. That point is found by taking the partial derivatives of E with respect to a and b and setting each to zero. Your derivatives should be

$$\mathb{\frac{\partial E}{\partial a}}=0 , \mathb{\frac{\partial E}{\partial b}}= 0$$

and your first equation should read

$$E(a,b) = \sum_{j=1} ^{N} (y_{j} - (a + bx_{j}))^2$$

What is missing in the matrix?

$$A = \left [ \begin{array}{cc} \frac{1}{x} \ \ \overline{x} \\ \ \ \frac{x \cdot x}{N} \end{array} \right ]$$

Looks like is should be this

$$A = \left[ {\begin{array}{*{20}c} 1 & {\overline x } \\ {\overline x } & {\frac{{ x \cdot x }}{N}} \\ \end{array}} \right]$$

 

[OlderDan]

Hi Dan and thanks for Your answer,

Then taking the partial derivative of E. First step is that to write the complete sum-formula ??

/Fred

Setting the two partial derivatives of E to zero gives you two equations for a and b. The partial with respct to a is simpler and yields an equation in a and b that involves only that averages of x and y. The partial with respect to b is a bit more complicated, involving the dot product of x and y and the dot product of x with itself. With the corrected A matrix you should be able to show that those equations are equivalent to the matrix equation you posted.

 

[Mathman23]

Setting the two partial derivatives of E to zero gives you two equations for a and b. The partial with respct to a is simpler and yields an equation in a and b that involves only that averages of x and y. The partial with respect to b is a bit more complicated, involving the dot product of x and y and the dot product of x with itself. With the corrected A matrix you should be able to show that those equations are equivalent to the matrix equation you posted.

How do I take the partial derivative of that sum-formula?

Sincerley

Fred

 

[OlderDan]

How do I take the partial derivative of that sum-formula?

Sincerley

Fred

When you take the partial derivative with respect to a, everything else is constant. You know that the derivative of a sum of functions is the sum of the derivatives of the functions. In other words, the derivative "moves inside" the sum. You will be taking the derivative of every term in the sum and you will be left with a sum of terms where each term is a sum of other terms. You will want to separate those into separate sums. For the derivative wrt a, there will be three sums, a sum of all the y, a sum of all the x times a constant, and the sum of a constant. You can reduce that equation to an equation involving the two constants (a and b) and the averages of x and y. If you get that one, I think you will see how to do the derivative wrt b, but it is a bit more complicated.

Give it a try and post your result. If you don't get it right, I or someone else will post it.

 

[Mathman23]

Hello Dan,

If I differentiate $$E(a,b)$$ first with respect to a and then to b.

I got the following two equations.

$$\begin{array}{cc} \frac{\partial E}{ \partial a} = 2(ax_{j} - y_{j} +b_{j}) x_{j} \ \ \mathrm{and} \ \ \frac{\partial E}{ \partial b} = 2(b_{j} + ax_{j} -y_{j}) \end{array}$$

Is this what You mean?

Again thank You very much for Your help.

Sincerley and Best Regards
Fred

 

[OlderDan]

Hello Dan,

If I differentiate $$E(a,b)$$ first with respect to a and then to b.

I got the following two equations.

$$\begin{array}{cc} \frac{\partial E}{ \partial a} = 2(ax_{j} - y_{j} +b_{j}) x_{j} \ \ \mathrm{and} \ \ \frac{\partial E}{ \partial b} = 2(b_{j} + ax_{j} -y_{j}) \end{array}$$

Is this what You mean?

Again thank You very much for Your help.

Sincerley and Best Regards
Fred

The derivatives are still sums of terms

$$E(a,b) = \sum_{j=1} ^{N} (y_{j} - (a + bx_{j}))^2$$

$$E(a,b) = \sum_{j=1} ^{N} (y_{j} - a - bx_{j})^2$$

$$\frac{\partial E}{ \partial a} = \sum_{j=1} ^{N} 2(y_{j} - a - bx_{j})(-1) = -2\sum_{j=1} ^{N} y_{j} + 2a\sum_{j=1} ^{N} 1 + 2b\sum_{j=1} ^{N} x_{j} = 0$$

$$\frac{\partial E}{ \partial b} = \sum_{j=1} ^{N} 2(y_{j} - a - bx_{j})(-x_{j}) = -2\sum_{j=1} ^{N} x_{j}y_{j} + 2a\sum_{j=1} ^{N} x_{j} + 2b\sum_{j=1} ^{N} x_{j}^2 = 0$$

Divide each of these results by -2N and you should start to identify some terms as simple averages. One will reduce to the parameter a. Other terms will involve the dot product of x and y, and the dot product of x with itself. The resulting equations can be put into the matrix form you were given.

 

[Mathman23]

Hello Dan and many thanks for Your answer,

I divide the three terms in first and second with $$-2N$$
and get the following results:


$$-2 \sum _{j=1} ^{N} y_{j} \rightarrow \frac{-N y}{2} \rightarrow \frac{-N}{2} \sum_{j=1} ^ {n} y$$

$$-2a \sum _{j=1} ^{N} 1 \rightarrow a \rightarrow a \sum_{j=1} ^{N} 1$$

$$-2b \sum _{j=1} ^{N} x_{j} \rightarrow bx \rightarrow b \sum_{j=1} ^{N} x$$

The three sums in the second part yields:

$$-2 \sum _{j=1} ^{N} x_{j} y_{j} \rightarrow xy \rightarrow \sum_{j=1} ^{N} xy$$

$$-2a \sum _{j=1} ^{N} x_{j} \rightarrow ax \rightarrow a \sum_{j=1} ^{N} x$$

$$-2b \sum _{j=1} ^{N} {x_{j}}^2 \rightarrow b x^2 \rightarrow b \sum_{j=1} ^{N} x^2$$

If I insert the above in an array A I get the following:

$$A= \left[ \begin{array}{cc} 1 & x \\ x & x^2 \end{array} \right ]$$

Which can also we written as:

$$A \left [ \begin{array}{c} a \\ b \end{array} \right ] = ??$$

But how do I conclude that the product of these two arrays equal the array

$$C = \left[ \begin{array}{c} y \\ xy \end{array} \right ]$$ ??

Sincerely and Best regards

/Fred

$$\frac{\partial E}{ \partial b} = \sum_{j=1} ^{N} 2(y_{j} - a - bx_{j})(-x_{j}) = -2\sum_{j=1} ^{N} x_{j}y_{j} -2a\sum_{j=1} ^{N} x_{j} -2b\sum_{j=1} ^{N} x_{j}^2 = 0$$

Divide each of these results by -2N and you should start to identify some terms as simple averages. One will reduce to the parameter a. Other terms will involve the dot product of x and y, and the dot product of x with itself. The resulting equations can be put into the matrix form you were given.

 

[OlderDan]

Hello Dan and many thanks for Your answer,

I divide the three terms in first and second with $$-2N$$
and get the following results:


$$-2 \sum _{j=1} ^{N} y_{j} \rightarrow \frac{-N y}{2} \rightarrow \frac{-N}{2} \sum_{j=1} ^ {n} y$$

You have not divided by -2N. I see I made a foolish sign error in my previous post. I will go back and fix my error, but here are the correct equations

$$\frac{\partial E}{ \partial a} = \sum_{j=1} ^{N} 2(y_{j} - a - bx_{j})(-1) = -2\sum_{j=1} ^{N} y_{j} +2a\sum_{j=1} ^{N} 1 +2b\sum_{j=1} ^{N} x_{j} = 0$$

$$\frac{\partial E}{ \partial b} = \sum_{j=1} ^{N} 2(y_{j} - a - bx_{j})(-x_{j}) = -2\sum_{j=1} ^{N} x_{j}y_{j} + 2a\sum_{j=1} ^{N} x_{j} + 2b\sum_{j=1} ^{N} x_{j}^2 = 0$$

$$-2\sum_{j=1} ^{N} y_{j} + 2a\sum_{j=1} ^{N} 1 + 2b\sum_{j=1} ^{N} x_{j} = 0 \rightarrow \frac{1}{N} \sum_{j=1} ^{N} y_{j} - \frac{a}{N}\sum_{j=1} ^{N} 1 - \frac{b}{N} \sum_{j=1} ^{N} x_{j} = \overline{y} - a - b \overline{x} = 0$$

See if you can fix the second equation and take it from there. Keep in mind that a sum over a product of corresponding vector components is a dot product.

 

[Mathman23]

You have not divided by -2N. I see I made a foolish sign error in my previous post. I will go back and fix my error, but here are the correct equations

$$\frac{\partial E}{ \partial a} = \sum_{j=1} ^{N} 2(y_{j} - a - bx_{j})(-1) = -2\sum_{j=1} ^{N} y_{j} +2a\sum_{j=1} ^{N} 1 +2b\sum_{j=1} ^{N} x_{j} = 0$$

$$\frac{\partial E}{ \partial b} = \sum_{j=1} ^{N} 2(y_{j} - a - bx_{j})(-x_{j}) = -2\sum_{j=1} ^{N} x_{j}y_{j} + 2a\sum_{j=1} ^{N} x_{j} + 2b\sum_{j=1} ^{N} x_{j}^2 = 0$$

$$-2\sum_{j=1} ^{N} y_{j} + 2a\sum_{j=1} ^{N} 1 + 2b\sum_{j=1} ^{N} x_{j} = 0 \rightarrow \frac{1}{N} \sum_{j=1} ^{N} y_{j} - \frac{a}{N}\sum_{j=1} ^{N} 1 - \frac{b}{N} \sum_{j=1} ^{N} x_{j} = \overline{y} - a - b \overline{x} = 0$$

See if you can fix the second equation and take it from there. Keep in mind that a sum over a product of corresponding vector components is a dot product.

HI Dan and many for thanks for Your answers,

In the second equation I get the following:

$$-2\sum_{j=1} ^{N} x_{j}y_{j} + 2a\sum_{j=1} ^{N} x_{j} + 2b\sum_{j=1} ^{N} x_{j}^2 = 0 \rightarrow \sum_{j=1} ^{N} \frac{x_{j} y_{j}}{N} - \frac{a}{N} \sum_{j=1} ^{N} x_{j} - b \sum_{j=1} ^{N} \frac{x_{j}^2}{N} = (\overline{x} \cdot \overline{y}) - a \overline{x} - b(\overline{x} \cdot \overline{x}) = 0$$

If I rewrite Your result combined with mine I got the following set of equations:

$$\begin{array}{ccc} a + b \cdot \overline{x} & = & \overline{y} \\ a \cdot \overline{x} + b \cdot (\overline{x} \cdot \overline{x}) & = & \overline{x} \cdot \overline{y} \end{array}$$

This can also be written as the inhomogeneous linear equation present in the original problem:

$$\left[ {\begin{array}{*{20}c} 1 & {\overline x } \\ {\overline x } & {\frac{{ x \cdot x }}{N}} \\\end{array}} \right] \cdot \left [ \begin{array}{c} a \\ b \end{array} \right ] = \left [ \begin{array}{c} \overline{y} \\ \frac{x \cdot y}{N}\end{array} \right ]$$

One question then remains, how does one prove the claim that solving the above system let's one obtain the minimum value for E(a,b) ??

Sincerely and Best Regards,

Fred

 

[OlderDan]

One question then remains, how does one prove the claim that solving the above system let's one obtain the minimum value for E(a,b) ??

Fred

In the calculus of one variable you learned that maxima and minima are found by setting the first derivative of a function to zero. In multivariate calculus, the analogous thing is setting the partial derivatives equal to zero. For a function of two variables you have

$$df(a,b) = \frac{\partial f(a,b)}{\partial a}da + \frac{\partial f(a,b)}{\partial b}db$$

If you change a slightly without changing b, the change in f(a,b) is just the first term on the right. When you change b without changing a, the change in f(a,b) is just the second term. When both those terms are zero, because the partial derivatives are both zero, an arbitrary small change in both a and b gives no change in f(a,b). There are three possibilities. If both partials represent minima, the function is a minimum. If both are maxima, the function is at a maximum. If one is a minimum, and the other is a maximum it is referred to as a "saddle point" because the graph of f(a,b) has the shape of a saddle. In this problem, the point is a minimum. You could prove that by taking second derivatives. I'll leave that for another problem.

 

[Mathman23]

Hi

I remember that from my calculas course. Thank You :-)

I was informed today that I also need to calculate the determinant of A.

$$A= \left[ {\begin{array}{*{20}c} 1 & {\overline x } \\ {\overline x } & {\frac{{ x \cdot x }}{N}} \\\end{array}} \right]$$

I get $$det(A) = \frac{x \cdot x}{N}$$

But if that's correct, for which values is A then invertible ?

Sincerley and Best Regards.

Fred

In the calculus of one variable you learned that maxima and minima are found by setting the first derivative of a function to zero. In multivariate calculus, the analogous thing is setting the partial derivatives equal to zero. For a function of two variables you have

$$df(a,b) = \frac{\partial f(a,b)}{\partial a}da + \frac{\partial f(a,b)}{\partial b}db$$

If you change a slightly without changing b, the change in f(a,b) is just the first term on the right. When you change b without changing a, the change in f(a,b) is just the second term. When both those terms are zero, because the partial derivatives are both zero, an arbitrary small change in both a and b gives no change in f(a,b). There are three possibilities. If both partials represent minima, the function is a minimum. If both are maxima, the function is at a maximum. If one is a minimum, and the other is a maximum it is referred to as a "saddle point" because the graph of f(a,b) has the shape of a saddle. In this problem, the point is a minimum. You could prove that by taking second derivatives. I'll leave that for another problem.

 

[OlderDan]

Hi

I remember that from my calculas course. Thank You :-)

I was informed today that I also need to calculate the determinant of A.

$$A= \left[ {\begin{array}{*{20}c} 1 & {\overline x } \\ {\overline x } & {\frac{{ x \cdot x }}{N}} \\\end{array}} \right]$$

I get $$det(A) = \frac{x \cdot x}{N}$$

But if that's correct, for which values is A then invertible ?

Sincerley and Best Regards.

Fred

You are missing a term in the determinant.

 

[Mathman23]

Hi

I have re-calculated the determinant for A.

I get the following result

$$det(A) = (\frac{1}{N} -1) x^2 \rightarrow (\frac{1}{N} -1) \sum_{j=1} ^{N} (-x_{j}^2)$$

To answer my second question I need to set the above equation to equal zero, in order to determin for which values the matrix A is invertible.

$$det(A) = (\frac{1}{N} -1) x^2 = 0$$

But for which variable do I need to solve the above equation ?

Sincerely and Best Regards

Fred

You are missing a term in the determinant.

 

[OlderDan]

Hi

I have re-calculated the determinant for A.

I get the following result

$$det(A) = (\frac{1}{N} -1) x^2 \rightarrow (\frac{1}{N} -1) \sum_{j=1} ^{N} (-x_{j}^2)$$

To answer my second question I need to set the above equation to equal zero, in order to determin for which values the matrix A is invertible.

$$det(A) = (\frac{1}{N} -1) x^2 = 0$$

But for which variable do I need to solve the above equation ?

Sincerely and Best Regards

Fred

That is still not correct

$$\overline{x} = \frac{1}{N} \sum_{j=1} ^N x_{j}$$

The second term in the determinant is

$$-\overline{x}^2 = -\left[\frac{1}{N} \sum_{j=1} ^N x_{j}\right]^2 = -\frac{1}{N^2}\left[ \sum_{j=1} ^N x_{j}\right]^2$$

The square of the sum is not the same thing as the sum of the squares

$$\frac{x \cdot x}{N} = \frac{1}{N} \sum_{j=1} ^N x_{j}^2$$

 

[Mathman23]

That is still not correct

$$\overline{x} = \frac{1}{N} \sum_{j=1} ^N x_{j}$$

The second term in the determinant is

$$-\overline{x}^2 = -\left[\frac{1}{N} \sum_{j=1} ^N x_{j}\right]^2 = -\frac{1}{N^2}\left[ \sum_{j=1} ^N x_{j}\right]^2$$

The square of the sum is not the same thing as the sum of the squares

$$\frac{x \cdot x}{N} = \frac{1}{N} \sum_{j=1} ^N x_{j}^2$$

For which values is A invertible then ?

Hi Dan and again many thanks for Your answer :)

Correct me if I'm wrong this is the determinant for A??

$$-\overline{x}^2 = -\left[\frac{1}{N} \sum_{j=1} ^N x_{j}\right]^2 = -\frac{1}{N^2}\left[ \sum_{j=1} ^N x_{j}\right]^2$$

Sincerely and
Fred

 

[OlderDan]

For which values is A invertible then ?

Hi Dan and again many thanks for Your answer :)

Correct me if I'm wrong this is the determinant for A??

$$-\overline{x}^2 = -\left[\frac{1}{N} \sum_{j=1} ^N x_{j}\right]^2 = -\frac{1}{N^2}\left[ \sum_{j=1} ^N x_{j}\right]^2$$

Sincerely and
Fred

$$A= \left[ {\begin{array}{*{20}c} 1 & {\overline x } \\ {\overline x } & {\frac{{ x \cdot x }}{N}} \\\end{array}} \right]$$

The determinant of a 2 by 2 matirx is the difference of the cross products.

$$Det(A) = \frac{x \cdot x}{N} -\overline{x}^2 = \frac{1}{N} \sum_{j=1} ^N x_{j}^2 - \frac{1}{N^2}\left[ \sum_{j=1} ^N x_{j}\right]^2$$

In order to have an inverse, the determinant must not be zero. The inverse of a 2 by 2 is shown here

http://mathworld.wolfram.com/MatrixInverse.html

Operating from the left with the inverse on

$$\left[ {\begin{array}{*{20}c} 1 & {\overline x } \\ {\overline x } & {\frac{{ x \cdot x }}{N}} \\\end{array}} \right] \cdot \left [ \begin{array}{c} a \\ b \end{array} \right ] = \left [ \begin{array}{c} \overline{y} \\ \frac{x \cdot y}{N}\end{array} \right ]$$

Gives you

$$\left [ \begin{array}{c} a \\ b \end{array} \right ] = \left[ {\begin{array}{*{20}c} 1 & {\overline x } \\ {\overline x } & {\frac{{ x \cdot x }}{N}} \\\end{array}} \right]^{-1} \cdot \left [ \begin{array}{c} \overline{y} \\ \frac{x \cdot y}{N}\end{array} \right ]$$

Which is equivalent to a pair of equations for the coefficients a and b. When you invert the matrix, you are almost done.

 

[Mathman23]

Hi

Then by using this formula I calculate the inverse of A.


$$A^{-1} = (x^2 - \frac{1}{x^2}) \left[ {\begin{array}{*{20}c} {\frac{{ x \cdot x }}{N}} & -{\overline x } \\ -{\overline x } & 1\\\end{array}} \right] = \left[ {\begin{array}{*{20}c} {\frac{{( x \cdot x \cdot x \cdot x) -1 }}{N}} & {\frac{{-(( x \cdot x \cdot x \cdot x) -1) }}{x}} } \\ {\frac{{-(( x \cdot x \cdot x \cdot x) -1 )}}{x}} & x \cdot x - \frac{1}{x \cdot x}\\\end{array}} \right]$$

Is this correct?

Best Regards

Fred

p.s. Thank You again Dan for all Your answers :-)

$$A= \left[ {\begin{array}{*{20}c} 1 & {\overline x } \\ {\overline x } & {\frac{{ x \cdot x }}{N}} \\\end{array}} \right]$$

The determinant of a 2 by 2 matirx is the difference of the cross products.

$$Det(A) = \frac{x \cdot x}{N} -\overline{x}^2 = \frac{1}{N} \sum_{j=1} ^N x_{j}^2 - \frac{1}{N^2}\left[ \sum_{j=1} ^N x_{j}\right]^2$$

In order to have an inverse, the determinant must not be zero. The inverse of a 2 by 2 is shown here

http://mathworld.wolfram.com/MatrixInverse.html

Operating from the left with the inverse on

$$\left[ {\begin{array}{*{20}c} 1 & {\overline x } \\ {\overline x } & {\frac{{ x \cdot x }}{N}} \\\end{array}} \right] \cdot \left [ \begin{array}{c} a \\ b \end{array} \right ] = \left [ \begin{array}{c} \overline{y} \\ \frac{x \cdot y}{N}\end{array} \right ]$$

Gives you

$$\left [ \begin{array}{c} a \\ b \end{array} \right ] = \left[ {\begin{array}{*{20}c} 1 & {\overline x } \\ {\overline x } & {\frac{{ x \cdot x }}{N}} \\\end{array}} \right]^{-1} \cdot \left [ \begin{array}{c} \overline{y} \\ \frac{x \cdot y}{N}\end{array} \right ]$$

Which is equivalent to a pair of equations for the coefficients a and b. When you invert the matrix, you are almost done.

 

[OlderDan]

Hi

Then by using this formula I calculate the inverse of A.


$$A^{-1} = (x^2 - \frac{1}{x^2}) \left[ {\begin{array}{*{20}c} {\frac{{ x \cdot x }}{N}} & -{\overline x } \\ -{\overline x } & 1\\\end{array}} \right] = \left[ {\begin{array}{*{20}c} {\frac{{( x \cdot x \cdot x \cdot x) -1 }}{N}} & {\frac{{-(( x \cdot x \cdot x \cdot x) -1) }}{x}} } \\ {\frac{{-(( x \cdot x \cdot x \cdot x) -1 )}}{x}} & x \cdot x - \frac{1}{x \cdot x}\\\end{array}} \right]$$

Is this correct?

Best Regards

Fred

p.s. Thank You again Dan for all Your answers :-)

Your factor in front of the matrix is not correct. You really need to be careful with the notation. Since the dot product of a vector with itself is a scalar, or the square of the norm (length) of the vector there is a shorthand notation often used to write

$$x \cdot x = x^2$$

That is a bit dangerous to do in this problem, because in addition to $$x^2$$, there is also an $$\overline{x}^2$$, which is a totally different thing. I suggest you keep the dot product notation everywhere you have a dot product in your expressions. In that form

$$Det(A) = \frac{x \cdot x}{N} - \overline{x}^2 = \frac{x \cdot x -N \overline{x}^2}{N}$$

$$\frac{1}{Det(A)} = \frac{N}{x \cdot x - N\overline{x}^2}}$$

This is the factor you need to multiply the matrix. The matrix itself looks fine.

 

[Mathman23]

Hi Dan and thanks,

This should be right inverse matrix(A) ??

$$A^{-1} = \left[ \begin{array}{*{40} cc} \frac{N (x \cdot x)}{(N(x \cdot x) -(N^2 (\overline{x}^2))} & \frac{-\overline{x }N}{x \cdot x - N \overline{x} ^2} \\ \frac{-\overline{x} N}{x \cdot x - N \overline{x}^2} & \frac{N}{x \cdot x - N \overline{x}^2} \end{array} \right]$$

Moving one to the next part of the problem: For which values is Matrix A invertible??

$$\left[ \begin{array}{c} a & b \end{array} \right] = \left[ \begin{array}{*{40} cc} \frac{N (x \cdot x)}{(N(x \cdot x) -(N^2 (\overline{x}^2))} & \frac{-\overline{x }N}{x \cdot x - N \overline{x} ^2} \\ \frac{-\overline{x} N}{x \cdot x - N \overline{x}^2} & \frac{N}{x \cdot x - N \overline{x}^2} \end{array} \right] \left[ \begin{array}{c} \overline{y} \\ \frac{x \cdot y}{N} \end{array} \right]= \left[ \begin{array}{c} \frac{N (x \cdot x) \cdot \overline{y}}{(N(x \cdot x) -(N^2 (\overline{x}^2))} + \frac{-\overline{x }N \overline{y}}{x \cdot x - N \overline{x} ^2} \\ \frac{-\overline{x }N(x \cdot y)}{N(x \cdot x) - N^2 \overline{x} ^2} + \frac{N (x \cdot y)}{N(x \cdot x) - N^2 \overline{x}^2} \end{array} \right]$$

Is it possible to write out $$\left[ \begin{array}{c} a & b \end{array} \right]$$ to a more compact form ??

Sincerely and Best Regards

Fred

Your factor in front of the matrix is not correct. You really need to be careful with the notation. Since the dot product of a vector with itself is a scalar, or the square of the norm (length) of the vector there is a shorthand notation often used to write

$$x \cdot x = x^2$$

That is a bit dangerous to do in this problem, because in addition to $$x^2$$, there is also an $$\overline{x}^2$$, which is a totally different thing. I suggest you keep the dot product notation everywhere you have a dot product in your expressions. In that form

$$Det(A) = \frac{x \cdot x}{N} - \overline{x}^2 = \frac{x \cdot x -N \overline{x}^2}{N}$$

$$\frac{1}{Det(A)} = \frac{N}{x \cdot x - N\overline{x}^2}}$$

This is the factor you need to multiply the matrix. The matrix itself looks fine.

 

[OlderDan]

Hi Dan and thanks,

This should be right inverse matrix(A) ??

$$A^{-1} = \left[ \begin{array}{*{40} cc} \frac{N (x \cdot x)}{(N(x \cdot x) -(N^2 (\overline{x}^2))} & \frac{-\overline{x }N}{x \cdot x - N \overline{x} ^2} \\ \frac{-\overline{x} N}{x \cdot x - N \overline{x}^2} & \frac{N}{x \cdot x - N \overline{x}^2} \end{array} \right]$$

Moving one to the next part of the problem: For which values is Matrix A invertible??

$$\left[ \begin{array}{c} a & b \end{array} \right] = \left[ \begin{array}{*{40} cc} \frac{N (x \cdot x)}{(N(x \cdot x) -(N^2 (\overline{x}^2))} & \frac{-\overline{x }N}{x \cdot x - N \overline{x} ^2} \\ \frac{-\overline{x} N}{x \cdot x - N \overline{x}^2} & \frac{N}{x \cdot x - N \overline{x}^2} \end{array} \right] \left[ \begin{array}{c} \overline{y} \\ \frac{x \cdot y}{N} \end{array} \right]= \left[ \begin{array}{c} \frac{N (x \cdot x) \cdot \overline{y}}{(N(x \cdot x) -(N^2 (\overline{x}^2))} + \frac{-\overline{x }N \overline{y}}{x \cdot x - N \overline{x} ^2} \\ \frac{-\overline{x }N(x \cdot y)}{N(x \cdot x) - N^2 \overline{x} ^2} + \frac{N (x \cdot y)}{N(x \cdot x) - N^2 \overline{x}^2} \end{array} \right]$$

Is it possible to write out $$\left[ \begin{array}{c} a & b \end{array} \right]$$ to a more compact form ??

Sincerely and Best Regards

Fred

You can reduce the upper left term in your inverse by dividing numeratior and denominator by N. Other than that it looks fine.
$$A^{-1} = \frac{N}{x \cdot x - N\overline{x}^2}} \left[ {\begin{array}{*{20}c} {\frac{{ x \cdot x }}{N}} & -{\overline x } \\ -{\overline x } & 1\\\end{array}} \right] = \left[ \begin{array}{*{40} cc} \frac{ x \cdot x}{x \cdot x -N \overline{x}^2} & \frac{-\overline{x }N}{x \cdot x - N \overline{x} ^2} \\ \frac{-\overline{x} N}{x \cdot x - N \overline{x}^2} & \frac{N}{x \cdot x - N \overline{x}^2} \end{array} \right]$$

Your matrix multiplication is not fine however. You do not have the terms matched up properly. Review your matrix multiplication rules and give this another try. The expressions for a and b will not be very "simple" in any case, but you could help yourself by leaving the 1/det(A) term out in front of the matrix product, since it is a common factor in every term. It will also be a common factor in a and b.

 

[Mathman23]

Hi

I try again.


Is its correct now?

$$\left[ \begin{array}{c} a \\ b \end{array} \right] = \left[ \begin{array}{*{40} cc} \frac{ x \cdot x}{x \cdot x -N \overline{x}^2} & \frac{-\overline{x }N}{x \cdot x - N \overline{x} ^2} \\ \frac{-\overline{x} N}{x \cdot x - N \overline{x}^2} & \frac{N}{x \cdot x - N \overline{x}^2} \end{array} \right] \left[ \begin{array}{c} \overline{y} \\ \frac{x \cdot y}{N} \end{array} \right] = \left[ \begin{array}{c} \frac{( x \cdot x) \overline{y}}{x \cdot x -N \overline{x}^2} + \frac{-\overline{x }N (x \cdot y)}{(x \cdot x)N - N^2 \overline{x} ^2} \\ \frac{-\overline{x }N(\overline{y})}{x \cdot x - N \overline{x} ^2} + \frac{N (x \cdot y)}{N(x \cdot x) - N^2 \overline{x}^2} \end{array} \right]$$

Sincerely and Best Regards

Fred

You can reduce the upper left term in your inverse by dividing numeratior and denominator by N. Other than that it looks fine.
$$A^{-1} = \frac{N}{x \cdot x - N\overline{x}^2}} \left[ {\begin{array}{*{20}c} {\frac{{ x \cdot x }}{N}} & -{\overline x } \\ -{\overline x } & 1\\\end{array}} \right] = \left[ \begin{array}{*{40} cc} \frac{ x \cdot x}{x \cdot x -N \overline{x}^2} & \frac{-\overline{x }N}{x \cdot x - N \overline{x} ^2} \\ \frac{-\overline{x} N}{x \cdot x - N \overline{x}^2} & \frac{N}{x \cdot x - N \overline{x}^2}\end{array} \right]$$

Your matrix multiplication is not fine however. You do not have the terms matched up properly. Review your matrix multiplication rules and give this another try. The expressions for a and b will not be very "simple" in any case, but you could help yourself by leaving the 1/det(A) term out in front of the matrix product, since it is a common factor in every term. It will also be a common factor in a and b.

 

[OlderDan]

Hi

I try again.


Is its correct now?

$$\left[ \begin{array}{c} a \\ b \end{array} \right] = \left[ \begin{array}{*{40} cc} \frac{ x \cdot x}{x \cdot x -N \overline{x}^2} & \frac{-\overline{x }N}{x \cdot x - N \overline{x} ^2} \\ \frac{-\overline{x} N}{x \cdot x - N \overline{x}^2} & \frac{N}{x \cdot x - N \overline{x}^2} \end{array} \right] \left[ \begin{array}{c} \overline{y} \\ \frac{x \cdot y}{N} \end{array} \right] = \left[ \begin{array}{c} \frac{( x \cdot x) \overline{y}}{x \cdot x -N \overline{x}^2} + \frac{-\overline{x }N (x \cdot y)}{(x \cdot x)N - N^2 \overline{x} ^2} \\ \frac{-\overline{x }N(\overline{y})}{x \cdot x - N \overline{x} ^2} + \frac{N (x \cdot y)}{N(x \cdot x) - N^2 \overline{x}^2} \end{array} \right]$$

Sincerely and Best Regards

Fred

It looks correct, but it should be simplified. You again have factors of N that you can cancel in the second term of a and the second term of b. You should be combining the two terms in each case to write them over common denominators. The common denominator will be there if you get rid of the unnecessary N factors. Not only that, it will be the same denominator in b that you have in a. That would be obvious if you left the 1/Det(A) term out in front, and it would save you a lot of writing.