Building a least square problem design matrix

[papasmurfff]

Homework Statement

Suppose we want to to fit an arbitrary function f(t) with a polynomial of
degree n - 1 using the values of the function in an arbitrary set of points z = [z1; z2; : : : ; zm].

how would do i construct the least squares problem Ax=b. in other words, how would i construct the matrix A and b in terms of f(t) and z ?

Homework Equations

The Attempt at a Solution

vandermonde's matrix

 

[Ray Vickson]

Homework Statement

Suppose we want to to fit an arbitrary function f(t) with a polynomial of
degree n - 1 using the values of the function in an arbitrary set of points z = [z1; z2; : : : ; zm].

how would do i construct the least squares problem Ax=b. in other words, how would i construct the matrix A and b in terms of f(t) and z ?

Homework Equations

The Attempt at a Solution

vandermonde's matrix

Suppose, for example, you want to fit a quadratic y = a + b*z = c*z^2 to data (z1,y1), (z2,y2),..., (zn,yn). You thus want to fit a linear form Y = c0*X0 + c1*X1 + c2*X2 to the data, where x0[j] = 1, x1[j] = zj and x2[j] = zj^2 for j = 1,...,n. So, you want to minimize S = sum_{j=1..n}(yj - c0*x0[j]-c1*x1[j]-c2*x2[j])^2. Differentiate wrt c0, c1 and c2 and equate the derivatives to zero. You will get a system of linear equations for c0, c1 c2. If you actually sit down and do this you will be answering your own question.

RGV

 

[papasmurfff]

thank you RGV. if i put it in the form of vandermonde matrix would that work also?

 

[Ray Vickson]

thank you RGV. if i put it in the form of vandermonde matrix would that work also?

The vandermonde matrix is the matrix of columns X0[j] = 1,j=1..,n, X1[j = zj], i=1..n and X2[j] = zj^2, j=1..n. It is not the A-matrix of the least-squares method, although you build A starting from the vandermonde matrix. Like I said: write things out in detail to see what is happening.

RGV

 

[Stephen Tashi]

Ray,

Just a thought. In your example, suppose we let

$$A = \begin{pmatrix} 1 & z_1 & z_1^2 \\ 1 & z_2 & z_2^2 \\ . & . & . \\ 1 & z_n & z_n^2 \end{pmatrix}$$

$$x = \begin{pmatrix} c_0 \\ c_1 \\ c_2 \end{pmatrix}$$

$$b = \begin{pmatrix} y_1 & y_2 & ... & y_n \end{pmatrix}$$

And get a least squares solution to the overdetermined system of equations $Ax = b$ by using the Moore-Penrose inverse of A. Have we solved an equivalent problem?

I don't know whether papasmurff is taking a course that might cover that.