Building a least square problem design matrix

In summary: He has not mentioned it, but then again, he has not mentioned a lot of other things he might know.I don't know whether papasmurff is taking a course that might cover that. He has not mentioned it, but then again, he has not mentioned a lot of other things he might know.In summary, the conversation discusses fitting an arbitrary function f(t) with a polynomial of degree n-1 using the values of the function at an arbitrary set of points. The question posed is how to construct the least squares problem Ax=b, specifically how to construct the matrix A and vector b in terms of f(t) and z. The conversation also mentions the use of the vandermonde matrix in this problem
  • #1
papasmurfff
3
0

Homework Statement


Suppose we want to to fit an arbitrary function f(t) with a polynomial of
degree n - 1 using the values of the function in an arbitrary set of points z = [z1; z2; : : : ; zm].

how would do i construct the least squares problem Ax=b. in other words, how would i construct the matrix A and b in terms of f(t) and z ?


Homework Equations





The Attempt at a Solution



vandermonde's matrix
 
Physics news on Phys.org
  • #2
papasmurfff said:

Homework Statement


Suppose we want to to fit an arbitrary function f(t) with a polynomial of
degree n - 1 using the values of the function in an arbitrary set of points z = [z1; z2; : : : ; zm].

how would do i construct the least squares problem Ax=b. in other words, how would i construct the matrix A and b in terms of f(t) and z ?


Homework Equations





The Attempt at a Solution



vandermonde's matrix

Suppose, for example, you want to fit a quadratic y = a + b*z = c*z^2 to data (z1,y1), (z2,y2),..., (zn,yn). You thus want to fit a linear form Y = c0*X0 + c1*X1 + c2*X2 to the data, where x0[j] = 1, x1[j] = zj and x2[j] = zj^2 for j = 1,...,n. So, you want to minimize S = sum_{j=1..n}(yj - c0*x0[j]-c1*x1[j]-c2*x2[j])^2. Differentiate wrt c0, c1 and c2 and equate the derivatives to zero. You will get a system of linear equations for c0, c1 c2. If you actually sit down and do this you will be answering your own question.

RGV
 
  • #3
thank you RGV. if i put it in the form of vandermonde matrix would that work also?
 
  • #4
papasmurfff said:
thank you RGV. if i put it in the form of vandermonde matrix would that work also?

The vandermonde matrix is the matrix of columns X0[j] = 1,j=1..,n, X1[j = zj], i=1..n and X2[j] = zj^2, j=1..n. It is not the A-matrix of the least-squares method, although you build A starting from the vandermonde matrix. Like I said: write things out in detail to see what is happening.

RGV
 
  • #5
Ray,

Just a thought. In your example, suppose we let

[tex] A = \begin{pmatrix} 1 & z_1 & z_1^2 \\ 1 & z_2 & z_2^2 \\ . & . & . \\ 1 & z_n & z_n^2 \end{pmatrix} [/tex]

[tex] x = \begin{pmatrix} c_0 \\ c_1 \\ c_2 \end{pmatrix} [/tex]

[tex] b = \begin{pmatrix} y_1 & y_2 & ... & y_n \end{pmatrix} [/tex]

And get a least squares solution to the overdetermined system of equations [itex] Ax = b [/itex] by using the Moore-Penrose inverse of A. Have we solved an equivalent problem?

I don't know whether papasmurff is taking a course that might cover that.
 

1. What is a least square problem design matrix?

A least square problem design matrix is a matrix used in linear regression to represent the relationship between the independent variables and the dependent variable. It is used to calculate the coefficients that best fit the data and minimize the sum of squared errors.

2. How do you construct a least square problem design matrix?

To construct a least square problem design matrix, you need to first determine the number of independent variables and the number of data points. Then, you need to assign each independent variable to a column and each data point to a row. Finally, you need to add a column of 1s to represent the intercept term.

3. What is the purpose of a least square problem design matrix?

The purpose of a least square problem design matrix is to find the best fit line or curve for a given set of data points. It allows us to estimate the relationship between the independent variables and the dependent variable and make predictions based on this relationship.

4. How does the least square problem design matrix help with linear regression?

The least square problem design matrix is used in linear regression to calculate the coefficients that minimize the sum of squared errors. These coefficients are used to create a linear model that represents the relationship between the independent variables and the dependent variable.

5. What are some common mistakes when building a least square problem design matrix?

Some common mistakes when building a least square problem design matrix include not including the intercept term or adding too many independent variables. It is also important to make sure that the data is correctly assigned to the rows and columns and that there are no missing values or errors in the data.

Similar threads

  • Calculus and Beyond Homework Help
Replies
1
Views
1K
  • Calculus and Beyond Homework Help
Replies
8
Views
2K
  • Calculus and Beyond Homework Help
Replies
3
Views
1K
  • Calculus and Beyond Homework Help
Replies
3
Views
779
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
1K
  • Calculus and Beyond Homework Help
Replies
1
Views
465
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
952
  • Calculus and Beyond Homework Help
Replies
3
Views
1K
  • Calculus and Beyond Homework Help
Replies
4
Views
1K
  • Calculus and Beyond Homework Help
Replies
12
Views
3K
Back
Top