Least squares Definition and 158 Threads

Common method of fitting the experimental data to the model is using the least squares method. The goal is to find parameters such that sum of squares of differences between experimental and model values (objective function) are minimized. Objective function is commonly differentiable with...

I started by converting the LSM from sum to integral form: $$f(x_c) = \sum_i[S(x_i)-F(x_i,;a,b,...)]^2 to f(x_c) = \int( S(x) - F(x-x_c)^2 dx$$ Since we are not interested in the other parameters (like offset), I assumed that they are fitted correctly and thus ignored them, turning...

I'm posting to inquire about a possible typo in the given answer in the back of the book, or if maybe I did something wrong, because my answer does not agree with the one stated. So the exercise is about finding the least squares approximation. The norm is the ##L^2## norm and the corresponding...

Hi guys, I was learning machine learning and I found something a bit confusing. When I studied physics I saw the method of least squares to find the best parameters for the given data, in this case we assume we know the equation and we just minimize the error. So if it is a straight line model...

Hello, Regression analysis is about finding/estimating the coefficients for a particular function ##f## that would best fit the data. The function ##f## could be a straight line, an exponential, a power law, etc. The goal remains the same: finding the coefficients. If the data does not show a...

I'm computing the trajectory of a moving body and my net is composed by 5 stations. My observations are DTOA: difference in time of Arrival (they have been linearized). I am trying to use Least Squares with a linear model: Y = Ax + b, where Y are the observed measurements (DTOA), A the design...

Hi! Basically this is the exercise: Given the covariance of x and y is -12 and the variance of x is 6,5, using the least squares line of best fit connecting x and y yo estimate the value of x when y=15 x 2 5 9 7 9 10 7 y 25 17 11 10 8 7 13 any help would mean everything, I'm desperate :(

Consider the problem in the attached image. The difference between A and B is 0.0020(20). How does one use the least squares method, particularly in matrix form, to find the best value of the masses of A and B respectively, as well as the Q-value? Aren't more measurements needed for the masses...

I'm a little confused how to do this homework problem, I can't seem to obtain the correct answer. I took my vectors v1, v2, and v3 and set up a matrix. So I made my matrix: V = [ (6,0,0,1)T, (0,1,-1,0)T, (1,1,0,-6)T ] and then I had u = [ (0,5,4,0) T ]. I then went to solve using least...

Does anyone know the command or how to find the least squares solution of Ax=b on Ti-89 graphing calculator? I'm trying to check my answers on Ti-89 for those linear algebra problems.

Hello. I have listened to a great lecture, which gave helpful intuitive insight into correlation and regression (basic stuff). But there are formulas, which I cannot grasp intuitively and don't know their origin. To remember them I would like to understand what's happening in each part of the...

Summary: I need to Identify my linear model matrix using least squares . The aim is to approach an overdetermined system Matrix [A] by knowing pairs of [x] and [y] input data in the complex space. I need to do a linear model identification using least squared method. My model to identify is a...

I just went over analysis of a data set that was analzed using Linear Regression (OLS, I believe) and I saw Newton's method was used. Just curious, how is it used? I assume to minimize the cost function, but this function was not made explicit. Anyone know? Thanks.

Hey! :o I want to determine an approximation of a cubic polynomial that has at the points $$x_0=-2, \ x_1=-1, \ x_2=0 , \ x_3=3, \ x_4=3.5$$ the values $$y_0=-33, \ y_1=-20, \ y_2=-20.1, \ y_3=-4.3 , \ y_4=32.5$$ using the least squares method. So we are looking for a cubic polynomial $p(x)$...

I'm analysing some data and my task is to get a line that best fits the data, using least square I'm getting these dashed curves (red and blue) with low correlation factors. Is there another method that takes into consideration the amount of data placed into the direction of a line?

I am trying to use Weighted Least Squares with a linear model: Y = Xβ + ε, where Y are some observed measurements and β is a vector of estimates. For example, in this case β has two terms: intercept and slope. The weighted least squares solution, as shown here, involves a weight matrix, W...

Homework Statement We had a laboration for calculating ε_r in a parallel plate capacitor which we stuffed with plastic plates. All data we picked up was the area A, the distance d (and thus 1/d) and the capacitance C. We are now supposed to use the least squares-method to find ε_r, something we...

I've come across a problem with my least squares fits and I think someone else must have analyzed this, but I don't know where to find it. I have a converged least squares fit of my spectroscopic data. Unfortunately, the physical model, on which the fit is based, is mediocre. The deviations...

I have some experience with non-linear least squares curve fitting. For instance, if I want to fit a Gaussian curve to a set of data, I would use a non-linear least squares technique. A "model" matrix is implemented and combined with the observed data. The solution is found by applying well...

Hi, I'm trying to understand which mathematical actions I need to perform to be able to arrive at the solution shown in the uploaded picture. Thank you.

Suppose that I have an overdetermined equation system in matrix form: Ax = b Where x and b are column vectors, and A has the same number of rows as b, and x has less rows than both. The least-squares method could be used here to obtain the best possible approximative solution. Let's call this...

I am doing a nonlinear least squares estimation on a function of 14 variables (meaning that, to estimate ##y=f(x)##, I minimize ##\Sigma_i(y_i-(\hat x_i))^2## ). I do this using the quasi-Newton algorithm in MATLAB. This also gives the Hessian (matrix of second derivatives) at the minimizing...

Homework Statement Find the least squares approximation of cos^3(x) by a combination of sin(x) and cos(x) over the interval (0, 2pi) Homework EquationsThe Attempt at a Solution I know how to find a least squares approximation with vectors, but I don't even know how to start with a function...

Homework Statement I am designing a MIMO communication system, with input signal s, channel H and transform matrix T. The received signal is corrupted by noise. Homework Equations [/B] The received signal is r = Hs+n And then it is transformed (compressed) by: y = Tr And then its...

Homework Statement In R^3 with inner product calculate all the least square solutions, and choose the one with shorter length, of the system: x + y + z = 1 x + z = 0 y = 0 2. The attempt at a solution So I applied the formula A^T A x = A^T b with A as being the matrix with row 1 (1,1,1)...

Hi, I have a over constrained least squares problem. I also have the correct solution to the problem. Now I need to determine which of the vectors are contributing how much information that is close to the correct solution. I am sure there should be some methodology for this analysis, but I...

Homework Statement Hi! I've been interpolating a data set using Legendre polynomials and Newton's method and now I've been asked to, given a data set, approximate the function using the Least Squares Method with the following fitting function: ##g(r)=a+be^{cr}##, where ##a##, ##b## and ##c##...

Homework Statement Suppose that object 1 weighs \theta_1 and object two weights \theta_2. Then each object is weighed once and then together getting three observations y_1,y_2,y_3. The scale gives unbiased weights with normally distributed error (constant variance) Find the least square...

Homework Statement note a linear regression model with the response variable Y=(Y1..Yn) on a predictor variable X=(X1..Xn). the least squares estimates of the intercept and slope a(hat) and B(hat) are the values that minimize the function: (see attached image) and the problem reads on further...

My apologies for having to post in an image, my latex skills are not good enough for the question at hand :( a) There is no solution since the system has more unknowns than equations (the equations are equal giving 1=2 which does not make sense). b) I get a solution of \begin{bmatrix}1 \\1 \\...

It is my understanding that you can use linear least squares to fit a plethora of different functions (quadratic, cubic, quartic etc). The requirement of linearity applies to the coefficients (i.e B in (y-Bx)^2). It seems to me that I can find a solution such that a coefficient b_i^2=c_i, in...

Hello I have a doubt with the least squares fitting (linear fitting). The low-level statistics textbooks only take into account the statistical error of fitting, but not the error of the fitted points. How is the error of the fitted points taken into account, and included in the total error...

I am trying to solve a large least squares inversion (inverting data for the modeled sources), and find that my parameters describing 1 source are highly correlated with the parameters describing the second source. Can anyone recommend a technique or reference which discusses how to reduce the...

I have a large weighted least squares system with millions of observations. I can't store the least squares matrix in memory, so I only store the normal equations matrix: A^T W A where A is dense and n-by-p and W = diag(w_1,...,w_n). It takes a very long time to build this normal equations...

Homework Statement After turnig of magnetic-optic pit, cold cloud of atom 87 Rb is expanding. Size of cloud after time t, is given with relation: where, k_B is Boltzman constant, m mass of 87 Rb. Draw a plot, then use least squares method to find temperature T, and initial size of cloud...

Hey, PF I'm reading the following derivation of least squares, and I'm trying to figure out how the author went from the last step at the bottom of pg. 7 to the final equation (11) at the top of pg. 8. [http://isites.harvard.edu/fs/docs/icb.topic515975.files/OLSDerivation.pdf] More...

Homework Statement \begin{bmatrix} 3x_{1}& 7x_{2}& 4x_{3} \\ 3x_{1}& 4x_{2}& 5x_{3} \\ x_{1}& 10x_{2}& 8x_{3} \\ 8x_{1}& 8x_{2}& 6x_{3} \\ \end{bmatrix} = \begin{bmatrix} 26 \\ 16 \\ 33 \\ 46 \\ \end{bmatrix} the measurements represented by equations 1 and 3 above can be trusted more than those...

I also made a graph which is not pictured. 1.) Calculate the least squares line. Put the equation in the form of: y-hat = a + bx. I got: y hat = 11.304 + 106.218x a.) Find correlation coefficient. Is it significant? (use the p-value to decide) I got: r = 0.913... no it...

Homework Statement Hi guys, so the problem is as follows: A set of n independent measurements y_{i}, i=1...n are treated as Gaussian, each with standard deviations \sigma_{i}. Each measurement corresponds to a value of a control variable x_{i}. The expectation value of y is given by...

Hy, I wonder which is the good solution for this problem: Nonlinear least square problem: function: y = x / (a + b.x) linearization: 1/y = a/x + b substitution: v = 1/y , u = 1/x >> model v = b + a.u What we did in school: x = [1 2 4 7]; y = [2 1 0.4 0.1]; v=1./y; u=1./x; n = length(x)...

Hi folks, 1. Homework Statement I don't fully understand the question statement, how is it supposed to be read? Question: Give a formula for the minimizer x* (to be read as x-star) of the function ƒ:ℝn → ℝ, x → ƒ(x) = ||Ax-b||22, where A∈ℝm×n and b∈ℝm are given. You can assume that A has rank...

Hy I want to know how to make linearization for some function,...what should by in Non-linear least squares problems. In my book I have only this example how to do: http://i.imgur.com/MUFiHkr.pngSomeone could me help how to do, some receipt of method what I need to do? Non-linear least...

Homework Statement The number of twists ##y## required to break a certain rod is a function of the percentages ##u## and ##v## of each of two chemical components present in the rod. The following function is proposed ##y(u, v) = a_{1} + a_{2} exp(u^{2}) + a_{3}\sqrt{v} + a_{4}uv## (1) where the...

Hi, I have an ordinary least squares setup y = Ac where A is an NxM (N>>M) matrix, c the unknown coefficients and y the measurements. Now WEIGHTED least squares allows to weight the MEASUREMENTS if, for example, some measurements are more important or contain a lower variance. However...

Homework Statement We want to determine the coefficients of a polynomial of the form: ##p(x)=c_{1}x^2 +c_{2}x+c_{3}##The polynomial ##p(x)## must satisfy the constraint ##p(1)=1##. We would also like ##p(x)## to satisfy the following 4 constraints: ##p(−1)=5## ##p(0)=−1## ##p(2)=6##...

Suppose we have an observation y = c+ e where c is an unknown constant and e is error with the pdf = exp(-e-1) for e >-1 . We want to determine the least square estimator of c given by the c* which minimizes the error cost function E(c) = .5(y-c)^2 Minimizing the error cost is done by taking...

Hello, When doing a weighted least squares fit of a model to data, I want to examine the residuals to see if their histogram matches the expected probability distribution. Since I am minimizing \chi^2 = \sum_i w_i \left[ y_i - Y(x_i) \right]^2 would I define my...

Homework Statement Ok, so I'm trying to fit a set of data (21000 points to be exact) to a sine function. Homework Equations Y = A*sin(ωt) The Attempt at a Solution I used NumPy to get the parameters A and ω with the least squares method. So far, so good. However, i appear to...

This isn't a homework problem - I'm just confused by something in a textbook that I'm reading (not for a class, either). I'd appreciate an intuitive clarification, or a link to a good explanation (can't seem to find anything useful on Google or in my textbook). My book states that one of the...

Hey guys! Having issues with a code I'm writing. It doesn't fit the typical format and I may have already accidentally posted it in the wrong forum, so here's a copy of what was there with the newest version of the code. It'll (eventually) be a four-parameter least squares minimisation, but for...