Measurement error analyses, fitting min/max slopes to data with error bars.

In summary, the conversation discusses how to take into account the individual errors in both the x and y directions when fitting a linear regression line to a dataset. The speaker mentions using weights and solving for a non-weighted regression line first in order to account for the errors in the x values. They also discuss the problem of small residuals compared to errors and the issue of finding the true error for the fit. The speaker is interested in finding mathematical ways to do this, particularly using Matlab.
  • #1
deccard
30
0
I have measurement dataset [tex] (x_i,y_i) [/tex] -pairs with error for each value [tex] \Delta x_i [/tex] and [tex] \Delta y_i [/tex] so that I can plot datapoints with vertical as well as horizontal errorbar. I want to fit linear regression line [tex]y=a_1 x + a_0[/tex] and also error lines to the data.

But how I take into account as I'm fitting regression line that each datapoint will have its error in x- and y-direction?

And what about the error lines so that I get min and max value of [tex]a_1, a_0[/tex]. I could use standard deviation, but then again this does not take into account the errors [tex] \Delta x_i [/tex] and [tex] \Delta y_i [/tex].

This picture enlightens my problem
http://www.chemistry.adelaide.edu.au/external/soc-rel/content/images/graph_er.png"

I'm interested only in mathematical ways to do this. I already know how to do this by hand. Especially any Matlab example would be greatly appreciated.
 
Last edited by a moderator:
Physics news on Phys.org
  • #2
So you want to use the indidvidual error of each measurement as as well as their "standard deviation" (a measurement of how well they all lie on a straight line) to compute the error of your fitting parameters?

How did you do it by hand, maybe it is just a matter of converting it into Matlab code (or whatever)?
 
  • #3
I have tried to figure this out by myself and I have managed to get the error for fitted regression line to data points with each its own error, but this still is not what i want.

So, again I have I have measurement dataset [tex](x_i,y_i), i=1..k[/tex]-pairs with error for each value [tex]\Delta x_i[/tex] and [tex]\Delta y_i[/tex] so that I can plot datapoints with vertical as well as horizontal errorbar. I'm going to use weighted fit to solve the regression line for the data.

I can use weights [tex]w_i=1/(\Delta y_i)^2[/tex] in order to find the solution vector [tex]S[/tex] that will minimize [tex](B - AS)^T diag(W)(B - AS)[/tex], where
[tex]A[/tex] is a [tex]k \times 2[/tex] matrix of columns of ones and [tex]x_i[/tex]values,
[tex]B[/tex] is a vector made of [tex]y_i[/tex]values,
and respectively [tex]W[/tex] is a vector made of [tex]w_i[/tex]values.

This is basically what Matlab's function lscov does.

But before I can use the weights, I need to remember that I have also defined errors for [tex]x_i[/tex] values. Thus I cannot directly use the weights [tex]w_i=1/(\Delta y_i)^2[/tex].

I'm get this around by actually solving first, "regular", non-weighted regression line with slope [tex]p[/tex]. Because we are fitting straight linear line to data we actually can find out what a certain [tex]\Delta x_i[/tex] error is going to be from y-axis point of view by multiplying it by the slope and so we get the total error for i:th point.

[tex]\sigma_i = \sqrt{\Delta y_i^2 +(p*\Delta x_i)^2}[/tex]

and the weights

[tex]w_i=1/(\sigma_i)^2[/tex]

The obvious flaw here is that we have to presume that the slope of weighted regression line is somewhat the same that it is for unweighted regression line. Any way to solve this?

And now the error estimates. For the previous fit we can get estimated standard error. Matlab defines it by

Code:
       X = inv(A'*inv(V)*A)*A'*inv(V)*B
       MSE = B'*(inv(V) - inv(V)*A*inv(A'*inv(V)*A)*A'*inv(V))*B./(M-N)
       S = inv(A'*inv(V)*A)*MSE
       STDX = sqrt(diag(S))

where V is covariance matrix and STDX is the estimated standard error.

Now comes the major problem. If I have regression fit that has relatively small residuals compared to the errors suggesting small random error and large systematic error. The error estimates will actually become smaller than they really are. How I'm going to find the true error for my fit?
Pere Callahan said:
So you want to use the indidvidual error of each measurement as as well as their "standard deviation" (a measurement of how well they all lie on a straight line) to compute the error of your fitting parameters?

How did you do it by hand, maybe it is just a matter of converting it into Matlab code (or whatever)?

I hope that the previous help to understand what I want. But yes, that is what I want.

Well actually by saying by hand I meant that I will draw the lines with ruler on real paper. So there is no way that I could convert that into Matlab code ;)
 

1. What is a measurement error analysis?

A measurement error analysis is a process used to quantify the amount of error present in a set of measurements. It involves identifying and understanding the sources of error, and then using statistical methods to estimate the magnitude of the error.

2. How do you determine the amount of error in a measurement?

To determine the amount of error in a measurement, you can use various statistical methods such as calculating the standard deviation or performing a regression analysis. These methods take into account the variability and uncertainty in the data points to provide an estimate of the overall measurement error.

3. What is the purpose of fitting min/max slopes to data with error bars?

The purpose of fitting min/max slopes to data with error bars is to visualize the range of possible slopes that can be derived from the data. This can help to identify any significant trends or relationships between variables and provide a more accurate interpretation of the data.

4. How do you interpret error bars in a measurement?

Error bars in a measurement represent the range of values that the true measurement is likely to fall within. The longer the error bars, the more uncertain the measurement is. The position of the error bars also indicates the direction of the error, with bars above the data point representing a positive error and bars below the data point representing a negative error.

5. What are some common sources of error in measurements?

Common sources of error in measurements include human error, instrument limitations, environmental factors, and sampling errors. Human error can occur during data collection or data entry, while instrument limitations can include accuracy and precision limitations. Environmental factors such as temperature and humidity can also affect measurements, and sampling errors can arise from using a small or biased sample size.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
13
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
778
  • Set Theory, Logic, Probability, Statistics
Replies
3
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
24
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
3
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
8
Views
5K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
28
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
1K
  • Set Theory, Logic, Probability, Statistics
2
Replies
37
Views
3K
Back
Top