Register to reply

Error Propagation

by 0xDEADBEEF
Tags: error, propagation
Share this thread:
0xDEADBEEF
#1
Nov20-12, 11:59 AM
P: 824
I am confused about calculating errors. I have learned if you take the variance covariance matrix [itex]\Sigma_{ij}[/itex] of a fit of function f(x,p) to data for parameters [itex]p_i[/itex] (for example by using Levenberg-Marquart) that the one sigma error interval for [itex]p_i[/itex] is [tex]\sigma_{p_i}=\sqrt{\Sigma_{ii}}[/tex] I only understand this, if there are no covariance terms. Why do we do this? I would have thought a better way to find the error would be to do diagonalize [itex]\Sigma[/itex], say the diagonal form is [itex]\Xi[/itex] with normalized eigenvectors [itex](\vec{v})_k[/itex]. Then we would have independent variables that have a Gaussian distribution and one can calculate the error on [itex]p_i[/itex] using error propagation, i.e. [tex]\sigma_{p_i} = \sqrt{\sum \Xi_{kk}\left\langle(\vec{v})_k\mid l_i \right\rangle}[/tex] where [itex]\left\langle(\vec{v})_k\mid l_i \right\rangle[/itex] is the [itex]i^\text{th}[/itex] component of [itex](\vec{v})_k[/itex]. If this is permissible, is there a name for it?
Phys.Org News Partner Science news on Phys.org
Physicists discuss quantum pigeonhole principle
Giant crater in Russia's far north sparks mystery
NASA Mars spacecraft prepare for close comet flyby
Stephen Tashi
#2
Nov20-12, 07:14 PM
Sci Advisor
P: 3,252
Quote Quote by 0xDEADBEEF View Post
I am confused about calculating errors. I have learned if you take the variance covariance matrix [itex]\Sigma_{ij}[/itex] of a fit of function f(x,p) to data for parameters [itex]p_i[/itex] (for example by using Levenberg-Marquart) that the one sigma error interval for [itex]p_i[/itex] is [tex]\sigma_{p_i}=\sqrt{\Sigma_{ii}}[/tex]
It is rather confusing how any process can purport to calculate a standard deviation for the paramters of a fit y = f(x,p) in the case when the data is of the form [itex] (x_i,y_i) [/itex]. There is no random sample of the parameters. How can any variation be assigned to them? My best guess is in post #7 of the thread: http://www.physicsforums.com/showthr...ght=parameters

I'm not sure what you mean by "the variance covariance matrix [itex] \Sigma_{i,j} [/itex] of a fit of the function f(x,p) to the data for parameters [itex] p_i [/itex]". What is the definition of that matrix?
0xDEADBEEF
#3
Nov21-12, 03:59 PM
P: 824
Well I guess that you know the theory better than I do, but the idea is somehow a correspondence between least squares and maximum likelihood.
So you have the sum of the squares of a fit function [itex]f(x,p_1,p_2,\dots)[/itex] to data [itex]x_i,y_i[/itex]

[tex]
sq(p_1,p_2,\dots) = \sum_i (f(x_i,p_1,p_2,\dots)-y_i)^2
[/tex]

And the residuals
[tex]r_i=f(x_i,p_1,p_2,\dots)-y_i[/tex]

for some optimal set of parameters [itex]p_k[/itex] that minimizes sq. If the residuals are gaussian then the variance of the residuals times the reciprocal of the Hessian of [itex]sq(p_1,p_2,\dots)[/itex] is somehow a measure of how confident one can be in the fitted parameters and it is also a variance-covariance matrix. This is how I understand it, but if I would really understand the theory I wouldn't be asking questions. Anyhow my question was why one only uses the diagonal elements of that matrix.

Stephen Tashi
#4
Nov25-12, 12:53 PM
Sci Advisor
P: 3,252
Error Propagation

Can your original question can be considered outside of the context of curve-fitting.

Quote Quote by 0xDEADBEEF View Post
I would have thought a better way to find the error would be to do diagonalize [itex]\Sigma[/itex], say the diagonal form is [itex]\Xi[/itex] with normalized eigenvectors [itex](\vec{v})_k[/itex]. Then we would have independent variables that have a Gaussian distribution and one can calculate the error on [itex]p_i[/itex] using error propagation, i.e. [tex]\sigma_{p_i} = \sqrt{\sum \Xi_{kk}\left\langle(\vec{v})_k\mid l_i \right\rangle}[/tex] where [itex]\left\langle(\vec{v})_k\mid l_i \right\rangle[/itex] is the [itex]i^\text{th}[/itex] component of [itex](\vec{v})_k[/itex]. If this is permissible, is there a name for it?
Suppose the [itex] p_i [/itex] are simply a set of random variables, not necessarily having the meaning of parameters in a curve fit. If the covariance matrix is [itex] \Sigma [/itex], are you proposing a method to get a different estimate for each [itex] \sigma^2_{p_i} [/itex] than using the diagonal element [itex] \Sigma_{i,i} [/itex] ?
0xDEADBEEF
#5
Nov26-12, 03:32 PM
P: 824
Exactly. Maybe the thing I am looking for already has a name. If we have a covariance matrix like this

[tex]\Sigma = \left( \begin{matrix} .1&100\\ 100&1000 \end{matrix} \right)[/tex]

The first parameter is varying very little while the second one is varying a lot. But the second parameter also has a large influence on the first parameter, and it seems to me that this does not get captured if we use .1 as the variance for the first parameter. So I was suggesting to diagonalize the matrix to get independent parameters and then something like error propagation to determine the "real" uncertainty of the first parameter. I tried to make an example but I don't know how to make random numbers with a given covariance matrix.
Stephen Tashi
#6
Nov26-12, 07:28 PM
Sci Advisor
P: 3,252
You could use a bivariate normal distribution and try to get the desired covariance matrix.

if you don't want to use the variance of a random variable to define its uncertainty, you'll have to state what definition for uncertainty that you want to use.

The variance of one random variable in a joint distribution, doesn't define a joint confidence interval for several variables. Perhaps you are trying to find a joint confidence interval.


Register to reply

Related Discussions
Error Propagation Introductory Physics Homework 0
Error propagation Introductory Physics Homework 1
Error Propagation General Math 3
Error propagation Introductory Physics Homework 1
Error propagation General Math 1