Error Propagation

0xDEADBEEF

I am confused about calculating errors. I have learned if you take the variance covariance matrix [itex]\Sigma_{ij}[/itex] of a fit of function f(x,p) to data for parameters [itex]p_i[/itex] (for example by using Levenberg-Marquart) that the one sigma error interval for [itex]p_i[/itex] is [tex]\sigma_{p_i}=\sqrt{\Sigma_{ii}}[/tex] I only understand this, if there are no covariance terms. Why do we do this? I would have thought a better way to find the error would be to do diagonalize [itex]\Sigma[/itex], say the diagonal form is [itex]\Xi[/itex] with normalized eigenvectors [itex](\vec{v})_k[/itex]. Then we would have independent variables that have a Gaussian distribution and one can calculate the error on [itex]p_i[/itex] using error propagation, i.e. [tex]\sigma_{p_i} = \sqrt{\sum \Xi_{kk}\left\langle(\vec{v})_k\mid l_i \right\rangle}[/tex] where [itex]\left\langle(\vec{v})_k\mid l_i \right\rangle[/itex] is the [itex]i^\text{th}[/itex] component of [itex](\vec{v})_k[/itex]. If this is permissible, is there a name for it?

Stephen Tashi

It is rather confusing how any process can purport to calculate a standard deviation for the paramters of a fit y = f(x,p) in the case when the data is of the form [itex] (x_i,y_i) [/itex]. There is no random sample of the parameters. How can any variation be assigned to them? My best guess is in post #7 of the thread: http://www.physicsforums.com/showthr...ght=parameters

I'm not sure what you mean by "the variance covariance matrix [itex] \Sigma_{i,j} [/itex] of a fit of the function f(x,p) to the data for parameters [itex] p_i [/itex]". What is the definition of that matrix?

0xDEADBEEF

Well I guess that you know the theory better than I do, but the idea is somehow a correspondence between least squares and maximum likelihood.
So you have the sum of the squares of a fit function [itex]f(x,p_1,p_2,\dots)[/itex] to data [itex]x_i,y_i[/itex]

[tex]
sq(p_1,p_2,\dots) = \sum_i (f(x_i,p_1,p_2,\dots)-y_i)^2
[/tex]

And the residuals
[tex]r_i=f(x_i,p_1,p_2,\dots)-y_i[/tex]

for some optimal set of parameters [itex]p_k[/itex] that minimizes sq. If the residuals are gaussian then the variance of the residuals times the reciprocal of the Hessian of [itex]sq(p_1,p_2,\dots)[/itex] is somehow a measure of how confident one can be in the fitted parameters and it is also a variance-covariance matrix. This is how I understand it, but if I would really understand the theory I wouldn't be asking questions. Anyhow my question was why one only uses the diagonal elements of that matrix.

Stephen Tashi

Can your original question can be considered outside of the context of curve-fitting.

Suppose the [itex] p_i [/itex] are simply a set of random variables, not necessarily having the meaning of parameters in a curve fit. If the covariance matrix is [itex] \Sigma [/itex], are you proposing a method to get a different estimate for each [itex] \sigma^2_{p_i} [/itex] than using the diagonal element [itex] \Sigma_{i,i} [/itex] ?

0xDEADBEEF

Exactly. Maybe the thing I am looking for already has a name. If we have a covariance matrix like this

[tex]\Sigma = \left( \begin{matrix} .1&100\\ 100&1000 \end{matrix} \right)[/tex]

The first parameter is varying very little while the second one is varying a lot. But the second parameter also has a large influence on the first parameter, and it seems to me that this does not get captured if we use .1 as the variance for the first parameter. So I was suggesting to diagonalize the matrix to get independent parameters and then something like error propagation to determine the "real" uncertainty of the first parameter. I tried to make an example but I don't know how to make random numbers with a given covariance matrix.

Stephen Tashi

You could use a bivariate normal distribution and try to get the desired covariance matrix.

if you don't want to use the variance of a random variable to define its uncertainty, you'll have to state what definition for uncertainty that you want to use.

The variance of one random variable in a joint distribution, doesn't define a joint confidence interval for several variables. Perhaps you are trying to find a joint confidence interval.