Error Propagation: Calc Errors w/ Variance Covariance Matrix

In summary: I am confused about calculating errors. I have learned if you take the variance covariance matrix \Sigma_{ij} of a fit of function f(x,p) to data for parameters p_i (for example by using Levenberg-Marquart) that the one sigma error interval for p_i is \sigma_{p_i}=\sqrt{\Sigma_{ii}}In summary, the one sigma error interval for p_i is determined by the variance covariance matrix \Sigma_{ij} of a fit of function f(x,p) to data for parameters p_i.
  • #1
0xDEADBEEF
816
1
I am confused about calculating errors. I have learned if you take the variance covariance matrix [itex]\Sigma_{ij}[/itex] of a fit of function f(x,p) to data for parameters [itex]p_i[/itex] (for example by using Levenberg-Marquart) that the one sigma error interval for [itex]p_i[/itex] is [tex]\sigma_{p_i}=\sqrt{\Sigma_{ii}}[/tex] I only understand this, if there are no covariance terms. Why do we do this? I would have thought a better way to find the error would be to do diagonalize [itex]\Sigma[/itex], say the diagonal form is [itex]\Xi[/itex] with normalized eigenvectors [itex](\vec{v})_k[/itex]. Then we would have independent variables that have a Gaussian distribution and one can calculate the error on [itex]p_i[/itex] using error propagation, i.e. [tex]\sigma_{p_i} = \sqrt{\sum \Xi_{kk}\left\langle(\vec{v})_k\mid l_i \right\rangle}[/tex] where [itex]\left\langle(\vec{v})_k\mid l_i \right\rangle[/itex] is the [itex]i^\text{th}[/itex] component of [itex](\vec{v})_k[/itex]. If this is permissible, is there a name for it?
 
Physics news on Phys.org
  • #2
0xDEADBEEF said:
I am confused about calculating errors. I have learned if you take the variance covariance matrix [itex]\Sigma_{ij}[/itex] of a fit of function f(x,p) to data for parameters [itex]p_i[/itex] (for example by using Levenberg-Marquart) that the one sigma error interval for [itex]p_i[/itex] is [tex]\sigma_{p_i}=\sqrt{\Sigma_{ii}}[/tex]

It is rather confusing how any process can purport to calculate a standard deviation for the paramters of a fit y = f(x,p) in the case when the data is of the form [itex] (x_i,y_i) [/itex]. There is no random sample of the parameters. How can any variation be assigned to them? My best guess is in post #7 of the thread: https://www.physicsforums.com/showthread.php?t=645291&highlight=parameters

I'm not sure what you mean by "the variance covariance matrix [itex] \Sigma_{i,j} [/itex] of a fit of the function f(x,p) to the data for parameters [itex] p_i [/itex]". What is the definition of that matrix?
 
  • #3
Well I guess that you know the theory better than I do, but the idea is somehow a correspondence between least squares and maximum likelihood.
So you have the sum of the squares of a fit function [itex]f(x,p_1,p_2,\dots)[/itex] to data [itex]x_i,y_i[/itex]

[tex]
sq(p_1,p_2,\dots) = \sum_i (f(x_i,p_1,p_2,\dots)-y_i)^2
[/tex]

And the residuals
[tex]r_i=f(x_i,p_1,p_2,\dots)-y_i[/tex]

for some optimal set of parameters [itex]p_k[/itex] that minimizes sq. If the residuals are gaussian then the variance of the residuals times the reciprocal of the Hessian of [itex]sq(p_1,p_2,\dots)[/itex] is somehow a measure of how confident one can be in the fitted parameters and it is also a variance-covariance matrix. This is how I understand it, but if I would really understand the theory I wouldn't be asking questions. Anyhow my question was why one only uses the diagonal elements of that matrix.
 
  • #4
Can your original question can be considered outside of the context of curve-fitting.

0xDEADBEEF said:
I would have thought a better way to find the error would be to do diagonalize [itex]\Sigma[/itex], say the diagonal form is [itex]\Xi[/itex] with normalized eigenvectors [itex](\vec{v})_k[/itex]. Then we would have independent variables that have a Gaussian distribution and one can calculate the error on [itex]p_i[/itex] using error propagation, i.e. [tex]\sigma_{p_i} = \sqrt{\sum \Xi_{kk}\left\langle(\vec{v})_k\mid l_i \right\rangle}[/tex] where [itex]\left\langle(\vec{v})_k\mid l_i \right\rangle[/itex] is the [itex]i^\text{th}[/itex] component of [itex](\vec{v})_k[/itex]. If this is permissible, is there a name for it?

Suppose the [itex] p_i [/itex] are simply a set of random variables, not necessarily having the meaning of parameters in a curve fit. If the covariance matrix is [itex] \Sigma [/itex], are you proposing a method to get a different estimate for each [itex] \sigma^2_{p_i} [/itex] than using the diagonal element [itex] \Sigma_{i,i} [/itex] ?
 
  • #5
Exactly. Maybe the thing I am looking for already has a name. If we have a covariance matrix like this

[tex]\Sigma = \left( \begin{matrix} .1&100\\ 100&1000 \end{matrix} \right)[/tex]

The first parameter is varying very little while the second one is varying a lot. But the second parameter also has a large influence on the first parameter, and it seems to me that this does not get captured if we use .1 as the variance for the first parameter. So I was suggesting to diagonalize the matrix to get independent parameters and then something like error propagation to determine the "real" uncertainty of the first parameter. I tried to make an example but I don't know how to make random numbers with a given covariance matrix.
 
  • #6
You could use a bivariate normal distribution and try to get the desired covariance matrix.

if you don't want to use the variance of a random variable to define its uncertainty, you'll have to state what definition for uncertainty that you want to use.

The variance of one random variable in a joint distribution, doesn't define a joint confidence interval for several variables. Perhaps you are trying to find a joint confidence interval.
 

1. What is error propagation?

Error propagation, also known as uncertainty propagation, is the process of determining the uncertainty in the output of a mathematical model or calculation based on the uncertainties in its input parameters.

2. How is error propagation calculated?

Error propagation is calculated using the variance-covariance matrix, which takes into account the uncertainties and correlations of each input parameter. The formula for error propagation involves taking the partial derivatives of the mathematical model with respect to each input parameter and multiplying them by their respective uncertainties, then squaring and summing these values.

3. What is the purpose of using a variance-covariance matrix in error propagation?

The variance-covariance matrix allows for a more accurate and comprehensive calculation of error propagation by taking into account the correlations between input parameters. This is important because the uncertainty in one parameter can affect the uncertainty in another parameter, and this must be accounted for in the final error estimate.

4. Can error propagation be applied to any mathematical model?

Yes, error propagation can be applied to any mathematical model that has input parameters with associated uncertainties. However, it is important to note that this method assumes that the mathematical model is linear and that the uncertainties in the input parameters are normally distributed.

5. How is error propagation used in scientific research?

Error propagation is used in scientific research to quantify the uncertainty in the results of mathematical models and calculations. This allows for a more accurate interpretation and understanding of the data, and helps researchers determine the reliability of their findings. It is particularly important in fields such as physics, engineering, and statistics.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
6
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
733
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
916
  • Other Physics Topics
Replies
1
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
812
  • Set Theory, Logic, Probability, Statistics
Replies
11
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
781
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
Replies
4
Views
905
Back
Top