Calculate covariance matrix of two given numbers of events

Click For Summary
SUMMARY

The discussion focuses on calculating the covariance matrix for two variables, N1 and N2, derived from event distributions defined by specific functions. The user successfully derived the covariance matrix V using the formula V(m,n) = sum(i = 1 to 2) (dNm/da_i)(dNn/da_i) Error^2(Ni), where Error^2(Ni) = Ni. The user encountered difficulties with off-diagonal terms and sought references for statistical proofs related to this calculation. The final expression provided by the user aligns with standard error propagation techniques in physics.

PREREQUISITES
  • Understanding of covariance and its mathematical definition
  • Familiarity with error propagation techniques in statistics
  • Knowledge of event distributions and their representations
  • Basic proficiency in LaTeX for mathematical expressions
NEXT STEPS
  • Study "Statistical Methods for Physics" by Frederick W. Byron for insights on covariance matrices
  • Learn about error propagation in "Introduction to Probability and Statistics" by William Mendenhall
  • Explore advanced statistical techniques in "The Elements of Statistical Learning" by Hastie, Tibshirani, and Friedman
  • Research LaTeX formatting for mathematical expressions to improve clarity in discussions
USEFUL FOR

Researchers, physicists, and statisticians who are involved in statistical analysis of event data and covariance calculations will benefit from this discussion.

Brais
Messages
7
Reaction score
0
Hi, I am trying to follow this paper: (arXiv link).
On page 18, Appendix A.1, the authors calculate a covariance matrix for two variables in a way I cannot understand.

Homework Statement


Variables N_1[\itex] <br /> and N_2[\itex], distributed on &lt;br /&gt; y \in [0, 1][\itex] as follows:&amp;lt;br /&amp;gt; f_1(y&amp;amp;amp;lt;y_0) = 0; f_1(y&amp;amp;amp;gt;= y_0) = \frac{1}{1-y_0}&amp;lt;br /&amp;gt; f_2 = 1[\itex]&amp;amp;lt;br /&amp;amp;gt; Define: N_{&amp;amp;amp;amp;lt;} = [\itex]amount of events falling below y_0[\itex], N_{&amp;amp;amp;amp;amp;amp;gt;}[\itex] analogously...&amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;gt; &amp;amp;amp;amp;amp;lt;h2&amp;amp;amp;amp;amp;gt;Homework Equations&amp;amp;amp;amp;amp;lt;/h2&amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;gt; Then, we can calculate:&amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;gt; N_2 = \frac{1}{y_0}N_{&amp;amp;amp;amp;amp;amp;amp;lt;}[\itex]&amp;amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;amp;gt; N_1 = -\frac{1-y_0}{y_0}N_{&amp;amp;amp;amp;amp;amp;amp;amp;lt;} + N_{&amp;amp;amp;amp;amp;amp;amp;amp;gt;}[\itex]&amp;amp;amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;amp;amp;gt; Covariance of two variables: Cov(a,b) = &amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;(a·b)^2&amp;amp;amp;amp;amp;amp;amp;amp;amp;gt; - &amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;a·b&amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;^2[\itex]&amp;amp;amp;amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;amp;amp;amp;gt; &amp;amp;amp;amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;amp;amp;amp;gt; &amp;amp;amp;amp;amp;amp;amp;amp;lt;h2&amp;amp;amp;amp;amp;amp;amp;amp;gt;The Attempt at a Solution&amp;amp;amp;amp;amp;amp;amp;amp;lt;/h2&amp;amp;amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;amp;amp;amp;gt; After calculating the two previous results myself (N1 and N2), I tried to calculate the covariance matrix V (please see pdf). I tried to do it from the definition of covariance, but I didn&amp;amp;amp;amp;amp;amp;amp;amp;amp;#039;t get anywhere (or to different results). So I decided to guess if the authors were doing the usual error propagation from N_1, N_2[\itex] as a function of N_{&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;}, N_{&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;}[\itex].&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt; Considering the usual in physics for N large, \sigma(N_k) = \sqrt(N_k), k \in {&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;, &amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;}[\itex], and then, for instance \sigma^2(N_2) =\sum_k \frac{\partial N_2}{\partial N_k} \sigma(N_k) = \frac{1}{y_0 ^2} N_{&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;}[\itex] (the right result).&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt; However, I cannot extend this to the off-diagonal terms.&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt; &amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt; Could somebody please help me? Thanks!&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt; &amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;br /&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt; EDIT: I cannot see the latex expressions, and instead I see all my itex \itex ! How can I solve this? Thanks!
 
Last edited:
Physics news on Phys.org
Right...
after some trial and error, I got to an expression for the right solution of the covariance matrix:

Given the (2x2) covariance matrix V, and the variables:

N1 = a1/c
N2 = -(1-c)a1/c + a2

I calculated V(m,n) = sum(i = 1 to 2) (dNm/da_i)(dNn/da_i) Error^2(Ni),
where Error^2(Ni) = Ni, and m, n are either 1 or 2.

This works and I get the right solution. Also, the expression makes sense... However, I couldn't find this in a book on statistics. Could somebody point me in a good direction? Do you know of any book/page were I can see the proof of this?

Thank you! And sorry for the wrong latex formulae in the previous post, but I don't know how to fix it. I counted the number of itex \itex and it is right :S
 

Similar threads

  • · Replies 10 ·
Replies
10
Views
1K
Replies
5
Views
1K
  • · Replies 4 ·
Replies
4
Views
2K
Replies
6
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
Replies
6
Views
2K
Replies
9
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K