Calculating Covariance in Bivariate Normal Distribution - Step by Step Guide

  • Context: Graduate 
  • Thread starter Thread starter jimmy1
  • Start date Start date
  • Tags Tags
    Covariance
Click For Summary

Discussion Overview

The discussion revolves around calculating the covariance between two dependent normally distributed random variables, X and Y. Participants explore the challenges of deriving covariance or correlation coefficients from theoretical frameworks, particularly in the context of bivariate normal distributions, and the implications of needing empirical data for such calculations.

Discussion Character

  • Exploratory, Technical explanation, Debate/contested

Main Points Raised

  • One participant notes the formula for covariance, Cov(XY) = E(XY) - E(X)E(Y), but expresses confusion about calculating E(XY) without already knowing the correlation coefficient.
  • Another suggests that covariance can be numerically calculated through empirical observations, providing an example with daily humidity and temperature measurements.
  • A participant emphasizes the need for a theoretical expression for covariance rather than an empirical result, questioning how to derive covariance when dependent variables are involved.
  • Some participants mention that if only marginal distributions are known, empirical methods or copulas may be necessary to obtain joint distributions.
  • One participant asserts that without empirical data, the bivariate normal distribution remains undefined, as it relies on known covariance or correlation.
  • There is a reiteration that theoretical distributions require knowledge of correlation or covariance, with empirical data being the primary means to obtain these values unless theoretical information is available.
  • A later post references a document discussing the relationship between marginal distributions and correlation in determining joint distributions, noting that this is only true under certain conditions, such as multivariate normality.

Areas of Agreement / Disagreement

Participants generally agree that empirical data is necessary to calculate covariance or correlation in the absence of theoretical parameters. However, there is some contention regarding the sufficiency of marginal distributions and correlation in determining joint distributions, with references to specific conditions under which this holds true.

Contextual Notes

Participants highlight limitations in deriving covariance without empirical data and the dependency on definitions of distributions. The discussion also touches on the complexity of joint distributions when only marginal distributions and correlation are known.

jimmy1
Messages
60
Reaction score
0
I have 2 normally distributed dependent random variables, X and Y, and I have the mean and variance of both of them, and I want to find the covariance (or correlation) between X and Y.

Now the formula for the covariance is Cov(XY) = E(XY) - E(X)E(Y). So I tried to calculate E(XY) via the bivariate normal distribution, but it seems that to use the bivariate normal I need to provide the correlation coefficient as a parameter, but this is the parameter that I'm trying to actually find.

So how would I find an expression for the covariance of X and Y? To find E(XY), it seems you need to use P(XY), but to use this bivariate probability you need to provide the covariance (or correlation coefficient). So how do I get around this problem??
 
Physics news on Phys.org
You can numerically calculate the covariance by taking multiple observations from the two distributions and multiplying their values. The mean of their product is E(XY). For example, let X = daily humidity and Y = daily temperature. If I measure humidity and temperature over 100 days (or at 100 locations), I will have 100 ordered pairs of (X,Y), from which I can calculate E(XY).
 
Yes, with observations from the two distribitions I could calculate E(XY), but I need an expression for the covariance of X and Y, and not really an empirical result. That is given (X,Y) are both normally distributed with mean (m1, m2), and standard deviation (s1,s2) how do I find the covariance of XY, when they are dependent.

What really confuses me is that for all these bivariate distributions you need to supply a correlation coefficient or some sort of covariance parameter, yet there seems to be no way of actually obtaining these covariances for dependent variables?

So how does one actually use any of these bivariate distributions if it's not possible to theoretically get the covariances??
 
If you think that two variables are jointly distributed but you only know the marginal distributions, the simplest way to obtain the joint dist. is to calculate the covariance empirically. Other than that, there are copulas: http://en.wikipedia.org/wiki/Copula_(statistics)
 
jimmy1 said:
I have 2 normally distributed dependent random variables, X and Y, and I have the mean and variance of both of them, and I want to find the covariance (or correlation) between X and Y.

Now the formula for the covariance is Cov(XY) = E(XY) - E(X)E(Y). So I tried to calculate E(XY) via the bivariate normal distribution, but it seems that to use the bivariate normal I need to provide the correlation coefficient as a parameter, but this is the parameter that I'm trying to actually find.

So how would I find an expression for the covariance of X and Y? To find E(XY), it seems you need to use P(XY), but to use this bivariate probability you need to provide the covariance (or correlation coefficient). So how do I get around this problem??

When dealing with the theoretical distribution only (no empirical data), there is no way around the "problem". A bivariate normal distribution is defined by the covariance (or equivalent alternatives). If you don't have something, it remains undefined.
 
mathman said:
When dealing with the theoretical distribution only (no empirical data), there is no way around the "problem". A bivariate normal distribution is defined by the covariance (or equivalent alternatives). If you don't have something, it remains undefined.

So would I be right in concluding that when using any sort of multivariate distribution you have to know the correlation (covariance) between the random variables, and the only way to get these covariances is empirically?
 
jimmy1 said:
So would I be right in concluding that when using any sort of multivariate distribution you have to know the correlation (covariance) between the random variables, and the only way to get these covariances is empirically?

Yes - unless there is some given theoretical information you can use.
 
http://www.math.ethz.ch/%7Estrauman/preprints/pitfalls.pdf

Where not otherwise stated, we consider bivariate distributions of the random
vector (X; Y)^t.
Fallacy 1. Marginal distributions and correlation determine the joint distribution.
This is true if we restrict our attention to the multivariate normal distribution or
the elliptical distributions. For example, if we know that (X; Y)^t have a bivariate
normal distribution, then the expectations and variances of X and Y and the correlation
r(X; Y) uniquely determine the joint distribution. However, if we only know
the marginal distributions of X and Y and the correlation then there are many possible
bivariate distributions for (X; Y)^t. The distribution of (X; Y)^t is not uniquely
determined by F1, F2 and r(X; Y ). We illustrate this with examples, interesting in
their own right.
 
Last edited by a moderator:

Similar threads

MHB Distribution, expected value, variance, covariance and correlation
  • · Replies 14 ·
Replies
14
Views
2K
Undergrad Linear regression and random variables
  • · Replies 30 ·
2
Replies
30
Views
5K
Graduate Calculating Bivariate Normal Probabilities
  • · Replies 1 ·
Replies
1
Views
2K
Graduate Joint Used to Show lack of Correlation?
  • · Replies 43 ·
2
Replies
43
Views
6K
Undergrad Covariance of Fourier conjugates for Gaussian distributions
  • · Replies 12 ·
Replies
12
Views
3K
Graduate 2D space and 1D time evolution of a random field
  • · Replies 5 ·
Replies
5
Views
2K
Undergrad What Are the Key Differences Between OLS and GLM Models in Statistical Analysis?
  • · Replies 3 ·
Replies
3
Views
3K
Graduate Bivariate gaussians for ion beam optics
  • · Replies 7 ·
Replies
7
Views
3K
Undergrad Weird covariance matrix for linear fit
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad How Do You Estimate Covariance in Physics Experiments?
  • · Replies 3 ·
Replies
3
Views
2K