Sum independent normal variables

In summary: Yes, if you have the distribution of each component and the covariance, you can determine the joint distribution of the correlated normal random variables. However, if you only have the distribution of each component and do not know the covariance, you cannot determine the joint distribution or prove that the sum of the variables is normal.Yes, if you have the distribution of each component and the covariance, you can determine the joint distribution of the correlated normal random variables. However, if you only have the distribution of each component and do not know the covariance, you cannot determine the joint distribution or prove that the sum of the variables is normal.In summary, the conversation discusses the proof that a finite sum of independent normal random variables is normal. It
  • #1
mathman
Science Advisor
8,140
572
TL;DR Summary
The sum of normally distributed random variables is normal
(I know how to prove it). Prove that a finite sum of of independent normal random variables is normal. I suspect that independence may not be necessary.
 
Physics news on Phys.org
  • #2
Independence is necessary. Suppose, for example that ##X_1 \sim \mathcal{N}(\mu=0,\sigma=1)## and ##X_2 = -X_1## then ##X_2 \sim \mathcal{N}(\mu=0,\sigma=1)## but ##X_1+X_2=0## which is not normally distributed.
 
  • Like
Likes WWGD
  • #3
Dale said:
Independence is necessary. Suppose, for example that ##X_1 \sim \mathcal{N}(\mu=0,\sigma=1)## and ##X_2 = -X_1## then ##X_2 \sim \mathcal{N}(\mu=0,\sigma=1)## but ##X_1+X_2=0## which is not normally distributed.
I was wondering about random variables which are correlated, but with##|\rho|\lt 1## or in general as long as they are linearly in dependent. In that case they could be transformed into a set of independent random variables.
 
  • #4
your original claim is true iff they are jointly normally distributed. This is implied by the standard claim that an random n-vector is a multivariate gaussian iff every 1-dimensional marginal one is a normal r.v.

It depends on pedantry, but Dale's counterexample actually isn't one-- it is just a case of a "normal random variable" with mean zero and variance of zero. Not particularly satisfying but it is useful as a vacuous truth.
 
  • #6
Iirc ,you can use generating functions for the proof.
 
  • #7
WWGD said:
Iirc ,you can use generating functions for the proof.
A proof w/ MGFs would have the same form the characteristic function proof in the link
 
  • #8
Are their theorems that deal with the general question? - Given a set of random variables ##\{X_1,X_2,..,X_M\}##, when does their exist some subspace ##S## of the vector space of random variables such that ##S## contains each ##X_i## and ##S## has a basis of mutually independent random variables?

mathman said:
I was wondering about random variables which are correlated, but with##|\rho|\lt 1## or in general as long as they are linearly in dependent.

"Linearly independent" presumably implies we are considering a set of random variables to be a vector space under the operations of multiplication by scalars and addition of the random variables.

In that case they could be transformed into a set of independent random variables.

Suppose we take "transformed" to mean transformed by a linear transformation in a vector space. If the vector space containing the random variables ##{X_1,X_2,...X_M}## has a finite basis ##B_1,B_2,...B_n## consisting of mutually independent random variables then (trivially) for each ##X_k## there exists a possibly non-invertible linear transformation ##T## that transforms some linear combination of the ##B_i## into ##X_k##.

If the smallest vector space containing the ##X_i## is infinite dimensional (e.g the vector space of all measurable functions on the real number line) , I don't know what happens.

I don't recall any texts that focus on vector spaces of random variables. Since the product of random variables is also a random variable, the topic for textbooks seems to be the algebra of random variables. But that approach downplays the concept of probability distributions.
 
  • #9
BWV said:
by the CLT, a finite sum of random variables from any distribution with finite variance will converge to normal

On the sum of 2 normals, some proofs in wikipedia

https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables
My original question was specifically for finite sums. The Wiki article gives the answer for a pair of correlated variables which then implies it holds for any finite sum.
 
  • #10
mathman said:
The Wiki article gives the answer for a pair of correlated variables which then implies it holds for any finite sum.
Yes, if that did not work, then all of Modern Portfolio Theory would fail
 
  • #11
Stephen Tashi said:
Are their theorems that deal with the general question? - Given a set of random variables ##\{X_1,X_2,..,X_M\}##, when does their exist some subspace ##S## of the vector space of random variables such that ##S## contains each ##X_i## and ##S## has a basis of mutually independent random variables?

I'm not sure this is the right question. Given a single random variable, it plus itself is just 2 time itself, since you're not adding two independent copies of itself. So anyone dimensional subspace for example.If you are wondering about sets of random variables that are stable under adding independent copies, then I think it's just the normal random variables, since if you repeatedly add a random variable to itself you get something that slowly deforms into a gaussian. I guess the space of all things deforming into gaussians also works.
 
  • #12
Office_Shredder said:
I'm not sure this is the right question. Given a single random variable, it plus itself is just 2 time itself, since you're not adding two independent copies of itself. So anyone dimensional subspace for example.If you are wondering about sets of random variables that are stable under adding independent copies, then I think it's just the normal random variables, since if you repeatedly add a random variable to itself you get something that slowly deforms into a gaussian. I guess the space of all things deforming into gaussians also works.
Don't mean to go far OT, but I am kind of curious about transformations that preserve normality.

An interesting thing is that covariance defines an inner product on the space of random variables.
 
  • #13
WWGD said:
Don't mean to go far OT, but I am kind of curious about transformations that preserve normality.

An interesting thing is that covariance defines an inner product on the space of random variables.
Wouldn't any linear or affine transformation of a normal RV preserve normality?
 
  • #14
mathman said:
The Wiki article gives the answer for a pair of correlated variables

Yes, the article gives an answer for a pair of correlated normal random variables that have have a joint bivariate normal distribution, but not for more general case of correlated normal random variables whose joint distribution is not specified.
 
  • #15
Stephen Tashi said:
Yes, the article gives an answer for a pair of correlated normal random variables that have have a joint bivariate normal distribution, but not for more general case of correlated normal random variables whose joint distribution is not specified.
So if you have the distribution of each component and the covariance, don't you also have the joint distribution?
 
  • #17
BWV said:
Wouldn't any linear or affine transformation of a normal RV preserve normality?

I'd say yes,

The topic
but I am kind of curious about transformations that preserve normality.
could include all transformations that preserve normality so perhaps it should be specialized to particular kinds of transformation to eliminate relatively trivial ones. For example if ##X## has a normal distribution with mean zero and we define the transformation ##T(x) = x## except for x = 3 or -3, where we define T(x) = -x, have we have transformed ##X## to a (technically) different normally distributed random variable?
 
  • Like
Likes WWGD
  • #18
Office_Shredder said:
I'm not sure this is the right question. Given a single random variable, it plus itself is just 2 time itself, since you're not adding two independent copies of itself. So anyone dimensional subspace for example.
The question I propose is whether each ##X_i## is an element of the same subspace ##S##. So the fact that each ##X_i## can be regarded as being in the 1 dimensional subspace generated by itself, doesn't answer that.
 
  • #19
It appears that a joint distribution of two dependent normal variables may not be normal. However it is not clear to me whether or not the sum has a normal distribution.
 
  • #20
mathman said:
It appears that a joint distribution of two dependent normal variables may not be normal. However it is not clear to me whether or not the sum has a normal distribution.
Just use , e.g., for X~N(0,1), the sum X+(-X).
 
  • #21
WWGD said:
Just use , e.g., for X~N(0,1), the sum X+(-X).
This is an artificial special case where the variance = 0. It could be called a normal distribution.
 
  • #23
Stephen Tashi said:
Seems bogus to me, as η contains a discontinuous function σ , and is only 'Normal' because the Bernoulli distribution is hidden until you add it to ξ. I think if you wrote out the characteristic functions for the example like in the proof here: https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables

, it would be obvious as you would have the composition of Bernoulli CF of
1616620111790.png
in the gaussian integral and they are really not two normal distributions
 
  • #24
It seems we can set the Copula in such a way that the sum violates normality. I will give it a try, should be a nice exercise. Never tried it before, but should work.
 
Last edited:
  • Like
Likes Stephen Tashi
  • #25
BWV said:
Seems bogus to me, as η contains a discontinuous function σ

How would we state a theorem that avoids that counterexample?

If we say "Let ##\eta## be a normally distributed random variable that does not contain a discontinuous function", what would that mean?

On attempt to define "##\eta## does not contain a discontinuous function" might be: There doesn not exist a finite set of random variables ##\{X_1, X_2,...,X_n\}## such that at least one of the ##X_i## does not have a continuous distribution and such that for some function ##f## we have ##\eta = f(X_1,X_2,...X_n)##. However, given such great freedom for people to choose ##f## and the ##X_i## we might eliminate all normally distributed ##\eta## from consideration.

In practical situations the evidence about how ##\eta## can be represented as functions of other random variables would come from joint measurements of ##\eta## and those random variables or functions of those random variables. So perhaps the definition would need to be of the form "##\eta## does not contain a discontinuous function with respect to the random variable ##X## means that ...".
 
  • #26
WWGD said:
It seems we can set the Copula in such a way that the sum violates normality. I will give it a try, should be a nice exercise. Never tried it before, but should work.

Be careful! https://en.wikipedia.org/wiki/David_X._Li :smile:
 
  • Like
Likes WWGD
  • #27
Stephen Tashi said:
How would we state a theorem that avoids that counterexample?

If we say "Let ##\eta## be a normally distributed random variable that does not contain a discontinuous function", what would that mean?

On attempt to define "##\eta## does not contain a discontinuous function" might be: There doesn not exist a finite set of random variables ##\{X_1, X_2,...,X_n\}## such that at least one of the ##X_i## does not have a continuous distribution and such that for some function ##f## we have ##\eta = f(X_1,X_2,...X_n)##. However, given such great freedom for people to choose ##f## and the ##X_i## we might eliminate all normally distributed ##\eta## from consideration.

In practical situations the evidence about how ##\eta## can be represented as functions of other random variables would come from joint measurements of ##\eta## and those random variables or functions of those random variables. So perhaps the definition would need to be of the form "##\eta## does not contain a discontinuous function with respect to the random variable ##X## means that ...".

Just also define the generating process for the RV - perhaps the sum of normally distributed RVs where each can be completely described by the Gaussian MGF or CF
 
  • #28
BWV said:
Just also define the generating process for the RV - perhaps the sum of normally distributed RVs where each can be completely described by the Gaussian MGF or CF

However, the existence of one sort of generating process for a random variable doesn't rule out the existence of a different generating processes for it.

I think the goal is prove a theorem of the form:

If ##X## and ##Y## are normally distributed random variables with joint distribution ##J(X,Y)## and ... some conditions... then X + Y is normally distributed.

To get anything interesting the "some conditions" must be conditions that aren't trivially equivalent to assuming ##J(X,Y)## is a bivariate normal distribution.
 
  • #29
Also if D is the Dirichlet Function and A is a standard normal is B= D(A)*A normal? Seems it would pass the measure theoretic criteria - it is L-integrable for example. But if C is another standard normal, then C-B has measure zero
 
  • #30
Stephen Tashi said:
However, the existence of one sort of generating process for a random variable doesn't rule out the existence of a different generating processes for it.

I think the goal is prove a theorem of the form:

If ##X## and ##Y## are normally distributed random variables with joint distribution ##J(X,Y)## and ... some conditions... then X + Y is normally distributed.

To get anything interesting the "some conditions" must be conditions that aren't trivially equivalent to assuming ##J(X,Y)## is a bivariate normal distribution.

but you have to define normal by some generating function. If A = the sum of 100 uniform RVs is A normal? In practice it would be treated as such, but then there is some arbitrary number of uniform RVs where the sum is not normal. If you use the full limit and A is the infinite sum of appropriately scaled uniform RVs, then you could subtract some RH unbounded interval in the sum, which would also be normal, and get a non-normal RV
 
  • #31
BWV said:
but you have to define normal by some generating function.

A normal random variable ##\eta## can , in many different ways, be represented as the sum of two other independent normal random variables.

I don't understand how you want to restrict the types of representations of ##\eta## that are permitted.
 
  • #33
mathman said:
Added note: These random variables are uncorrelated.
Is that correct? The intension of the η=σξ example is to randomly flip the sign on half of ξ, which does not change the distribution, but half of η is perfectly negatively correlated to ξ, and this information is recovered in P(ξ+η=0)=P(σ=−1)=1/2.

ISTM there are problems with the construction on this, but they are above my pay grade. ξ is a function on ℝ, while σ is discrete. How would you actually flip the sign randomly on half of ℝ? any countably finite number in σ would not change the outcome of the distribution, and if σ in uncountably infinite, then how is P(ξ+η=0) not 0?
 
  • #34
BWV said:
How would you actually flip the sign randomly on half of ℝ?
Why would that be necessary? If we grant that you can take a random sample from ##\xi##, in order to realize a sample of ##\xi \sigma## you only need one to decide whether to flip the value of that particular random sample by using one realization of ##\sigma##.

The probability of any particular realization of a sample from a normally distributed random variable is zero, so if zero probability events are going to be a conceptual problem, they are already present in the idea of taking a sample from ##\xi##.
 
  • #35
BWV said:
Is that correct? The intension of the η=σξ example is to randomly flip the sign on half of ξ, which does not change the distribution, but half of η is perfectly negatively correlated to ξ, and this information is recovered in P(ξ+η=0)=P(σ=−1)=1/2.

ISTM there are problems with the construction on this, but they are above my pay grade. ξ is a function on ℝ, while σ is discrete. How would you actually flip the sign randomly on half of ℝ? any countably finite number in σ would not change the outcome of the distribution, and if σ in uncountably infinite, then how is P(ξ+η=0) not 0?
Covariance ##E(\xi\eta)=E(\sigma\xi^2)=E(\sigma)E(\xi^2)=0##, since ##E(\sigma)=0## while ##\sigma## is independent of ##\xi## and both means ##=0##.
 
<h2>1. What is the formula for calculating the sum of independent normal variables?</h2><p>The formula for calculating the sum of independent normal variables is: <strong>sum = μ1 + μ2 + ... + μn</strong>, where μ represents the mean of each variable.</p><h2>2. Can the sum of independent normal variables be negative?</h2><p>Yes, the sum of independent normal variables can be negative. This can occur when the individual variables have negative means or when the sum of their means is negative.</p><h2>3. How do you interpret the sum of independent normal variables?</h2><p>The sum of independent normal variables represents the combined effect of each individual variable. It can be thought of as the total or overall mean of the group of variables.</p><h2>4. What is the relationship between the variance of the sum and the individual variances?</h2><p>The variance of the sum of independent normal variables is equal to the sum of the individual variances. This can be written as: <strong>var(sum) = var(μ1) + var(μ2) + ... + var(μn)</strong>.</p><h2>5. How does the central limit theorem apply to the sum of independent normal variables?</h2><p>The central limit theorem states that the sum of a large number of independent random variables will tend towards a normal distribution, regardless of the distribution of the individual variables. Therefore, the sum of independent normal variables will also follow a normal distribution.</p>

1. What is the formula for calculating the sum of independent normal variables?

The formula for calculating the sum of independent normal variables is: sum = μ1 + μ2 + ... + μn, where μ represents the mean of each variable.

2. Can the sum of independent normal variables be negative?

Yes, the sum of independent normal variables can be negative. This can occur when the individual variables have negative means or when the sum of their means is negative.

3. How do you interpret the sum of independent normal variables?

The sum of independent normal variables represents the combined effect of each individual variable. It can be thought of as the total or overall mean of the group of variables.

4. What is the relationship between the variance of the sum and the individual variances?

The variance of the sum of independent normal variables is equal to the sum of the individual variances. This can be written as: var(sum) = var(μ1) + var(μ2) + ... + var(μn).

5. How does the central limit theorem apply to the sum of independent normal variables?

The central limit theorem states that the sum of a large number of independent random variables will tend towards a normal distribution, regardless of the distribution of the individual variables. Therefore, the sum of independent normal variables will also follow a normal distribution.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
3
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
30
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
8
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
9
Views
400
  • Set Theory, Logic, Probability, Statistics
Replies
10
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
880
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
1K
Back
Top