Weak Law of Large Numbers versus Central Limit Theorem

In summary, the Weak Law of Large Numbers states that as the sample size increases, the sample mean will converge to the population mean, but it does not provide any information about the distribution of the sample mean. On the other hand, the Central Limit Theorem states that as the sample size increases, the distribution of the sample mean will approach a normal distribution, regardless of the underlying distribution of the population. This means that even if the population is not normally distributed, the sample mean will still follow a normal distribution as the sample size increases. Therefore, while the Weak Law of Large Numbers focuses on the convergence of the sample mean to the population mean, the Central Limit Theorem provides information on the distribution of the sample mean.
  • #1
maverick280857
1,789
4
Hi

I was studying the WLLN and the CLT. A form of WLLN states that if [itex]X_{n}[/itex] is a sequence of random variables, it satisfies WLLN if there exist sequences [itex]a_{n}[/itex] and [itex]b_{n}[/itex] such that [itex]b_{n}[/itex] is positive and increasing to infinity such that

[tex]\frac{S_{n}-a_{n}}{b_{n}} \rightarrow 0[/tex]

[convergence in probability and hence convergence in law] where [itex]S_{n} = \sum_{i=1}^{n}X_{i}[/itex]. For now, suppose the random variables are independent and identically distributed and also have finite variance [itex]\sigma^2[/itex].

The Lindeberg Levy Central Limit Theorem states that

[tex]\frac{S_{n}-E(S_{n})}{\sqrt{Var(S_{n})}} \rightarrow N(0,1)[/tex]

[convergence in law]

Now, if we take [itex]a_{n} = E(S_{n})[/itex] and [itex]b_{n} = \sqrt{Var(S_{n})} = \sigma\sqrt{n}[/itex], conditions of both the theorems are satisfied. But, the limiting random variables are different. In the first case, the normalized random variable tends to a random variable degenerate at 0 (in law/distribution) whereas using CLT, it tends to a Normally distributed random variable with mean 0 and variance 1.

Does this mean that convergence in law is not unique? What are the implications of these results?

Thanks.
 
Last edited:
Physics news on Phys.org
  • #2
Anyone? :confused:
 
  • #3
In which sense are the "conditions of the WLLN" satisfied for your choice of [tex]b_n[/tex]? The usual version of the WLLN applies to [tex]b_n = n[/tex], so I'm afraid I don't understand.
 
  • #4
Thanks for your reply ichbinfrodo.

ichbinfrodo said:
In which sense are the "conditions of the WLLN" satisfied for your choice of [tex]b_n[/tex]? The usual version of the WLLN applies to [tex]b_n = n[/tex], so I'm afraid I don't understand.

Is it so? As far as I've studied, the conditions on the sequence [itex]b_{n}[/itex] are that [itex]b_{n} > 0[/itex] and [itex]b_{n}[/itex] is increasing to infinity, i.e.

[tex]Lim_{n\rightarrow \infty}b_{n} = \infty[/tex]

In this particular case, [itex]b_{n} = \sigma\sqrt{n}[/itex], which is positive and increasing to infinity.

[itex]b_{n}[/itex] is a norming sequence for the partial sums (and [itex]a_{n}[/itex] a centering sequence). Why should [itex]b_{n} = n[/itex] necessarily?
 
  • #5
What you seemed to be saying in your original post is: If there are [tex](a_n)[/tex], [tex](b_n) \uparrow \infty[/tex] such that
[tex]
\frac{S_n - a_n}{b_n} \stackrel{\mathbb{P}}{\rightarrow} 0 \tag{*}
[/tex]
holds, then the WLLN holds for [tex](X_n)[/tex] i.e.
[tex]
\frac{S_n - E(S_n)}{n} \stackrel{\mathbb{P}}{\rightarrow} 0
[/tex]
Now I don't see why your particular sequences [tex](a_n)[/tex], [tex](b_n)[/tex] should satisfy [tex](*)[/tex].
 
  • #6
I am not saying that the Weak Law of Large Numbers implies the Central Limit Theorem. I am just saying that both are applicable for the particular example and choices of [itex]a_{n}[/itex] and [itex]b_{n}[/itex].

The WLLN (with [itex]a_{n} = n\mu[/itex] and [itex]b_{n} = \sigma\sqrt{n}[/itex]) gives

[tex]

\frac{S_n - n\mu}{\sigma\sqrt{n}} \stackrel{\mathbb{P}}{\rightarrow} 0 \tag{*}

[/tex]

and thus [tex]

\frac{S_n - n\mu}{\sigma\sqrt{n}} \stackrel{\mathbb{L}}{\rightarrow} 0 \tag{**}

[/tex]

The Central Limit Theorem gives

[tex]

\frac{S_n - E(S_n)}{\sqrt{Var(S_{n})}} \stackrel{\mathbb{L}}{\rightarrow} Z

[/tex]

Where Z is a N(0,1) random variable.

In this case, [itex]a_{n} = E(S_{n}) = n\mu[/itex] and [itex]b_{n} = \sqrt{Var(S_{n})} = \sqrt{n\sigma^2}[/itex].

So maybe there are additional constraints I am unaware of.
 
  • #7
Can you state the exact version of the WLLN you are applying?
I clearly must have misinterpreted you in my last post, since (*) is actually the precondition and not the claim of the version of the WLLN I thought you had stated in your original post.
 
  • #8
ichbinfrodo said:
Can you state the exact version of the WLLN you are applying?
I clearly must have misinterpreted you in my last post, since (*) is actually the precondition and not the claim of the version of the WLLN I thought you had stated in your original post.

I'm sorry for the delay. I am quoting below, the statements of the theorems I was referring to.

Weak Law of Large Numbers: Let [itex]\{x_{n}\}[/itex] be a sequence of random variables. Define [itex]S_{n} = \sum_{i=1}^{n}X_{i}[/itex]. Then [itex]\{x_{n}\}[/itex] satisfies the Weak Law of Large Numbers (WLLN) if [itex]\exists[/itex] sequences [itex]\{a_{n}\}[/itex] and [itex]\{b_{n}\}[/itex] such that [itex]b_{n}\ > 0[/itex] [itex]\forall[/itex] [itex]n \in \mathbb{N}[/itex] and [itex]\{b_{n}\} \uparrow \infty[/itex], satisfying

[tex]
\frac{S_{n}-a_{n}}{b_{n}} \stackrel{p}{\rightarrow} 0
[/tex]

Corollary: Since convergence in probability implies convergence in law, we must have

[tex]
\frac{S_{n}-a_{n}}{b_{n}} \stackrel{L}{\rightarrow} 0
[/tex]

whenever the hypotheses of the theorem are satisfied. Further, the above theorem (and the corollary) can be applied to a sequence of independent and identically distributed random variables.

Lindeberg Levy Central Limit Theorem: Let [itex]\{x_{n}\}[/itex] be a sequence of of independent and identically distributed random variables with mean [itex]\mu[/itex] and finite variance [itex]\sigma^2[/itex]. Define [itex]S_{n} = \sum_{i=1}^{n}X_{i}[/itex]. Then, we have

[tex]
\frac{S_{n}-E(S_{n})}{\sqrt{Var(S_{n})}} \stackrel{L}{\rightarrow} X
[/tex]

where [itex]X[/itex] ~ [itex]N(0,1)[/itex].
 
  • #9
maverick280857 said:
Now, if we take [itex]a_{n} = E(S_{n})[/itex] and [itex]b_{n} = \sqrt{Var(S_{n})} = \sigma\sqrt{n}[/itex], conditions of both the theorems are satisfied.

I think you're misreading the conditions and conclusions for the WLLN. It's more of a definition than a theorem in the sense of the CLT. That is, the conditions are not only that [itex]b_n[/itex] be positive and increasing, but also that [tex]\frac{S_n - a_n}{b_n}\to 0[/tex]. The conclusion is "[itex]x_n[/itex] satisfies the WLLN." It does *not* say that any positive, increasing [itex]b_n[/itex] will give you the convergence result; you have to show that it grows quickly enough to overcome the growth in the sum. The cannonical rate, corresponding to an independent sequence, is [itex]b_n \propto n [/itex]; a slower rate will not work for an independent sequence (check this if you don't believe me). A faster growth rate for [itex]b_n[/itex], of course, is no problem.

The CLT is what happens when you shrink the sum by a slower rate than in the WLLN, so that it doesn't become a degenerate random variable, but not so slowly that the result diverges, either. The interesting thing is that the distribution of the result doesn't depend on the particular distributions of the sequence.

For dependent sequences, the critical growth rates for the WLLN and CLT can differ from [itex]n[/itex] and [itex]\sqrt{n}[/itex], respectively. And there are versions of the CLT that don't require independence and/or identical distributions. Check out the wikipedia page on CLT for leads.
 
Last edited by a moderator:
  • #10
quadraphonics said:
I think you're misreading the conditions and conclusions for the WLLN.
(Just for the record, that's what I was trying to say in my previous posts.)
 
  • #11
Thank you both quadraphonics and ichbinfrodo.

quadraphonics said:
The cannonical rate, corresponding to an independent sequence, is [itex]b_n \propto n [/itex]; a slower rate will not work for an independent sequence (check this if you don't believe me). A faster growth rate for [itex]b_n[/itex], of course, is no problem.

What exactly do you mean by 'canonical rate'?

The CLT is what happens when you shrink the sum by a slower rate than in the WLLN, so that it doesn't become a degenerate random variable, but not so slowly that the result diverges, either. The interesting thing is that the distribution of the result doesn't depend on the particular distributions of the sequence.

This is an interesting way of looking at it, thanks.

And there are versions of the CLT that don't require independence and/or identical distributions. Check out the wikipedia page on CLT for leads.

Yes, I saw the page. In my specific problem, I was referring to the i.i.d. case because I'm doing an introductory level course on probability and statistics, so I haven't been formally introduced to other forms of the CLT.

Thanks again for your help :approve:
 
  • #12
maverick280857 said:
What exactly do you mean by 'canonical rate'?

It's the rate that corresponds to an independent sequence (or any sequence with constant "innovation energy"); this is the cannonical example that's used in every textbook in the world. For such a sequence, the power in the sum grows linearly, and so [itex]b_n[/itex] must grow at least linearly for the WLLN to apply. For certain dependent sequences, the rate can differ from linear, but these are typically treated as special cases or counterexamples.
 
  • #13
Thanks again. I haven't encountered these terms in any textbook I've consulted so far (e.g. Feller, Hogg/Craig, Papoulis).
 

Related to Weak Law of Large Numbers versus Central Limit Theorem

1. What is the difference between the Weak Law of Large Numbers (WLLN) and the Central Limit Theorem (CLT)?

The WLLN and CLT are both fundamental theorems in probability theory that explain the behavior of a sequence of independent and identically distributed random variables. The main difference between the two is that the WLLN focuses on the convergence of the sample mean to the true mean as the sample size increases, while the CLT focuses on the convergence of the sample mean to a normal distribution as the sample size increases.

2. Which theorem should be used for large sample sizes?

The CLT should be used for large sample sizes. As the sample size increases, the CLT guarantees that the sample mean will converge to a normal distribution, regardless of the underlying distribution of the random variable. This makes it a powerful tool for analyzing large datasets and making statistical inferences.

3. Can the WLLN and CLT be applied to non-independent and non-identically distributed random variables?

No, the WLLN and CLT are only applicable to sequences of independent and identically distributed random variables. If these conditions are not met, the theorems do not hold and other methods must be used to analyze the data.

4. Are there any assumptions or limitations to the WLLN and CLT?

Yes, both theorems have certain assumptions and limitations. The WLLN assumes that the random variables are independent and identically distributed, and that the expected value of each random variable exists. The CLT assumes that the random variables are independent and identically distributed, and that the variance of each random variable exists and is finite.

5. How can the WLLN and CLT be used in practical applications?

The WLLN and CLT are widely used in statistics and data analysis to make inferences about large datasets. They are also used in quality control and process improvement to monitor and control processes. In addition, the CLT is used in hypothesis testing, confidence intervals, and other statistical methods to make decisions and draw conclusions.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
8
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
  • Calculus and Beyond Homework Help
Replies
1
Views
392
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
14
Views
5K
  • Linear and Abstract Algebra
Replies
1
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
3
Views
995
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
6K
Back
Top