Weak Law of Large Numbers versus Central Limit Theorem

[maverick280857]

Hi

I was studying the WLLN and the CLT. A form of WLLN states that if $X_{n}$ is a sequence of random variables, it satisfies WLLN if there exist sequences $a_{n}$ and $b_{n}$ such that $b_{n}$ is positive and increasing to infinity such that

$$\frac{S_{n}-a_{n}}{b_{n}} \rightarrow 0$$

[convergence in probability and hence convergence in law] where $S_{n} = \sum_{i=1}^{n}X_{i}$. For now, suppose the random variables are independent and identically distributed and also have finite variance $\sigma^2$.

The Lindeberg Levy Central Limit Theorem states that

$$\frac{S_{n}-E(S_{n})}{\sqrt{Var(S_{n})}} \rightarrow N(0,1)$$

[convergence in law]

Now, if we take $a_{n} = E(S_{n})$ and $b_{n} = \sqrt{Var(S_{n})} = \sigma\sqrt{n}$, conditions of both the theorems are satisfied. But, the limiting random variables are different. In the first case, the normalized random variable tends to a random variable degenerate at 0 (in law/distribution) whereas using CLT, it tends to a Normally distributed random variable with mean 0 and variance 1.

Does this mean that convergence in law is not unique? What are the implications of these results?

Thanks.

 

[maverick280857]

Anyone?

 

[ichbinfrodo]

In which sense are the "conditions of the WLLN" satisfied for your choice of $$b_n$$? The usual version of the WLLN applies to $$b_n = n$$, so I'm afraid I don't understand.

 

[maverick280857]

Thanks for your reply ichbinfrodo.

In which sense are the "conditions of the WLLN" satisfied for your choice of $$b_n$$? The usual version of the WLLN applies to $$b_n = n$$, so I'm afraid I don't understand.

Is it so? As far as I've studied, the conditions on the sequence $b_{n}$ are that $b_{n} > 0$ and $b_{n}$ is increasing to infinity, i.e.

$$Lim_{n\rightarrow \infty}b_{n} = \infty$$

In this particular case, $b_{n} = \sigma\sqrt{n}$, which is positive and increasing to infinity.

$b_{n}$ is a norming sequence for the partial sums (and $a_{n}$ a centering sequence). Why should $b_{n} = n$ necessarily?

 

[ichbinfrodo]

What you seemed to be saying in your original post is: If there are $$(a_n)$$, $$(b_n) \uparrow \infty$$ such that
$$\frac{S_n - a_n}{b_n} \stackrel{\mathbb{P}}{\rightarrow} 0 \tag{*}$$
holds, then the WLLN holds for $$(X_n)$$ i.e.
$$\frac{S_n - E(S_n)}{n} \stackrel{\mathbb{P}}{\rightarrow} 0$$
Now I don't see why your particular sequences $$(a_n)$$, $$(b_n)$$ should satisfy $$(*)$$.

 

[maverick280857]

I am not saying that the Weak Law of Large Numbers implies the Central Limit Theorem. I am just saying that both are applicable for the particular example and choices of $a_{n}$ and $b_{n}$.

The WLLN (with $a_{n} = n\mu$ and $b_{n} = \sigma\sqrt{n}$) gives

$$\frac{S_n - n\mu}{\sigma\sqrt{n}} \stackrel{\mathbb{P}}{\rightarrow} 0 \tag{*}$$

and thus $$\frac{S_n - n\mu}{\sigma\sqrt{n}} \stackrel{\mathbb{L}}{\rightarrow} 0 \tag{**}$$

The Central Limit Theorem gives

$$\frac{S_n - E(S_n)}{\sqrt{Var(S_{n})}} \stackrel{\mathbb{L}}{\rightarrow} Z$$

Where Z is a N(0,1) random variable.

In this case, $a_{n} = E(S_{n}) = n\mu$ and $b_{n} = \sqrt{Var(S_{n})} = \sqrt{n\sigma^2}$.

So maybe there are additional constraints I am unaware of.

 

[ichbinfrodo]

Can you state the exact version of the WLLN you are applying?
I clearly must have misinterpreted you in my last post, since (*) is actually the precondition and not the claim of the version of the WLLN I thought you had stated in your original post.

 

[maverick280857]

Can you state the exact version of the WLLN you are applying?
I clearly must have misinterpreted you in my last post, since (*) is actually the precondition and not the claim of the version of the WLLN I thought you had stated in your original post.

I'm sorry for the delay. I am quoting below, the statements of the theorems I was referring to.

Weak Law of Large Numbers: Let $\{x_{n}\}$ be a sequence of random variables. Define $S_{n} = \sum_{i=1}^{n}X_{i}$. Then $\{x_{n}\}$ satisfies the Weak Law of Large Numbers (WLLN) if $\exists$ sequences $\{a_{n}\}$ and $\{b_{n}\}$ such that $b_{n}\ > 0$ $\forall$ $n \in \mathbb{N}$ and $\{b_{n}\} \uparrow \infty$, satisfying

$$\frac{S_{n}-a_{n}}{b_{n}} \stackrel{p}{\rightarrow} 0$$

Corollary: Since convergence in probability implies convergence in law, we must have

$$\frac{S_{n}-a_{n}}{b_{n}} \stackrel{L}{\rightarrow} 0$$

whenever the hypotheses of the theorem are satisfied. Further, the above theorem (and the corollary) can be applied to a sequence of independent and identically distributed random variables.

Lindeberg Levy Central Limit Theorem: Let $\{x_{n}\}$ be a sequence of of independent and identically distributed random variables with mean $\mu$ and finite variance $\sigma^2$. Define $S_{n} = \sum_{i=1}^{n}X_{i}$. Then, we have

$$\frac{S_{n}-E(S_{n})}{\sqrt{Var(S_{n})}} \stackrel{L}{\rightarrow} X$$

where $X$ ~ $N(0,1)$.

 

[quadraphonics]

Now, if we take $a_{n} = E(S_{n})$ and $b_{n} = \sqrt{Var(S_{n})} = \sigma\sqrt{n}$, conditions of both the theorems are satisfied.

I think you're misreading the conditions and conclusions for the WLLN. It's more of a definition than a theorem in the sense of the CLT. That is, the conditions are not only that $b_n$ be positive and increasing, but also that $$\frac{S_n - a_n}{b_n}\to 0$$. The conclusion is "$x_n$ satisfies the WLLN." It does *not* say that any positive, increasing $b_n$ will give you the convergence result; you have to show that it grows quickly enough to overcome the growth in the sum. The cannonical rate, corresponding to an independent sequence, is $b_n \propto n$; a slower rate will not work for an independent sequence (check this if you don't believe me). A faster growth rate for $b_n$, of course, is no problem.

The CLT is what happens when you shrink the sum by a slower rate than in the WLLN, so that it doesn't become a degenerate random variable, but not so slowly that the result diverges, either. The interesting thing is that the distribution of the result doesn't depend on the particular distributions of the sequence.

For dependent sequences, the critical growth rates for the WLLN and CLT can differ from $n$ and $\sqrt{n}$, respectively. And there are versions of the CLT that don't require independence and/or identical distributions. Check out the wikipedia page on CLT for leads.

 

[ichbinfrodo]

I think you're misreading the conditions and conclusions for the WLLN.

(Just for the record, that's what I was trying to say in my previous posts.)

 

[maverick280857]

Thank you both quadraphonics and ichbinfrodo.

The cannonical rate, corresponding to an independent sequence, is $b_n \propto n$; a slower rate will not work for an independent sequence (check this if you don't believe me). A faster growth rate for $b_n$, of course, is no problem.

What exactly do you mean by 'canonical rate'?

The CLT is what happens when you shrink the sum by a slower rate than in the WLLN, so that it doesn't become a degenerate random variable, but not so slowly that the result diverges, either. The interesting thing is that the distribution of the result doesn't depend on the particular distributions of the sequence.

This is an interesting way of looking at it, thanks.

And there are versions of the CLT that don't require independence and/or identical distributions. Check out the wikipedia page on CLT for leads.

Yes, I saw the page. In my specific problem, I was referring to the i.i.d. case because I'm doing an introductory level course on probability and statistics, so I haven't been formally introduced to other forms of the CLT.

Thanks again for your help

 

[quadraphonics]

What exactly do you mean by 'canonical rate'?

It's the rate that corresponds to an independent sequence (or any sequence with constant "innovation energy"); this is the cannonical example that's used in every textbook in the world. For such a sequence, the power in the sum grows linearly, and so $b_n$ must grow at least linearly for the WLLN to apply. For certain dependent sequences, the rate can differ from linear, but these are typically treated as special cases or counterexamples.

 

[maverick280857]

Thanks again. I haven't encountered these terms in any textbook I've consulted so far (e.g. Feller, Hogg/Craig, Papoulis).