Law of Large Numbers:
You will see that this law has two considerations based in convergence of random variables:
Let {Xn: n = 1, 2, 3, ...} be a sequence of random variables, and Sj = Sumation of Xj, from j = 1 to n.
1. Weak Law of Large Numbers.
If a sequence of random variables, {Xn} are uncorrelated and their second moments (second moment is the variance by the Moment Generating function) have a common bound, then (Sn - E[Sn])/n converges to zero in probability.
2. Weak Law of Large Numbers.
If a sequence of random variables, {Xn} are independent and identically distributed and have finite mean m, then (Sn/n) converges to m in probability.
3. Strong Law of Large Numbers.
If a sequence of random variables, {Xn} are uncorrelated and their second moments have a common bound, then (Sn - E[Sn])/n converges to zero almost sure (note why this is stronger than convergence in probability. Remember, convergence almost sure implies convergence in probability).
4. Strong Law of Large Numbers.
If a sequence of random variables, {Xn} are independent and identically distributed and have finite mean m, then (Sn/n) converges to m almost sure.
In other words,
- Weak Law of Large Numbers: the mean of a sequence of random variables converges to the population mean in probability.
- Strong Law of Large Numbers: the mean of a sequence of random variables converges to the population mean almost sure.
basically, Chebyshev's Inequality says that 75% of your data would be two times the standard deviation from the mean, and 95% three times.