Proving the Law of Large Numbers & Chebyshev's Theorem

[bomba923]

Well um, I was wondering...w/o simulation,

How do you prove the Law of Large Numbers?
And what's Chebyshev's Theorem? (somewhere, i heard it was mentioned, but what is it?)

 

[jetoso]

Law of Large Numbers:
You will see that this law has two considerations based in convergence of random variables:
Let {Xn: n = 1, 2, 3, ...} be a sequence of random variables, and Sj = Sumation of Xj, from j = 1 to n.

1. Weak Law of Large Numbers.
If a sequence of random variables, {Xn} are uncorrelated and their second moments (second moment is the variance by the Moment Generating function) have a common bound, then (Sn - E[Sn])/n converges to zero in probability.

2. Weak Law of Large Numbers.
If a sequence of random variables, {Xn} are independent and identically distributed and have finite mean m, then (Sn/n) converges to m in probability.

3. Strong Law of Large Numbers.
If a sequence of random variables, {Xn} are uncorrelated and their second moments have a common bound, then (Sn - E[Sn])/n converges to zero almost sure (note why this is stronger than convergence in probability. Remember, convergence almost sure implies convergence in probability).

4. Strong Law of Large Numbers.
If a sequence of random variables, {Xn} are independent and identically distributed and have finite mean m, then (Sn/n) converges to m almost sure.


In other words,
- Weak Law of Large Numbers: the mean of a sequence of random variables converges to the population mean in probability.
- Strong Law of Large Numbers: the mean of a sequence of random variables converges to the population mean almost sure.

Basicly, Chebyshev's Inequality says that 75% of your data would be two times the standard deviation from the mean, and 95% three times.

 

[tacman]

The Law of Large Numbers states that as the number of trials in a random experiment increases, the average of the outcomes will converge to the expected value. In other words, the more times an experiment is repeated, the closer the average outcome will be to the expected value. This is a fundamental concept in probability theory and is often used in statistical analyses.

To prove the Law of Large Numbers, we can use mathematical induction. First, we assume that the expected value exists and is finite. Then, we can show that as the number of trials increases, the average of the outcomes approaches the expected value. This can be done by using the properties of expected value and the fact that the average of a larger sample size is more representative of the true population mean.

Chebyshev's Theorem, also known as the Chebyshev's Inequality, is a fundamental result in probability theory that provides an upper bound for the probability of a random variable deviating from its expected value. It states that for any random variable with mean μ and variance σ^2, the probability that the random variable deviates from its mean by more than k standard deviations is at most 1/k^2. In other words, the probability of a random variable being within k standard deviations from its mean is at least 1-1/k^2.

To prove Chebyshev's Theorem, we can use Markov's Inequality and the definition of variance. By applying Markov's Inequality and manipulating the expression, we can arrive at the desired result. This theorem is useful in determining the likelihood of extreme events occurring and is often used in statistical analyses to set confidence intervals.