# Empirical and theoretical probability

1. Aug 14, 2011

### Rasalhague

What's the difference? How are empirical and theoretical probability defined in general? If some people use the terms in different ways to others, is one definition more standard?

(This question is part of a broader quest to work out how the statistics terms "population, sample, sampling, probability" relate to the probability theory terms "sample space, observation space, probability measure, distribution, random variable".)

Perhaps empirical probability is a probability measure associated with a sample space, called "the population", representing all possible outcomes of a physical process that happens to be the object of study. This page by Stefan Waner makes a distinction between empirical probability and estimated probability. Estimated probability sounds like the natural probability measure associated with a sample space which is a finite subset of the population. Statistics texts call such a sample space a "sample". They call a sample space derived somehow from a set of samples, and their associated measures, a "sampling". Perhaps probability measures associated with "samplings" (memorable terminology!), would also be considered estimated probabilities by Waner.

But then there seems to be another tradition, better represented on the top results in Google, according to which empirical probability is the name given to Waner's estimated probability; that is, the natural measure associated with the kind of sample space which context determines is a sample. So these people's empirical probability is relative frequency, a fancy word for proportion? (And synonymous with "posterior probability".) In this tradition, theoretical probability appears to mean a probability measure on the population, considered as an object inaccessible to direct observation, whose properties (called by statesticians "parameters") can only be infered from samples and their (empirical) probability measures, whose parameters (in the general mathematical sense) are called "statistics".

http://www.regentsprep.org/Regents/math/algtrig/ATS6/LTheo.htm
http://en.wikipedia.org/wiki/Empirical_probability

And here is someone who identifies a particular result as an empirical, as opposed to theoretical, probability on the (assumed) grounds that it's an estimate based on empirical data, rather than theory... except that it does rely on a theory, namely that the sample - of what has been recorded so far - is representative of what will happen in the future. Or maybe this default theory is so simple it doesn't count.

Ah, here's a link that elaborates on that second (more standard?) tradition:

http://faculty.vassar.edu/lowry/ch2pt3.html

Could we say that no probability measure with an infinite sample space can ever be an empirical probability measure (=Waner's estimated probability measure)? But then, what if our object of interest was itself a mathematical idea, such as the normal probability measure with such-and-such a mean and standard deviation, and the real line for its sample space? Is every real number "capable in principle of being observed"? I guess yes, so, in this context, this normal probability measure would be an empirical probability measure. Maybe we need to include something about context, and how these tools are applied.

Well, I'm probably getting tangled up in philisophy now. That last link has a nice description:

Last edited: Aug 14, 2011