- #1

- 530

- 7

Here the standard MLE approach falls over, so how would you go about estimating p?

You are using an out of date browser. It may not display this or other websites correctly.

You should upgrade or use an alternative browser.

You should upgrade or use an alternative browser.

- Thread starter bpet
- Start date

- #1

- 530

- 7

Here the standard MLE approach falls over, so how would you go about estimating p?

- #2

Stephen Tashi

Science Advisor

- 7,638

- 1,493

For example, suppose you have some data

Imagining how we could have data from a singular distribution is an interesting challenge. Even saying we have data from a non-sigular continuos distribution is usually a lie, although apparently a harmless lie.

When we say that we "have data" from a garden variety distribution, such as the uniform distribution on [0,1], we mean that we have data that consists of some truncated values, so we have only rational numbers and only limited precision.

We could imagine having data with infinite precision if we imagine that the numbers are expressed in some symbolic form, such as [itex] \frac{\pi}{3} , \frac{e}{3},\frac{\sqrt{2}}{7} [/itex] etc. However, are we getting into some subtle logical contradiction by doing that? This has to do with imagining that there is a process that can sample a distribution, develop a system of symbolic representation that is adequate to exactly represent the value of each observation and express the data in that form.

- #3

- 530

- 7

Imagining how we could have data from a singular distribution is an interesting challenge. Even saying we have data from a non-sigular continuos distribution is usually a lie, although apparently a harmless lie.

When we say that we "have data" from a garden variety distribution, such as the uniform distribution on [0,1], we mean that we have data that consists of some truncated values, so we have only rational numbers and only limited precision.

We could imagine having data with infinite precision if we imagine that the numbers are expressed in some symbolic form, such as [itex] \frac{\pi}{3} , \frac{e}{3},\frac{\sqrt{2}}{7} [/itex] etc. However, are we getting into some subtle logical contradiction by doing that? This has to do with imagining that there is a process that can sample a distribution, develop a system of symbolic representation that is adequate to exactly represent the value of each observation and express the data in that form.

Interesting point. I guess, when working with finite precision data we are only observing events with non-zero probability, e.g. "X is in the interval (x-dx,x+dx)".

To apply MLE to data drawn from absolutely continuous distributions would also require that the dx are sufficiently small yet equal for any x - but wouldn't that lead to problems when the data is a mix of continuous and singular data, e.g sensitivity to the magnitude of dx?

- #4

Stephen Tashi

Science Advisor

- 7,638

- 1,493

It wonder if some of that measure theory that I was supposed to learn once-upon-a-time says when a mixed distribution can be represented that way. If a mixed discrete and continuous distribution can be respesented as Y(X) for some continuous distribution X then my intuition says that in a practical situation, on could Monte Carlo X using numbers of finite precision and get a good represenation of sampling Y.

I suppose to do actual math, we'd have to formulate a precise definitions of our goals - something to the effect that an "approximable" distribution of Y is one that is the limit (in distribution) of a sequence of random variables Y

- #5

- 530

- 7

...If a mixed discrete and continuous distribution can be respesented as Y(X) for some continuous distribution X then my intuition says that in a practical situation, on could Monte Carlo X using numbers of finite precision and get a good represenation of sampling Y.

We could use the inverse cdf approach, i.e. X ~ U(0,1) and set Y=inf{x:F(x)>=X} (this works for any distribution with known cdf, mixture or otherwise).

I suppose to do actual math, we'd have to formulate a precise definitions of our goals - something to the effect that an "approximable" distribution of Y is one that is the limit (in distribution) of a sequence of random variables Y, where the Yare discrete distributions ( thinking of them as samples from continuous distributions taken with a limited precision that becomes more precise as i increases).

I wonder if explicit discretization can be avoided by directly fitting the cdf to the empirical cdf, for example by minimizing the KS distance. Is much known about the properties of these sort of estimators?

- #6

chiro

Science Advisor

- 4,790

- 132

Here the standard MLE approach falls over, so how would you go about estimating p?

Hey bpet.

There are other estimators besides MLE.

For the uniform, one estimator is using the Method of Moments. Basically this boils down to using moment information (first moment is the mean) and solving equations for estimators using the number of moments required being equal to the number of parameters estimated.

You can use these in the cases where say a continuous distribution has no time-changing derivative like in the uniform case.

Share:

- Replies
- 1

- Views
- 2K