Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Histogram to PDF

  1. Oct 3, 2015 #1
    Say I have a large data set of 1,000,000 points. If I plot a histogram of this data, I get a bar chart with bins along the x-axis and the number of items in each bin along the y-axis.

    If I take the number of items in each bin and divide this by the total number of items (1,000,000 in this case), have I arrived at an experimental approximation of the probability density function?

    Everything I know says that yes, dividing the histogram by the total number of points gets me to an experimental approximation of the PDF, but I want someone who is more familiar with this to confirm. Thank you!
     
  2. jcsd
  3. Oct 3, 2015 #2

    andrewkirk

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    Yes, it does.

    Strictly speaking it's an estimate, not an approximation. In almost all most cases they'll be the same thing. Better to call it an estimate though. It's the best estimate you can make in the absence of any other info.
     
  4. Oct 3, 2015 #3
    Yes, estimate is a much better word! Thank you for clarifying.
     
  5. Oct 4, 2015 #4
    If you have access to the dataset, you can also use density estimation techniques like the kernel density estimation.

    You can find a very good explanation of this technique here: http://www.mglerner.com/blog/?p=28

    I hope this helps !
     
  6. Oct 8, 2015 #5
    Wow, this is really cool. I am playing with KDE techniques now and the results look great. At the very least, it really helps to "smooth" the discrete data to generate a more accurate PDF.
     
  7. Oct 11, 2015 #6

    gill1109

    User Avatar
    Gold Member

    Divide by the total number of items *and* by the width of the bins. Now you have an estimate of the probability density function. (Now you have a function such that the area under the "curve" equals to one).

    How good an estimator is it? And how to choose the bin-width? That has been studied in many papers, for instance in http://link.springer.com/article/10.1007/BF01025868
    On the histogram as a density estimator: L 2 theory
    David Freedman, Persi Diaconis
    Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete
    December 1981, Volume 57, Issue 4, pp 453-476
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Histogram to PDF
  1. Histogram = PDF? (Replies: 3)

  2. What a histogram is? (Replies: 0)

Loading...