Derivation of Vasicek Entropy Estimator

  • Context: Graduate 
  • Thread starter Thread starter thrillhouse86
  • Start date Start date
  • Tags Tags
    Derivation Entropy
Click For Summary
SUMMARY

The discussion centers on the derivation of the Vasicek entropy estimator, specifically the transformation of the differential entropy formula H(f) = -∫ f(x) log(f(x)) dx into the form H(f) = ∫ log(d/dp F^{-1}(p)) dp. The participants highlight the challenge of applying the derivative of an inverse function, leading to the expression dF^{-1}(p)/dp = 1/f(F^{-1}(p)). The conversation suggests that integration by substitution with x = F^{-1}(p) may be a viable approach to solve the problem.

PREREQUISITES
  • Understanding of differential entropy and probability distribution functions
  • Familiarity with cumulative distribution functions and their inverses
  • Knowledge of calculus, specifically integration and differentiation
  • Experience with entropy estimators in statistical analysis
NEXT STEPS
  • Study the derivation of the Vasicek entropy estimator in detail
  • Learn about integration by substitution techniques in calculus
  • Explore the properties of cumulative distribution functions and their inverses
  • Investigate other entropy estimators used in statistical modeling
USEFUL FOR

Statisticians, data scientists, and researchers in quantitative finance seeking to understand entropy estimators and their applications in probability theory.

thrillhouse86
Messages
77
Reaction score
0
Hey All - I am trying to solve a problem that should be really easy (at least every paper I read says the step is!)

I'm trying to understand where the Vasicek entropy estimator comes from:

I can write the differential entropy of a system as:
[tex] H(f) = -\int^{\infty}_{-\infty} f(x)log(f(x))dx[/tex]

where f(x) is your probability distribution function

Apparently it is an easy step that you can re-write this in the form:
[tex] H(f) = \int^{1}_{0} log(\frac{d}{dp}F^{-1}(p)) dp[/tex]

Where [tex]F^{1}[/tex] is the inverse of the culmative distribution function.

I've tried using the derivative of an inverse function that I learned when trying to find the derivative of inverse trig functions but all I got was:

[tex] \frac{d F^{-1}(p)}{dp} = \frac{1}{f(F^{-1}(p))}[/tex]

By the way just in case I need to point it out - this isn't homework - its a stupid problem which is making me feel incredibly stupid not being able to solve

Thanks,
Thrillhouse
 
Last edited:
Physics news on Phys.org
Maybe it's "integration by sustitution" with [itex]x = F^{-1}(p)[/itex]
 

Similar threads

  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 9 ·
Replies
9
Views
3K
  • · Replies 4 ·
Replies
4
Views
3K
  • · Replies 4 ·
Replies
4
Views
2K