Question about entropy, probability

Click For Summary
SUMMARY

The discussion centers on the concept of entropy in probability theory, specifically addressing how the entropy of a random variable X, defined as -E[ln(fX(X))], remains unchanged when X is translated by a constant. The key insight is that the probability density function (pdf) transforms as f_{X+c}(x) = f_{X}(x-c), which preserves the expected value. This demonstrates that adding a constant to the random variable does not alter its entropy, as the logarithmic transformation and expectation are invariant under this translation.

PREREQUISITES
  • Understanding of entropy in probability theory
  • Familiarity with probability density functions (pdf)
  • Knowledge of expected value and its properties
  • Basic calculus, particularly integration techniques
NEXT STEPS
  • Study the properties of entropy in information theory
  • Learn about transformations of random variables in probability
  • Explore the implications of the invariance of entropy under translations
  • Investigate applications of entropy in machine learning and data science
USEFUL FOR

Students of statistics, data scientists, and anyone interested in understanding the mathematical foundations of entropy and its applications in probability theory.

smk037
Messages
67
Reaction score
1

Homework Statement


The entropy of a random variable X is defined as -E[ln(fX(X))]
where fX(X) is the pdf of the random variable X

Show that the translation of X by a constant (e.g. adding a constant value to X) does not effect the entropy.


The Attempt at a Solution



To be honest, I have no idea where to start this. I tried to look at it as integral(x(ln(fX(X)), but it did not help. I don't understand how the translation by a constant would not effect the expected value, since I would expect it to change the value of the function, and therefore the natural log of the function, and therefore the expected value of the function.

Thanks in advance
 
Physics news on Phys.org
smk037 said:
I tried to look at it as integral(x(ln(fX(X))

It would be [tex]E[\ln f_X(X)]=\int_{-\infty}^{\infty} [\ln f_X(x)] f_X(x)\,dx[/tex]
 
Hint: Show and use

[tex]f_{X+c}(x) = f_{X}(x-c)[/tex]
 

Similar threads

  • · Replies 2 ·
Replies
2
Views
1K
  • · Replies 4 ·
Replies
4
Views
3K
  • · Replies 3 ·
Replies
3
Views
1K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 13 ·
Replies
13
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K