# Mutual information of a noisy function

• A
So let suppose I have a random variable Y that is defined as follows:
$$Y=\alpha x+ \mathcal{N}(\mu,\sigma) \text{ where } x \text{ } \epsilon \text{ }\mathbb {R}$$
and
$$\mathcal{N}(\mu,\sigma)\text{ is a i.i.d. normally distributed random variable with mean }\mu\text{ and variance }\sigma$$
So I know Y is a random variable, but x is not, however is seems to me that their is a probability measure
$$P(x,Y)\text{ } \epsilon\text{ }[0,1]$$
Therefore, the mutual information is
$$I(x;Y)=H(x)+H(x|Y)=H(Y)+H(Y|x)$$
However it seems that
$$H(x)=-\int_{x}p(x)lnp(x)dx$$
is not defined because x is not a random variable so is their really any mutual information between x and Y? is p(x,Y) an actual joint probability distribution? Any insight would be awesome thanks.

Last edited:

So let suppose I have a random variable Y that is defined as follows:
$$Y=\alpha x+ \mathcal{N}(\mu,\sigma) \text{ where } x \text{ } \epsilon \text{ }\mathbb {R}$$
and
$$\mathcal{N}(\mu,\sigma)\text{ is a i.i.d. normally distributed random variable with mean }\mu\text{ and variance }\sigma$$
So I know Y is a random variable, but x is not, however is seems to me that their is a probability measure
$$P(x,Y)\text{ } \epsilon\text{ }[0,1]$$
Therefore, the mutual information is
$$I(x;Y)=H(x)+H(x|Y)=H(Y)+H(Y|x)$$
However it seems that
$$H(x)=-\int_{x}p(x)lnp(x)dx$$
is not defined because x is not a random variable so is their really any mutual information between x and Y? is p(x,Y) an actual joint probability distribution? Any insight would be awesome thanks.

For some reason it is not showing the text between equations. Here it is in full

Do you know the value of ##x## (is ##x## a constant?), or you want to estimate it from ##Y##?

Do you know the value of ##x## (is ##x## a constant?), or you want to estimate it from ##Y##?
x is not constant it is the independent variable in this case. Without the noise term this would just be a simple linear function $$Y=\alpha x$$

Fine, but what it does represent in reality? When you observe ##Y##, do you know what is ##x##?

Fine, but what it does represent in reality? When you observe ##Y##, do you know what is ##x##?
Yes. Ultimately this is a regression problem. I have observed values x and observed values Y and want to know the mutual information between them with the knowledge that Y is a linear function of x. For example lets say I send a signal x which is received by a receiver that transforms x by multiplying it by a constant, but their is some unknown source of noise added between reception and transmission. I want to know how much of Y can be explained by x.

I suspect that ##x## is unknown at the time of observing ##Y##, which makes it random. Say you have two signals ##x_1## and ##x_2##, and you transmitted ##x_1##. You receive ##y=\alpha\,x_1+n##, where ##n## is the noise. ##x_1## is a number at the transmitter, but at the receiver it is random (it could be ##x_1## or ##x_2##) because it is unknown.

FactChecker