Hello,(adsbygoogle = window.adsbygoogle || []).push({});

I have a doubt about the distribution of random variables that maximize the differential entropy in a set of inequalities. It is well known that the Normal distribution maximizes the differential entropy. I have the following set of inequalities:

T1 < I(V;Y1|U)

T2 < I(U;Y2)

T3 < I(X1,X2;Y3|V)

T4 < I(X1,X2;Y3)

where, Y1=X1+N1, Y2=a*X1+N2, Y3=b*X1+X2+N3. N1,N2,N3 are Gaussian ~ N(0,1). The lower case a and b are positive real numbers a < b. U, V, X1 and X2 are random variables. I want to maximize that set of inequalities. I know the following:

(i) From T4, h(Y3) maximum is when Y3 is Gaussian then X1 and X2 are Gaussian.

(ii) From T2 we maximize it by having h(Y2) or h(a*X1+N2) maximum. From this by the Entropy Power Inequality (EPI) we bound -h(a*X1+N2|U) and have X1|U Gaussian.

(iii) From T1 we maximize it by having h(Y1|U) or h(X1+N1|U) maximum which we can do as -h(a*X1+N2|U) in the part ii can be bounded having Y1 Gaussian (satisfying the maximum entropy theorem).

The Question:

From T3, can I assume that jointly Gaussian distribution will maximize h(Y3|V) or h(b*X1+X2+N3) having the assumptions i,ii,iii ?

My aim is to show that jointly Gaussian distribution of U, V, X1 and X2 maximizes the set of inequalities. I hope anyone can help me out with this.

**Physics Forums - The Fusion of Science and Community**

Dismiss Notice

Join Physics Forums Today!

The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

# Maximum Entropy in Gaussian Setting

Loading...

Similar Threads - Maximum Entropy Gaussian | Date |
---|---|

I Why is the maximum likelihood estimation accurate? | Dec 20, 2017 |

I How do I normalise my data to a maximum of 100? | May 7, 2017 |

I Maximum likelihood w/ histogram, zero probability samples | Apr 25, 2017 |

I Why is the Maximum Likelihood Function a product? | Nov 3, 2016 |

Is Shannon Entropy subjective? | Jan 24, 2016 |

**Physics Forums - The Fusion of Science and Community**