Understanding Jensen's Inequality: Equal Conditions

[Steve Zissou]

Hello all,

Jensen's inequality says that for some random x,
f(E[x])≤E[f(x)]
if f(x) is convex.
Is there any generality that might help specify under what circumstances this inequality is...equal?

Thanks

 

[fresh_42]

Hello all,

Jensen's inequality says that for some random x,
f(E[x])≤E[f(x)]
if f(x) is convex.
Is there any generality that might help specify under what circumstances this inequality is...equal?

Thanks

If a real function $f : \mathbb{R} \longrightarrow \mathbb{R}$ satisfies $f \left( \int_{0}^{1} \mu \, d x \right) \leq \int_{0}^{1} (f \circ \mu) \, d x$ for all bounded, Lebesgue measurable functions $\mu \colon [0,1]\to \mathbb {R}$ then $f$ is convex. So if equality holds, $f$ is also concave (application on $-f$). The only functions which are both are linear functions, which in return satisfy the equation.

 

[Steve Zissou]

Wow, thanks for the quick reply fresh 42!
However I'm not entirely sure I understand your math notation. In what sense do we mean f(x) can only be "linear?"
Thanks again for helping me.

 

[fresh_42]

For linear functions the equality holds: $f(E(x)) = f(\int \mu dx) = \int (f \circ \mu ) dx = E(f(x)$.

So the question is: Can we conclude, that if the equality holds, then $f$ has to be linear?

So let us assume the equality. The theorem I quoted says, that under certain conditions, we can conclude that the inequality implies convexity. Then the inequality in the other direction (which we have by the assumption of equality) implies convexity of $-f$. But if $f$ and $-f$ both are convex, then $f$ is linear. So all examples of $f$ which fulfill this additional condition on functions $\mu$ that also fulfill the equality have to be necessarily linear. For expectation values we have a the probability density function which is bounded and Lebesgue measurable, so the additional condition for $\mu$ holds.

This means in your example of Jensen's inequality, equality holds if and only if $f$ is linear.

The in $E(.)$ hidden condition "for all probability distributions" is quite strong.

 

[Steve Zissou]

Ok fresh_42 let me see if I understand:
Convex case:
f(E[x])≤E[f(x)]
Concave case:
f(E[x])≥E[f(x)]
...so we're saying the only way to get the equality to hold is that both inequalities are true simultaneously. That means if the function f(x) is both convex and concave, which means it is a linear function.
Do I have it right?
Again, thanks a million!

 

[fresh_42]

Ok fresh_42 let me see if I understand:
Convex case:
f(E[x])≤E[f(x)]
Concave case:
f(E[x])≥E[f(x)]
...so we're saying the only way to get the equality to hold is that both inequalities are true simultaneously. That means if the function f(x) is both convex and concave, which means it is a linear function.
Do I have it right?
Again, thanks a million!

Yes. That's always true: $A = B \Longleftrightarrow A \leq B \textrm{ and } A \geq B \Longleftrightarrow A \leq B \textrm{ and } -A \leq -B$.
So if $f(E(x))=E(f(x))$ for all possible probability distributions, we can apply the reversal of Jensen's inequality and get convexity for $f$ and $-f$. $-f$ convex means $f$ is concave (and vice versa). So $f$ is both, which is only possible, if $f$ is linear.

You can also get the result for finite dimensions and discrete random variables from:
$f(\sum x_i \lambda_i) = \sum f(x_i) \lambda_i \;\; \forall_{x_i\, , \,\lambda_i} \,:\,\sum \lambda_i = 1 \Longrightarrow f(\lambda_1 x_1 + (1-\lambda_1)x_2)=\lambda_1 f(x_1) + (1-\lambda_1)f(x_2)$ and a little algebra with the condition of arbitrary $x_i$ and arbitrary probabilities $\lambda_i$. As i said: for all probability distributions is pretty strong.