Is hοh Monotonic If h Is Continuous?

[anachin6000]

So, it is known and easy to prove that if you have f : D -> G and g : G -> B then
-if both f and g have the same monotony => fοg is increasing
-if f and g have different monotony => fοg is decreasing
But the reciprocal of this is not always true (easy to prove with a contradicting example).
Though, it came to my mind that, if we have a function h : D -> D, a kind of of reciprocal might be valid for hοh.
I think that if hοh is monotonic it results that h is either decreasing or increasing, but I am not sure if it is true or not, neither how to prove or disprove it. This is actually my question, is it true and how you prove that?

 

[Samy_A]

So, it is known and easy to prove that if you have f : D -> G and g : G -> B then
-if both f and g have the same monotony => fοg is increasing
-if f and g have different monotony => fοg is decreasing
But the reciprocal of this is not always true (easy to prove with a contradicting example).
Though, it came to my mind that, if we have a function h : D -> D, a kind of of reciprocal might be valid for hοh.
I think that if hοh is monotonic it results that h is either decreasing or increasing, but I am not sure if it is true or not, neither how to prove or disprove it. This is actually my question, is it true and how you prove that?

Consider h, a real, not identically zero, smooth function with support in [3,4] and |h(x)|≤1 for all x ∈ ℝ.

 

[Samy_A]

This being said, if you add the condition that h is continuously differentiable, I think you can prove that if hοh is a monotone function while h is not, then hοh must be constant.

EDIT: continuity of h may well be sufficient.

EDIT2: no, not true, sorry.

Counter example:
Define h as follows:
for x≤0, f(x)=-x²
for x≥0, f(x)=g(x) where g is a non constant smooth function with support in [3,4] and range in [0,1].

Then, for x≤0, hoh(x)=h(-x²)=-x⁴.
For x>0, hoh(x)=h(h(x))=0.

 

[FactChecker]

Start with r₁<r₂ and track it through. I think that you will find that h monotonic => h(h) monotonic increasing.

But h(h) being monotonic implies nothing about monotonicity of h. Consider the function h(r) = r if r is rational; h(r)=-r otherwise.

 

[Samy_A]

Start with r₁<r₂ and track it through. I think that you will find that h monotonic => h(h) monotonic increasing.

But h(h) being monotonic implies nothing about monotonicity of h. Consider the function h(r) = r if r is rational; h(r)=-r otherwise.

Correct. If h is not continous, then h(h) being monotone implies nothing about monotonicity of h.

Although that was not what the OP asked for, I wondered about what happens if h is continuous.
As shown in posts #2 and #3, then also h(h) can be monotone without h being monotone. But the following seems true for continuous h:
On intervals where h(h) is strictly monotone, h will also be strictly monotone.

Proof (tentative):
Let's assume, wlog, that f=h(h) is a monotone increasing function.

1) Let's note that an injective continuous function on an interval is strictly monotone.

2) Let [a,b] be an interval in ℝ where f is strictly increasing.
For x,y ∈ [a,b], x≠y, h(x)≠h(y), for else f(x)=f(y), and f would be constant on [x,y]. It follows from 1) that h is strictly monotone on [a,b].

3) Now let's assume that we have an interval [a,b] where f is strictly increasing, and let's fix (again wlog) on the case where h is also strictly increasing on [a,b].
Take any c>b. If h(c)<h(b), the intermediate value theorem implies that there exist an x<b and an y>b such that h(x)=h(y). But then f would be constant on [x,y], contradicting the hypothesis that f is strictly increasing in [a,b].
Hence, ∀c>b, h(c)≥h(b).

4) Let [c,d] be another interval where f is strictly increasing, with c>b.
From 2) we know that h will be strictly monotone on [c,d]. From 3) we know that h(c)≥h(b), h(d)≥h(b).
If h were strictly decreasing on [c,d], h(c)>h(d)≥h(b). The intermediate value theorem then implies that ∃x ∈ [b,c[ satisfying h(x)=h(d).
That makes f constant on [x,d], a contradiction with f being strictly increasing on [c,d].
Hence h is also strictly increasing on [c,d].

Probably this proof can be made shorter.