# Minimizing distance between two function

1. Apr 13, 2013

### Hejdun

Consider the functions
$$g(x)=e^x/(1+e^x)^2$$
and
$$f(x) = vf_1(x) + (1-v)f_2(x)$$.
where
$$f_1(x)=\dfrac{1}{\sqrt{2\pi\sigma^2_1}}e^{-\dfrac{x^2}{2\sigma^2_1}},\,\,\, f_2(x)=\dfrac{1}{\sqrt{2\pi\sigma^2_2}}e^{-\dfrac{x^2}{2\sigma^2_2}}$$
and
$$-\infty < x < \infty, \,\,\, 0 < v < 1, \,\,\, \sigma^2_1 > 0, \,\,\, \sigma^2_2 > 0.$$

What I want to do basically is to approximate the function g(x) with f(x) by selecting proper values of $$v, \sigma^2_1, \sigma^2_2.$$ Just by inspection and manual iteration I concluded that something like $$v=0.3, \sigma^2_1=6.2, \sigma^2_2=1.95$$ provides a fairly good approximation, but of course, I want to carry out the analysis more formally.

First I considered minimizing
$$\int_{-\infty}^{\infty} ( g(x) - (vf_1(x) + (1-v)f_2(x)) )^2 dx.$$
The problems is that this integral will not yield an analytical result.

Second, what could also be of interest is to minimize abs(g(x)-f(x)) for all x so that we will at least know the maximum error that could occur.

My question is therefore twofold:
1) Do you have any suggestions about solving this problem analytically?
2) If resorting to numerical optimization when minimizing the integral above, how do I implement it in Mathematica? I tried the following code but it failed.

fz1 := (1/Sqrt[2*Pi*vz1])*Exp[-z^2/(2*vz1)]
fz2 := (1/Sqrt[2*Pi*vz2])*Exp[-z^2/(2*vz2)]
gz := Exp[z]/(1 + Exp[z])^2
uz := gz - (v*fz1 + (1 - v)*fz2)
Uz[vz1_?NumericQ, vz2_?NumericQ, v_?NumericQ] :=
NIntegrate[uz^2, {z, -Infinity, Infinity}]
NMinimize[{Uz, vz1 > 0 && vz2 > 0 && v > 0 && v < 1}, {vz1, vz2, v}]

All suggestions are most welcome!

2. Apr 13, 2013

### Staff: Mentor

Even if an analytic approach is possible (I don't know), a numeric solution is way easier.

How does Mathematica fail? Can you give some starting values?
You could try to remove the constraints on the variables. If the restrictions for v are really necessary, but problematic, you can use something like v=1/(1+e^w) with unbounded, free w.

3. Apr 13, 2013

### Hejdun

Mathematica reports:
NMinimize::nnum: The function value Uz is not a number at {v,vz1,vz2} = {0.652468,1.26614,1.36563}
but I don't see the error.

The restrictions are necessary, but I will try you suggestion!

Edit:
Now I tried the following code below and although I got several error messages the result
{2.9583*10^-6, {v -> 0.107599, vz1 -> 1.53645, vz2 -> 4.66904}}
seems good when plotting the functions.

fz1 := (1/Sqrt[2*Pi*vz1])*Exp[-z^2/(2*vz1)]
fz2 := (1/Sqrt[2*Pi*vz2])*Exp[-z^2/(2*vz2)]
gz := Exp[z]/(1 + Exp[z])^2
uz[vz1_?NumericQ, vz2_?NumericQ, v_?NumericQ] :=
gz - ((1/(1 + Exp[v]))*fz1 + (1 - (1/(1 + Exp[v])))*fz2)
NMinimize[{NIntegrate[uz^2, {z, -Infinity, Infinity}], vz1 > 0 && vz2 > 0}, {v, vz1, vz2}]

Last edited: Apr 13, 2013