Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Minimizing distance between two function

  1. Apr 13, 2013 #1
    Consider the functions
    [tex] g(x)=e^x/(1+e^x)^2 [/tex]
    [tex]f(x) = vf_1(x) + (1-v)f_2(x) [/tex].
    [tex] f_1(x)=\dfrac{1}{\sqrt{2\pi\sigma^2_1}}e^{-\dfrac{x^2}{2\sigma^2_1}},\,\,\, f_2(x)=\dfrac{1}{\sqrt{2\pi\sigma^2_2}}e^{-\dfrac{x^2}{2\sigma^2_2}}[/tex]
    [tex] -\infty < x < \infty, \,\,\, 0 < v < 1, \,\,\, \sigma^2_1 > 0, \,\,\, \sigma^2_2 > 0. [/tex]

    What I want to do basically is to approximate the function g(x) with f(x) by selecting proper values of [tex] v, \sigma^2_1, \sigma^2_2. [/tex] Just by inspection and manual iteration I concluded that something like [tex] v=0.3, \sigma^2_1=6.2, \sigma^2_2=1.95 [/tex] provides a fairly good approximation, but of course, I want to carry out the analysis more formally.

    First I considered minimizing
    [tex] \int_{-\infty}^{\infty} ( g(x) - (vf_1(x) + (1-v)f_2(x)) )^2 dx. [/tex]
    The problems is that this integral will not yield an analytical result.

    Second, what could also be of interest is to minimize abs(g(x)-f(x)) for all x so that we will at least know the maximum error that could occur.

    My question is therefore twofold:
    1) Do you have any suggestions about solving this problem analytically?
    2) If resorting to numerical optimization when minimizing the integral above, how do I implement it in Mathematica? I tried the following code but it failed.

    fz1 := (1/Sqrt[2*Pi*vz1])*Exp[-z^2/(2*vz1)]
    fz2 := (1/Sqrt[2*Pi*vz2])*Exp[-z^2/(2*vz2)]
    gz := Exp[z]/(1 + Exp[z])^2
    uz := gz - (v*fz1 + (1 - v)*fz2)
    Uz[vz1_?NumericQ, vz2_?NumericQ, v_?NumericQ] :=
    NIntegrate[uz^2, {z, -Infinity, Infinity}]
    NMinimize[{Uz, vz1 > 0 && vz2 > 0 && v > 0 && v < 1}, {vz1, vz2, v}]

    All suggestions are most welcome!
  2. jcsd
  3. Apr 13, 2013 #2


    User Avatar
    2017 Award

    Staff: Mentor

    Even if an analytic approach is possible (I don't know), a numeric solution is way easier.

    How does Mathematica fail? Can you give some starting values?
    You could try to remove the constraints on the variables. If the restrictions for v are really necessary, but problematic, you can use something like v=1/(1+e^w) with unbounded, free w.
  4. Apr 13, 2013 #3
    Mathematica reports:
    NMinimize::nnum: The function value Uz is not a number at {v,vz1,vz2} = {0.652468,1.26614,1.36563}
    but I don't see the error.

    The restrictions are necessary, but I will try you suggestion!

    Now I tried the following code below and although I got several error messages the result
    {2.9583*10^-6, {v -> 0.107599, vz1 -> 1.53645, vz2 -> 4.66904}}
    seems good when plotting the functions.

    fz1 := (1/Sqrt[2*Pi*vz1])*Exp[-z^2/(2*vz1)]
    fz2 := (1/Sqrt[2*Pi*vz2])*Exp[-z^2/(2*vz2)]
    gz := Exp[z]/(1 + Exp[z])^2
    uz[vz1_?NumericQ, vz2_?NumericQ, v_?NumericQ] :=
    gz - ((1/(1 + Exp[v]))*fz1 + (1 - (1/(1 + Exp[v])))*fz2)
    NMinimize[{NIntegrate[uz^2, {z, -Infinity, Infinity}], vz1 > 0 && vz2 > 0}, {v, vz1, vz2}]
    Last edited: Apr 13, 2013
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook