Minimizing distance between two function

  • Context: Graduate 
  • Thread starter Thread starter Hejdun
  • Start date Start date
  • Tags Tags
    Function
Click For Summary
SUMMARY

This discussion focuses on approximating the function g(x) = e^x/(1+e^x)^2 using a mixture of two Gaussian functions f(x) = vf_1(x) + (1-v)f_2(x), where f_1 and f_2 are defined with variances σ²₁ and σ²₂. The user initially attempted to minimize the integral of the squared difference between g(x) and f(x) but encountered difficulties with analytical solutions. Suggestions included using numerical optimization in Mathematica, where the user faced issues with constraints and function evaluations. Ultimately, a modified approach yielded promising results with values v = 0.107599, σ²₁ = 1.53645, and σ²₂ = 4.66904.

PREREQUISITES
  • Understanding of Gaussian functions and their properties
  • Familiarity with numerical optimization techniques
  • Experience using Mathematica for mathematical computations
  • Knowledge of integral calculus and error minimization methods
NEXT STEPS
  • Explore advanced numerical optimization techniques in Mathematica
  • Learn about the properties of Gaussian mixtures and their applications
  • Investigate alternative methods for error minimization in function approximation
  • Study the implementation of constraints in optimization problems in Mathematica
USEFUL FOR

Mathematicians, data scientists, and researchers involved in function approximation, numerical analysis, and optimization using Mathematica.

Hejdun
Messages
25
Reaction score
0
Consider the functions
g(x)=e^x/(1+e^x)^2
and
f(x) = vf_1(x) + (1-v)f_2(x).
where
f_1(x)=\dfrac{1}{\sqrt{2\pi\sigma^2_1}}e^{-\dfrac{x^2}{2\sigma^2_1}},\,\,\, f_2(x)=\dfrac{1}{\sqrt{2\pi\sigma^2_2}}e^{-\dfrac{x^2}{2\sigma^2_2}}
and
-\infty < x < \infty, \,\,\, 0 < v < 1, \,\,\, \sigma^2_1 > 0, \,\,\, \sigma^2_2 > 0.

What I want to do basically is to approximate the function g(x) with f(x) by selecting proper values of v, \sigma^2_1, \sigma^2_2. Just by inspection and manual iteration I concluded that something like v=0.3, \sigma^2_1=6.2, \sigma^2_2=1.95 provides a fairly good approximation, but of course, I want to carry out the analysis more formally.

First I considered minimizing
\int_{-\infty}^{\infty} ( g(x) - (vf_1(x) + (1-v)f_2(x)) )^2 dx.
The problems is that this integral will not yield an analytical result.

Second, what could also be of interest is to minimize abs(g(x)-f(x)) for all x so that we will at least know the maximum error that could occur.

My question is therefore twofold:
1) Do you have any suggestions about solving this problem analytically?
2) If resorting to numerical optimization when minimizing the integral above, how do I implement it in Mathematica? I tried the following code but it failed.

fz1 := (1/Sqrt[2*Pi*vz1])*Exp[-z^2/(2*vz1)]
fz2 := (1/Sqrt[2*Pi*vz2])*Exp[-z^2/(2*vz2)]
gz := Exp[z]/(1 + Exp[z])^2
uz := gz - (v*fz1 + (1 - v)*fz2)
Uz[vz1_?NumericQ, vz2_?NumericQ, v_?NumericQ] :=
NIntegrate[uz^2, {z, -Infinity, Infinity}]
NMinimize[{Uz, vz1 > 0 && vz2 > 0 && v > 0 && v < 1}, {vz1, vz2, v}]

All suggestions are most welcome!
 
Physics news on Phys.org
Even if an analytic approach is possible (I don't know), a numeric solution is way easier.

How does Mathematica fail? Can you give some starting values?
You could try to remove the constraints on the variables. If the restrictions for v are really necessary, but problematic, you can use something like v=1/(1+e^w) with unbounded, free w.
 
mfb said:
Even if an analytic approach is possible (I don't know), a numeric solution is way easier.

How does Mathematica fail? Can you give some starting values?
You could try to remove the constraints on the variables. If the restrictions for v are really necessary, but problematic, you can use something like v=1/(1+e^w) with unbounded, free w.

Mathematica reports:
NMinimize::nnum: The function value Uz is not a number at {v,vz1,vz2} = {0.652468,1.26614,1.36563}
but I don't see the error.

The restrictions are necessary, but I will try you suggestion!Edit:
Now I tried the following code below and although I got several error messages the result
{2.9583*10^-6, {v -> 0.107599, vz1 -> 1.53645, vz2 -> 4.66904}}
seems good when plotting the functions.

fz1 := (1/Sqrt[2*Pi*vz1])*Exp[-z^2/(2*vz1)]
fz2 := (1/Sqrt[2*Pi*vz2])*Exp[-z^2/(2*vz2)]
gz := Exp[z]/(1 + Exp[z])^2
uz[vz1_?NumericQ, vz2_?NumericQ, v_?NumericQ] :=
gz - ((1/(1 + Exp[v]))*fz1 + (1 - (1/(1 + Exp[v])))*fz2)
NMinimize[{NIntegrate[uz^2, {z, -Infinity, Infinity}], vz1 > 0 && vz2 > 0}, {v, vz1, vz2}]
 
Last edited:

Similar threads

  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 5 ·
Replies
5
Views
3K
Replies
5
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 3 ·
Replies
3
Views
1K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 3 ·
Replies
3
Views
1K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K