Minimizing distance between two function

  • Thread starter Hejdun
  • Start date
  • Tags
    Function
In summary, the conversation is discussing the problem of approximating the function g(x) with function f(x) by selecting proper values for v, \sigma^2_1, and \sigma^2_2. The speaker has manually found approximate values for these variables but wants to solve the problem more formally. They consider minimizing the integral of the squared difference between g(x) and f(x), but this will not yield an analytical result. They also suggest minimizing the absolute difference between the two functions for all values of x. The speaker asks for suggestions on how to solve this problem analytically and if using numerical optimization, how to implement it in Mathematica. They share their code and report that, although they received error messages, the
  • #1
Hejdun
25
0
Consider the functions
[tex] g(x)=e^x/(1+e^x)^2 [/tex]
and
[tex]f(x) = vf_1(x) + (1-v)f_2(x) [/tex].
where
[tex] f_1(x)=\dfrac{1}{\sqrt{2\pi\sigma^2_1}}e^{-\dfrac{x^2}{2\sigma^2_1}},\,\,\, f_2(x)=\dfrac{1}{\sqrt{2\pi\sigma^2_2}}e^{-\dfrac{x^2}{2\sigma^2_2}}[/tex]
and
[tex] -\infty < x < \infty, \,\,\, 0 < v < 1, \,\,\, \sigma^2_1 > 0, \,\,\, \sigma^2_2 > 0. [/tex]

What I want to do basically is to approximate the function g(x) with f(x) by selecting proper values of [tex] v, \sigma^2_1, \sigma^2_2. [/tex] Just by inspection and manual iteration I concluded that something like [tex] v=0.3, \sigma^2_1=6.2, \sigma^2_2=1.95 [/tex] provides a fairly good approximation, but of course, I want to carry out the analysis more formally.

First I considered minimizing
[tex] \int_{-\infty}^{\infty} ( g(x) - (vf_1(x) + (1-v)f_2(x)) )^2 dx. [/tex]
The problems is that this integral will not yield an analytical result.

Second, what could also be of interest is to minimize abs(g(x)-f(x)) for all x so that we will at least know the maximum error that could occur.

My question is therefore twofold:
1) Do you have any suggestions about solving this problem analytically?
2) If resorting to numerical optimization when minimizing the integral above, how do I implement it in Mathematica? I tried the following code but it failed.

fz1 := (1/Sqrt[2*Pi*vz1])*Exp[-z^2/(2*vz1)]
fz2 := (1/Sqrt[2*Pi*vz2])*Exp[-z^2/(2*vz2)]
gz := Exp[z]/(1 + Exp[z])^2
uz := gz - (v*fz1 + (1 - v)*fz2)
Uz[vz1_?NumericQ, vz2_?NumericQ, v_?NumericQ] :=
NIntegrate[uz^2, {z, -Infinity, Infinity}]
NMinimize[{Uz, vz1 > 0 && vz2 > 0 && v > 0 && v < 1}, {vz1, vz2, v}]

All suggestions are most welcome!
 
Physics news on Phys.org
  • #2
Even if an analytic approach is possible (I don't know), a numeric solution is way easier.

How does Mathematica fail? Can you give some starting values?
You could try to remove the constraints on the variables. If the restrictions for v are really necessary, but problematic, you can use something like v=1/(1+e^w) with unbounded, free w.
 
  • #3
mfb said:
Even if an analytic approach is possible (I don't know), a numeric solution is way easier.

How does Mathematica fail? Can you give some starting values?
You could try to remove the constraints on the variables. If the restrictions for v are really necessary, but problematic, you can use something like v=1/(1+e^w) with unbounded, free w.

Mathematica reports:
NMinimize::nnum: The function value Uz is not a number at {v,vz1,vz2} = {0.652468,1.26614,1.36563}
but I don't see the error.

The restrictions are necessary, but I will try you suggestion!Edit:
Now I tried the following code below and although I got several error messages the result
{2.9583*10^-6, {v -> 0.107599, vz1 -> 1.53645, vz2 -> 4.66904}}
seems good when plotting the functions.

fz1 := (1/Sqrt[2*Pi*vz1])*Exp[-z^2/(2*vz1)]
fz2 := (1/Sqrt[2*Pi*vz2])*Exp[-z^2/(2*vz2)]
gz := Exp[z]/(1 + Exp[z])^2
uz[vz1_?NumericQ, vz2_?NumericQ, v_?NumericQ] :=
gz - ((1/(1 + Exp[v]))*fz1 + (1 - (1/(1 + Exp[v])))*fz2)
NMinimize[{NIntegrate[uz^2, {z, -Infinity, Infinity}], vz1 > 0 && vz2 > 0}, {v, vz1, vz2}]
 
Last edited:

1. How do you define the distance between two functions?

The distance between two functions can be defined in multiple ways, depending on the context and purpose. One common way is to use the L2 norm, also known as the Euclidean distance, which calculates the square root of the sum of squared differences between the two functions at various points. Alternatively, other distance metrics such as the L1 norm or the Chebyshev distance can also be used.

2. What is the significance of minimizing the distance between two functions?

Minimizing the distance between two functions is often used in data analysis and machine learning to find the best possible fit between a set of data points and a given function. This can help identify patterns, make predictions, and improve the overall accuracy of the model. It can also be used to compare two different functions and determine which one is a better fit for the data.

3. How can you minimize the distance between two functions?

There are various techniques that can be used to minimize the distance between two functions, such as gradient descent, least squares regression, and other optimization algorithms. The specific method used will depend on the problem at hand and the characteristics of the functions being compared. It is important to carefully select the appropriate method to ensure accurate and efficient results.

4. Can minimizing the distance between two functions lead to overfitting?

Yes, minimizing the distance between two functions can sometimes result in overfitting, which means that the model is too closely tailored to the training data and may not perform well on new data. This can happen if the chosen function is too complex or if the distance metric is not properly chosen. To avoid overfitting, it is important to regularly evaluate the model's performance on new data and make adjustments as needed.

5. Are there any limitations to minimizing the distance between two functions?

Minimizing the distance between two functions has many practical applications, but it also has some limitations. One limitation is that it assumes the functions being compared are continuous and differentiable, which may not always be the case. Additionally, the chosen distance metric may not always accurately reflect the similarity between the two functions, leading to potentially inaccurate results. It is important to carefully consider these limitations and select appropriate methods when using this approach.

Similar threads

A A discrete version of the normal distribution
    • Calculus
Replies
2
Views
1K
I Finding the derivative of a characteristic function
    • Calculus
Replies
5
Views
1K
A Multiplying divergent integrals using Hardy fields approach
    • Calculus
Replies
3
Views
1K
I I want to expand a Gaussian wavepacket in terms of sines
    • Quantum Physics
Replies
2
Views
521
B Calculating shaded area: I'm getting discrepancies between methods
    • Calculus
Replies
3
Views
321
A Summing over continuum and uncountable numerocities
    • Calculus
Replies
1
Views
927
MHB Finding the Derivative of the Inverse Function of a Cubic Polynomial
    • Calculus
Replies
5
Views
2K
MHB Acceleration @ $\dfrac{5\pi}{4}: \sqrt{2}$
    • Calculus
Replies
4
Views
2K
MHB 6.2.15 Find the domain of each function.
    • Calculus
Replies
3
Views
940
I Inequality with integral and max of derivative
    • Calculus
Replies
3
Views
1K

Hot Threads

Recent Insights

Back
Top